Robots with memory: AI system tells glass from plastic cups

South Korean researchers have developed an AI technology that allows robots to remember the location and condition of objects they have previously seen and retrieve that information on their own when needed. The system gives robots a human-like ability to recall and verify their surroundings, and is expected to improve the task performance of home-care and service robots.

The Electronics and Telecommunications Research Institute (ETRI) announced Monday that it had jointly developed the technology with the Gwangju Institute of Science and Technology (GIST). The system, called SceneQueryPlanner, enables a robot to store a memory of its environment and actively search that memory to plan its actions.

Recent advances have allowed robots to understand human commands and carry out multiple tasks using large language models. In real-world settings, however, robots must identify not only where objects are but also what they are made of and what condition they are in — a far more demanding challenge.

To address this, the research team developed a technique that stores information about rooms, furniture, and objects — including their positions, materials, and states — as a three-dimensional scene graph. The system works like a spatial map of the environment, updating itself each time the robot observes something new.

The key innovation is that the robot can retrieve what it needs from stored memory at the right moment, without being prompted.

SceneQueryPlanner combines two complementary processes: "environment grounding," in which the robot physically moves through its surroundings to gather information, and "memory grounding," in which it selectively queries its stored memory for only the relevant details.

Together, these allow the robot to judge not only what task it needs to perform but also what additional information it should seek out before acting.

When given a command to sort cups, for example, the robot examines the cups on a table, queries its stored memory for material information, and distinguishes glass cups from plastic ones before moving each to its designated location.

To validate the technology, the team also developed AttPlan-Bench, a benchmark that evaluates performance across 10 types of complex tasks in simulated home environments.

In tests using Meta's AI model Llama 3.1, the task success rate rose from 46.8 percent to 69.6 percent, a gain of 22.8 percentage points. Tests on Alibaba's Qwen 2.5 showed a similar improvement, with the success rate climbing from 44.0 percent to 68.0 percent, up 24.0 percentage points.

Experiments using OpenAI's GPT-4o also outperformed existing approaches, with the success rate for complex tasks requiring identification of an object's material and condition rising by 17.2 percentage points.

The researchers expect the technology to be applied to physical AI systems — robots that perceive, reason, and act autonomously in complex real-world settings — including home-care and service robots.

"The key is that the robot systematically stores what it observes about its environment and retrieves that information on its own when needed," said Choi Jae-woo, a senior researcher at ETRI. "We will continue refining the technology so that robots can act reliably in real-world settings by drawing on their own experience and memory."

Kim Eui-hwan, a professor at GIST, said the work goes beyond simply storing a robot's memory. "The significance lies in advancing it into an active memory system that the robot can query and draw from on its own," he said.

The findings were presented Sunday at EMNLP 2026 Findings, an international conference on natural language processing.