Interactivist Summer Institute

July 22 - 26, 2003


Perceptual Anchoring: A Key Problem for Cognitive Robotics

Silvia Coradeschi and Alessandro Saffiotti(*)

Center for Applied Autonomous Sensor Systems
Orebro University, S-70182 Orebro, Sweden

phone: +46-19-303794
fax: +46-19-303463

(*) Contact person

MAJOR THEME: Robotic and computational models of interaction and cognition


"You are at a friend's house and your host asks you to go to the cellar and fetch the bottle of Barolo wine stored at the top of the green rack. You go down to the cellar, look around in order to identify the green rack, and visually scan the top of the rack to find a bottle-like object with a Barolo label. When you see it, you reach out your hand to grasp it, and bring it upstairs."

This vignette illustrates a mechanism that we constantly employ in our everyday life: the use of words to refer to objects in the physical world, and to communicate a specific reference to another agent. This example presents one peculiar instance of this mechanism, one in which the first agent ``knows'' which object he wants but cannot see it, while the second agent only has an incomplete description of the object but can see it. Put crudely, the two agents that embody two different types of processes: one that reasons about abstract representations of objects, and one that has access to perceptual data. One of the prerequisites for the successful cooperation between these processes is that they agree about the objects they talk about, that is, that there is a correspondence between the abstract representations and the perceptual data which refer to the same physical objects. In other words, there must be a correspondence between the names of things and their perceptual image. We call "anchoring" the process of establishing and maintaining this correspondence.

Not unlike our example, autonomous systems embedded in the physical world typically incorporate two different types of processes: high-level cognitive processes, that perform abstract reasoning and generate plans for actions; and sensory-motoric processes, that observe the physical world and execute actions in it. The crucial observation here is that these processes have different ways of referring to the same physical objects in the environment. Cognitive processes typically (although not necessarily) use symbols to denote objects, while sensory-motoric processes typically operate from sensor data that originate from observing these objects. If the overall system has to successfully perform its task, it needs to make sure that these processes "talk about" the same physical objects: that is, it has to perform anchoring.

Suppose for concreteness that a robot's planner has generated the action 'PickUp(bottle-22)', where the symbol 'bottle-22' denotes an object known by the planner to be a bottle and to contain Barolo wine. In order to execute this action, the robot might start a 'PickUp' operator implemented by visual-servoing the robot's arm with respect to a given region in the camera input. But which region?

Intuitively, the robot must make sure that the region used for controlling the arm is precisely the one generated by observing the object that the planner calls 'bottle-22'. That is, the robot must anchor the symbol 'bottle-22' to the right sensor data. How the "right" data can be identified from the sensor stream is part of the anchoring problem.

The above considerations suggest that anchoring must necessarily take place in any robotic system that comprises a symbolic reasoning component. Until recently, however, the anchoring problem was typically solved on a system-by-system basis, often using techniques from the pattern recognition or object tracking domains, and the solution was hidden in the code. The situation is now changing, and the field of autonomous robots is showing a tendency to engage in the study of the anchoring problem per se -- see for instance [1,2]. This study would allow us to develop a set of common principles and techniques for anchoring that can be easily applied across different systems and domains. From a more general perspective, a study of the anchoring problem would increase our understanding of the delicate issue of integration between symbolic reasoning and physical embodiment. The papers in this Special Issue discuss possible solutions to the anchoring problem in its different facets and in different application domains.

Having recognized the existence of the anchoring problem, the next step is to define it in a more precise way. This is an obvious prerequisite to being able to devise general theories and techniques to address it. We give the following definition.

We call "anchoring" the process of creating and maintaining the correspondence between symbols and sensor data that refer to the same physical objects. The "anchoring problem" is the problem of how to perform anchoring in an artificial system.

Anchoring as defined here is a difficult problem. It involves concepts which have interested philosophers for centuries and are still far from being fully understood. Nonetheless, we have to provide practical solutions to the anchoring problem if we want to build robotic systems that include a symbolic component. In our work, we have developed a formal framework to solve the anchoring problem in artificial robotic systems, reported in [3,4]. In the longer term, our research program on anchoring is meant to lead to a deeper understanding of the anchoring problem, together with general practical solutions that can be re-used in different systems and domains.


[1] S. Coradeschi and A. Saffiotti, editors. Anchoring symbols to sensor data in single and multiple robot systems. Proceedings of the AAAI Fall Symposium. Technical Report FS-01-01. AAAI Press, Menlo Park, CA, 2001. ISBN 1-57735-135-5.

[2] S. Coradeschi and A. Saffiotti, editors. Special Issue on Perceptual Anchoring, Robotics and Autonomous Systems 43, 2003. (In press)

[3] S. Coradeschi and A. Saffiotti. An Introduction to the Anchoring Problem. Robotics and Autonomous Systems 43, 2003. (In press)

[4] S. Coradeschi and A. Saffiotti. Anchoring Symbols to Sensor Data: preliminary report. Proc. of the 17th AAAI Conf, pp. 129-135. Austin, Texas, July 2000.


ISI 2003 Home Page