Can robots truly understand their surroundings?

A robot can map an entire building. But does it actually understand what’s inside it?

Imagine asking a robot to find your coffee mug.

A geometric map might tell it where the walls, doors, and furniture are. But finding that mug requires something more: understanding objects, their relationships, and the context in which they exist.

That’s the difference between mapping the world and making sense of it.

This is where 3D Scene Graphs (3DSGs) come into play. By connecting geometry, semantics, and spatial relationships, they provide robots with structured representations of their surroundings that can support reasoning, planning, and interaction.

But how far has this technology progressed, and what challenges remain before robots can reliably use it in the real world?

These questions are at the heart of our new survey published in IEEE Transactions on Robotics, resulting from a collaboration between researchers in Finland, Spain, and Switzerland.

Drawing on more than 150 scientific works, the paper provides a comprehensive overview of the rapidly evolving field of 3D scene graphs, from traditional SLAM approaches to recent advances involving foundation models and language-driven robotics. It also identifies key research challenges and discusses promising directions toward more capable and autonomous robotic systems.

Moving from geometric maps to meaningful representations of the world is an important step toward more intelligent robotics. This survey offers a perspective on where the field stands today and what is needed to move forward.

The article, 3-D Scene Graphs in Robotics: A Unified Representation Bridging Geometry, Semantics, and Action, is now available as open access in IEEE Transactions on Robotics.

Authored by Iacopo Maria Catalano, Carlos Cueto Zumaya, Julio Placed, Javier Civera, Wallace Moreira Bessa, and Jorge Peña Queralta, the publication brings together expertise from the University of Turku, the Aragón Institute of Technology, the University of Zaragoza, and the Zurich University of Applied Sciences.

Interested in how robots can move beyond mapping toward understanding their surroundings? Read the full article here.