3D Scene Change Understanding

Lead researcher

Chamuditha Galappaththige

Project based at

Lead Partner Organisation

Detecting changes in a 3D scene is a fundamental challenge in robotic inspection.  This project aims to develop a robust system for understanding changes in complex 3D scenes using only 2D RGB images.  We will focus on identifying structural or appearance changes that have occurred to an asset, distinguishing these meaningful changes from benign environmental variations due to differences in lighting or reflections. By enabling robots to accurately capture, reason about, and update representations of evolving environments, this research aims to improve the reliability and effectiveness of robotic asset inspection systems. 

Research Activities: 

The fundamental objective of this project is to construct novel techniques tailored for localising and understanding the changes in a complex 3D scene. We will: 

• localise changes that have occurred in a scene, using streams of data with no constraint on trajectories or viewpoints between the scene observations. 

• build a representation of change in the 3D scene and localise the changes incrementally in an online manner as the robot travels through the scene. 

• update the representation of the scene over time efficiently, leveraging the understanding of what has changed from the previous visit. 

• interpret and reason about the nature of localised changes between distinct instances of the 3D scene. 

Research Outcomes/ Current Findings: 

An approach to localising changes in a 3D scene that uses an understanding of the structure of the scene to build an explicit 3D representation of change. We leverage foundation models and image-level metrics to build initial estimates of changes in a scene, and then use the multiple viewpoints collected by the robot during inspection to filter out false detections and create more reliable and robust change predictions. Our approach achieves state-of-the-art performance for indoor and outdoor scenes, and under changes in lighting, including shadows and reflections. This work has been accepted to appear at the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2025. 

 Expected Impact: 

• Automated flagging of potential degradations or failures in assets with robot inspection 

• Fine-grained localisation of these degradations with segmentation masks 

• Detection of such failures over long time horizons (multiple time instances of inspection by the robot) 

• Reasoning about the types, temporal dynamics, and nature of failures over long time horizons