3D Reconstruction of Reflective and Textureless Regions

Lead researcher

James Gray

Project based at

Lead Partner Organisation

Existing 3D reconstruction approaches have difficulties accurately reconstructing textureless regions due to their low information content. Furthermore, reflective regions can provide false information which can confuse 3D reconstruction approaches. Our goal is to develop a 3D reconstruction approach that can appropriately handle textureless and reflective regions. 

Research Activities: 

Existing 3D reconstruction approaches fail to produce accurate and dense depth maps in textureless and reflective regions. Accordingly, we are developing a 3D reconstruction approach which will appropriately model textureless and reflective regions. We intend to leverage the ability of radiance field methodologies to render the appearance of reflections alongside other methods such as epipolar plane images and monocular depth estimation to identify reflections and determine geometry from the images and/or from the radiance field.

Expected Impact: 

This research has direct applications for asset inspection. Trendspek offers to perform this task using drones to capture the images and a 3D reconstruction pipeline to produce a digital representation of the asset. This has significant benefits over performing the asset inspection manually. Using drones to perform data capture is quicker, cheaper and less disruptive than capturing the data manually.  

However, existing 3D reconstruction approaches do not properly handle reflective and textureless regions. Accordingly, Trendspek do not offer their services for buildings which are too reflective. Since many buildings have highly reflective glass facades, this represents a significant opportunity for improvement. A 3D reconstruction approach that can appropriately model textureless and reflective regions has the potential to expand the range of assets that can be inspected using a drone-based reconstruction algorithm. Our approach can also allow previously generated models to be updated with improved fidelity, improving performance in scenarios where reflective and textureless regions are less prominent but still represent a challenge for reconstruction. Errors in even small regions of a model can have a significant impact on the utility and perception of fidelity in 3D reconstruction.  

More broadly, this work can find application where 3D reconstruction plays an important role, for example in augmented and virtual reality, robotic navigation, and manufacturing and quality control, particularly where reflections of lack of textural information presently limit performance or prevent deployment altogether.