Project

Underwater 3D Perception for Robotic Manipulation

Lead researcher

Yiran Wang

Project based at

Lead Partner Organisation

Project Description

An advanced underwater ROV system is being developed to enable robotic arm control for underwater tasks such as object grasping. The system uses stereo vision to collect environmental data, dynamically reconstruct 3D scenes, and identify target objects. Based on the reconstructed scene, it determines optimal grasping points for precise manipulation.

Underwater vision models face challenges such as light attenuation, scattering, and turbidity, which degrade performance. Real-time control constraints also limit computational resources. This project aims to improve the robustness of underwater visual perception by enhancing 3D scene reconstruction and object recognition, contributing to more effective underwater robotic manipulation for exploration, maintenance, and research.

Research Activities

Our project focuses on acquiring underwater 3D scene data using stereo vision and other sensors and predicting optimal grasping points for robotic manipulation. Key activities include:

  1. Robust Stereo Depth Estimation Framework: Designing a stereo vision-based depth estimation pipeline optimized for underwater conditions using deep learning and adaptive filtering.
  2. Real-Time 3D Reconstruction for Underwater Manipulation: Developing a low-latency 3D reconstruction method to enable precise perception and interaction with submerged objects.
  3. Curate and Collect Underwater Stereo Datasets: Gathering high-quality stereo image datasets from diverse underwater environments for training and validation.
  4. Real-World Testing with Reach Robotics: Deploying and evaluating the system in actual underwater scenarios to assess accuracy, robustness, and usability.
  5. Conduct Real-World Testing and Validation with Reach Robotics: Deploy and evaluate the system in real underwater scenarios with Reach Robotics, assessing its performance in terms of accuracy, robustness, and usability for practical application in marine robotics.

Expected Impact

Solving underwater stereo perception for manipulation would have a significant impact on various aspects of robotics, particularly in environments where precise control and real-time decision-making are critical. 

The stereovison pipeline we are working on will enable accurate 3D reconstruction of underwater environments, which is crucial for identifying and localising the objects of interest. Precise metric depth estimation improves grasp planning, ensuring that robotic arms can interact effectively with underwater objects. Our system will provide real-time feedback for correcting positioning errors in grasping and manipulation. 

Associated Researchers