RGB-D Local Implicit Function for Depth Completion of Transparent Objects

作者：Luyang Zhu, Arsalan Mousavian, Xiang Yu, Hammad Mazhar, Jozef van Eenbergen, Shoubhik Debnath, Dieter Fox · 年份：2021 · DOI：10.1109/cvpr46437.2021.00462 · 被引用次数：95 · 研究领域：Advanced Vision and Imaging、Optical measurement and interference techniques、Robotics and Sensor-Based Localization

Majority of the perception methods in robotics require depth information provided by RGB-D cameras. However, standard 3D sensors fail to capture depth of transparent objects due to refraction and absorption of light. In this paper, we introduce a new approach for depth completion of transparent objects from a single RGB-D image. Key to our approach is a local implicit neural representation built on ray-voxel pairs that allows our method to generalize to unseen objects and achieve fast inference speed. Based on this representation, we present a novel frame-work that can complete missing depth given noisy RGB-D input. We further improve the depth estimation iteratively using a self-correcting refinement model. To train the whole pipeline, we build a large scale synthetic dataset with transparent objects. Experiments demonstrate that our method performs significantly better than the current state-of-the-art methods on both synthetic and real world data. In addition, our approach improves the inference speed by a factor of 20 compared to the previous best method, ClearGrasp [43]. Code will be released at https://research.nvidia.com/publication/2021-03_RGB-D-Local-Implicit.