Hybrid Mesh-Neural Representation for 3D Transparent Object Reconstruction

作者：Jiamin Xu, Zihan Zhu, Hujun Bao, Weiwei Xu · 发表于：Computational Visual Media · 年份：2025 · DOI：10.26599/cvm.2025.9450328 · 被引用次数：10 · 研究领域：3D Shape Modeling and Analysis、Computer Graphics and Visualization Techniques、3D Surveying and Cultural Heritage

In this study, we propose a novel method to reconstruct the 3D shapes of transparent objects using images captured by handheld cameras under natural lighting conditions. It combines the advantages of an explicit mesh and multi-layer perceptron (MLP) network as a hybrid representation to simplify the capture settings used in recent studies. After obtaining an initial shape through multi-view silhouettes, we introduced surface-based local MLPs to encode the vertex displacement field (VDF) for reconstructing surface details. The design of local MLPs allowed representation of the VDF in a piecewise manner using two-layer MLP networks to support the optimization algorithm. Defining local MLPs on the surface instead of on the volume also reduced the search space. Such a hybrid representation enabled us to relax the ray-pixel correspondences that represent the light path constraint to our designed ray-cell correspondences, which significantly simplified the implementation of a single-image-based environment-matting algorithm. We evaluated our representation and reconstruction algorithm on several transparent objects based on ground truth models. The experimental results show that our method produces high-quality reconstructions that are superior to those of state-of-the-art methods using a simplified data-acquisition setup.