GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting

作者：Joanna Waczy'nska, P. Borycki, S. Tadeja, Jacek Tabor, P. Spurek · 发表于：arXiv.org · 年份：2024 · DOI：10.48550/arXiv.2402.01459 · 被引用次数：93 · 研究领域：Computer Science

Gaussian Splatting (GS) is a novel, state-of-the-art technique for rendering points in a 3D scene by approximating their contribution to image pixels through Gaussian distributions, warranting fast training and real-time rendering. The main drawback of GS is the absence of a well-defined approach for its conditioning due to the necessity of conditioning several hundred thousand Gaussian components. To solve this, we introduce the Gaussian Mesh Splatting (GaMeS) model, which allows modification of Gaussian components in a similar way as meshes. We parameterize each Gaussian component by the vertices of the mesh face. Furthermore, our model needs mesh initialization on input or estimated mesh during training. We also define Gaussian splats solely based on their location on the mesh, allowing for automatic adjustments in position, scale, and rotation during animation. As a result, we obtain a real-time rendering of editable GS.