• Author(s): Shaojie Ma, Yawei Luo, Yi Yang

This paper introduces the Mesh-adsorbed Gaussian Splatting (MaGS) method, a novel approach designed to address the dual challenges of 3D reconstruction and simulation. Traditional methods often struggle to balance the flexibility required for diverse scene reconstruction with the structured representation needed for effective motion simulation. MaGS resolves this dilemma by constraining 3D Gaussians to hover on the mesh surface, creating a hybrid mesh-Gaussian 3D representation. This representation combines the rendering flexibility of 3D Gaussians with the spatial coherence of meshes.

A key innovation in MaGS is the introduction of a learnable Relative Deformation Field (RDF), which models the relative displacement between the mesh and 3D Gaussians. This extends traditional mesh-driven deformation paradigms that rely solely on the as-rigid-as-possible (ARAP) prior, allowing for more precise motion capture of each 3D Gaussian. By jointly optimizing meshes, 3D Gaussians, and RDF, MaGS achieves both high rendering accuracy and realistic deformation. Extensive experiments conducted on the D-NeRF and NeRF-DS datasets demonstrate that MaGS outperforms current methods in both reconstruction and simulation. The results show significant improvements in rendering accuracy and the reduction of artifacts, highlighting the method’s robustness and effectiveness.

The paper also discusses the potential applications of MaGS in various fields, such as virtual reality, augmented reality, and digital content creation. By providing a unified framework for 3D reconstruction and simulation, MaGS opens up new possibilities for creating realistic and interactive digital experiences. The authors suggest directions for future research to further enhance the capabilities of this method. The Mesh-adsorbed Gaussian Splatting method represents a significant advancement in the field of 3D reconstruction and simulation. The proposed approach effectively addresses the limitations of traditional methods, offering a practical solution for accurately capturing and representing dynamic objects in various applications.