Automated mural restoration via semi supervised segmentation and prompt guided diffusion inpainting

作者：Zishan Xu, Shaokai Hua, Zilong Wang, Wei Chen, Jueting Liu, Tingting Xu, Zehua Wang · 发表于：npj Heritage Science · 年份：2025 · DOI：10.1038/s40494-025-02047-5 · 被引用次数：4 · 研究领域：Generative Adversarial Networks and Image Synthesis、Computer Graphics and Visualization Techniques、Image Processing and 3D Reconstruction

This study aims to address two major challenges in the digital restoration of murals: defect segmentation and large-scale defect repair. We propose a novel solution, combining the SAM-Adapter segmentation method with semi-supervised training and a Text-to-Inpainting strategy. In the aspect of defect segmentation, by introducing thresholding and semi-supervised training, we effectively enhanced the precision and applicability in handling sparse datasets. For large-scale defect repair, this study innovatively applies BPO (Black-box Prompt Optimization) technology, combined with GPT4V (ision) and Stable Diffusion, to achieve precise reconstruction of murals’ complex textures and structures through text-guided methods. Through extensive experimental validation, our method has demonstrated significant effectiveness in defect segmentation and large-scale repair, providing new perspectives and tools for the digital preservation of murals, and is of great significance for the protection of cultural heritage.