Word2Scene: Efficient remote sensing image scene generation with only one word via hybrid intelligence and low-rank representation

作者：Jiaxin Ren, Wanzeng Liu, Jun Chen, Shunxi Yin, Yuan Tao · 发表于：ISPRS Journal of Photogrammetry and Remote Sensing · 年份：2024 · DOI：10.1016/j.isprsjprs.2024.11.002 · 被引用次数：11 · 研究领域：Image Retrieval and Classification Techniques、Advanced Image and Video Retrieval Techniques、Remote-Sensing Image Classification

To address the numerous challenges existing in current remote sensing scene generation methods, such as the difficulty in capturing complex interrelations among geographical features and the integration of implicit expert knowledge into generative models , this paper proposes an efficient method for generating remote sensing scenes using hybrid intelligence and low-rank representation, named Word2Scene, which can generate complex scenes with just one word. This approach combines geographic expert knowledge to optimize the remote sensing scene description, enhancing the accuracy and interpretability of the input descriptions. By employing a diffusion model based on hybrid intelligence and low-rank representation techniques, this method endows the diffusion model with the capability to understand remote sensing scene concepts and significantly improves the training efficiency of the diffusion model. This study also introduces the geographic scene holistic perceptual similarity (GSHPS), a novel evaluation metric that holistically assesses the performance of generative models from a global perspective. Experimental results demonstrate that our proposed method outperforms existing state-of-the-art models in terms of remote sensing scene generation quality, efficiency, and realism. Compared to the original diffusion models, LPIPS decreased by 18.52% (from 0.81 to 0.66), and GSHPS increased by 28.57% (from 0.70 to 0.90), validating the effectiveness and advancement of our method. Mo...