HI4HC and AAAAD: Exploring a hierarchical method and dataset using hybrid intelligence for remote sensing scene captioning

作者：Jiaxin Ren, Wanzeng Liu, Jun Chen, Shunxi Yin · 发表于：International Journal of Applied Earth Observation and Geoinformation · 年份：2025 · DOI：10.1016/j.jag.2025.104491 · 被引用次数：3 · 研究领域：Multimodal Machine Learning Applications、Advanced Image and Video Retrieval Techniques、Image Retrieval and Classification Techniques

• First hierarchical element-relation-scene data structure for multimodal remote sensing. • Hybrid intelligence enhances remote sensing imagery captioning. • The first and only high-quality remote sensing imagery caption dataset AAAAD is now public. • AAAAD analyzed against mainstream remote sensing imagery text-image pair datasets. • AAAAD tested in remote sensing imagery generation. Remote sensing scene captioning is crucial for the deep understanding and intelligent analysis of Earth observation data. Many existing methods and datasets lack a fine-grained description of key geographical elements, fail to capture the full diversity of spatial relations, and are limited in their applicability to real-world geospatial scenarios. To address these shortcomings, we propose HI4HC (hybrid intelligence for remote sensing scene hierarchical captioning), a novel method that combines deep learning algorithms with expert knowledge to generate hierarchical captions for remote sensing scenes. This approach comprehensively describes scenes across three dimensions: geographical elements, spatial relations, and scene concepts, resulting in more accurate, detailed, and comprehensive captions. Leveraging HI4HC, we have constructed and made public a high-quality hierarchical caption dataset named AAAAD (adopt-amend-annihilate-add dataset). Extensive experiments show that AAAAD outperforms traditional single-level caption datasets in terms of the richness of geographical elements, the precision...