Query Adaptive Transformer and Multiprototype Rectification for Few-Shot Remote Sensing Image Segmentation

作者：Tianyi Gao, Zhi Gao, Hong Ji, Wei Ao, Weiwei Song · 发表于：IEEE Transactions on Geoscience and Remote Sensing · 年份：2024 · DOI：10.1109/tgrs.2024.3511622 · 被引用次数：57 · 研究领域：Infrared Target Detection Methodologies、Advanced Image Fusion Techniques、Remote-Sensing Image Classification

Deep learning has emerged as a powerful tool for semantic segmentation tasks. However, in some data-deficient and resource-limited scenarios, networks are constrained to learn novel concepts. To tackle this problem, few-shot segmentation (FSS) has been proposed by the machine learning community, aiming at segmenting novel objects by leveraging a handful of annotated samples. However, most advanced FSS algorithms suffer from severe performance degradation when directly applied to remote sensing image (RSI) domains. Challenges arise due to the unique characteristics of RSIs. To address the large intraclass variation brought by various imaging conditions and large-scale variations, a query adaptive transformer (QAT) is proposed. QAT incorporates query priors into the feature extraction process, adapting RSI features to novel objects from query RSIs. It is noteworthy that query priors are extracted via a query prior generation (QPG) module devoid of any query label information. To alleviate interference brought by complex object distribution, a multiprototype strategy is adopted instead of representing objects with a single prototype ambiguously. Moreover, we rectify the prototypes using a prior injection module (PIM), thereby fully leveraging the advantages offered by query priors. The superiority of our method is validated through comprehensive experiments on the public iSAID-$5^{i}$dataset and comparisons with state-of-the-art methods. Finally, we propose a novel cross-domain ...