AiRs: Adapter in Remote Sensing for Parameter-Efficient Transfer Learning

作者：Leiyi Hu, Hongfeng Yu, Wanxuan Lu, Dongshuo Yin, Xian Sun, Kun Fu · 发表于：IEEE Transactions on Geoscience and Remote Sensing · 年份：2024 · DOI：10.1109/tgrs.2024.3351889 · 被引用次数：23 · 研究领域：Remote-Sensing Image Classification、Domain Adaptation and Few-Shot Learning、Advanced Neural Network Applications

Remote sensing is stepping into the era of the foundation model, where the fine-tuning paradigm is widely adopted to transfer the profound knowledge of pretrained foundation models to downstream tasks. However, the full fine-tuning method would become inefficient in terms of training and storage, as the foundation models are getting larger and larger. Recently, a lot of deep learning research has proposed various parameter-efficient fine-tuning (PEFT) methods that perform well with a few trainable parameters. However, most of them focus on fine-tuning general foundation models without considering the special properties of remote sensing. In this article, we propose an adapter in remote sensing (AiRs) to fine-tune large foundation models for remote sensing downstream tasks by introducing the adapter-tuning framework. Specifically, we construct AiRs from two aspects: more expressive adaptation modules and a more efficient integration strategy. Specialized adaptation modules are applied to different functional layers in AiRs, which encode the inductive bias of remote sensing images and enhance the semantic concepts of geography. Moreover, AiRs establishes pathways between trainable modules with residual connections, which reduces training difficulty and improves performance. We conduct extensive experiments on object detection, semantic segmentation, and scene classification tasks. By training only 4.4% parameters of the pretrained backbone, AiRs surpasses the previous state-of-...