CAFF-DINO: Multi-spectral object detection transformers with cross-attention features fusion

作者：Kévin Helvig, B. Abeloos, Pauline Trouvé-Peloux · 年份：2024 · DOI：10.1109/cvprw63382.2024.00309 · 被引用次数：25 · 研究领域：Remote-Sensing Image Classification、Infrared Target Detection Methodologies、Advanced Image Fusion Techniques

Object detection on images can find benefit from coupling multiple spectra, each presenting specific useful features. However, building an efficient architecture coupling the different modalities is a complex task. Transformers, due to their ability to extract meaningful correlations between the different regions of the inputs appear as a promising way to perform features fusion across different spectra. This work presents a multi-spectral object detection architecture based on cross-attention features fusion (CAFF), combined with a transformer based detector (DINO). We demonstrate here the performance of the proposed approach in object detection compared with state-of-the-art approaches, on infrared-visible multi-spectral datasets. Moreover the robustness to systematic misalignment between image pairs is studied. The proposed approach is generic to any mono-spectrum transformer based detectors. The model developed in this study will be available in a dedicated github repository.