Joint Classification of Hyperspectral and LiDAR Data Using Binary-Tree Transformer Network

作者：Huacui Song, Yuanwei Yang, Xianjun Gao, Maqun Zhang, Shaohua Li, Bo Liu, Yanjun Wang, Yuan Kou · 发表于：Remote Sensing · 年份：2023 · DOI：10.3390/rs15112706 · 被引用次数：18 · 研究领域：Remote-Sensing Image Classification、Advanced Image Fusion Techniques、Remote Sensing and Land Use

The joint utilization of multi-source data is of great significance in geospatial observation applications, such as urban planning, disaster assessment, and military applications. However, this approach is confronted with challenges including inconsistent data structures, irrelevant physical properties, scarce training data, insufficient utilization of information and an imperfect feature fusion method. Therefore, this paper proposes a novel binary-tree Transformer network (BTRF-Net), which is used to fuse heterogeneous information and utilize complementarity among multi-source remote sensing data to enhance the joint classification performance of hyperspectral image (HSI) and light detection and ranging (LiDAR) data. Firstly, a hyperspectral network (HSI-Net) is employed to extract spectral and spatial features of hyperspectral images, while the elevation information of LiDAR data is extracted using the LiDAR network (LiDAR-Net). Secondly, a multi-source transformer complementor (MSTC) is designed that utilizes the complementarity and cooperation among multi-modal feature information in remote sensing images to better capture their correlation. The multi-head complementarity attention mechanism (MHCA) within this complementor can effectively capture global features and local texture information of images, hence achieving full feature fusion. Then, to fully obtain feature information of multi-source remote sensing images, this paper designs a complete binary tree structure, b...