Semantic Tokenization-Based Mamba for Hyperspectral Image Classification

作者：Ri Ming, Na Chen, Jiangtao Peng, Weiwei Sun, Zhijing Ye · 发表于：IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · 年份：2025 · DOI：10.1109/jstars.2025.3528122 · 被引用次数：11 · 研究领域：Remote-Sensing Image Classification、Remote Sensing and Land Use、Image Retrieval and Classification Techniques

Recently, the Transformer-based model has shown superior performance in hyperspectral image classification (HSIC) due to its excellent ability to model long-term dependencies on sequence data. An important component of the Transformer is the tokenizer, which can transform the features into semantic token sequences (STS). Nonetheless, Transformer's semantic tokenization strategy is hardly representative of local relatively important high-level semantics because of its global receptive field. Recently, the Mamba-based methods have shown even stronger spatial context modeling ability than Transformer for HSIC. However, these Mamba-based methods mainly focus on spectral and spatial dimensions. They tend to extract semantic information in very long feature sequences or represent semantic information in several typical tokens, which may ignore some important semantics of the HSIs. In order to represent the semantic information of HSIs more holistically in Mamba, this article proposes a semantic tokenization-based Mamba (STMamba) model. In STMamba, a spectral-spatial feature extraction (SSFE) module is used to extract the spectral-spatial joint features. Then a generated semantic token sequences module (GSTSM) is designed to transform the features into STS. Subsequently, the STS are fed into the semantic token state spatial model (STSSM) to capture relationships between different semantic tokens. Finally, the fused semantic token is passed into a classifier for classification. Exper...