Enhance Fine-Grained Visual Classification with Attention-Guided Region Selection and Contrastive Feature Alignment

作者：Jin Chen, Huan Wang · 发表于：IEEE International Joint Conference on Neural Network · 年份：2024 · DOI：10.1109/ijcnn60899.2024.10651230 · 被引用次数：2 · 研究领域：Computer Science

Vision Transformers (ViTs) have shown great potential in Fine-grained Visual Classification (FGVC). In FGVC task, higher input resolution is commonly employed to assist the model in capturing detailed feature representations. However, for ViTs, augmenting the image size leads to significant computational overhead. To improve the model’s ability to extract detailed features without altering the input size, we introduce a method called Attention-guided Region Selection and Contrastive Feature Alignment (ARS-CFA). ARS-CFA consists of three primary modules: Attention Score Aggregation (ASA), Focal Region Selection (FRS), and Contrastive Feature Alignment (CFA). The ASA module extracts attention weights between patches and aggregates them into relevance scores between the class token and other tokens. The FRS module selects discriminative regions based on relevance scores, obtaining finer-scale images for reinsertion into training. These fine-scale images, when paired with the original images, can also address the lack of similar samples within the same batch, enhancing the efficacy of contrastive loss. The CFA module adopts contrastive loss to align the features of paired images, mining discriminative features and suppressing noise. With these modules, we aptly combine the method of region selection with contrastive loss, significantly enhancing ViT’s capacity for extracting fine-grained feature representations. Experiments demonstrate the effectiveness of our approach compared w...