Cross-Modal Full-Mode Fine-Grained Alignment for Text-to-Image Person Retrieval

作者：Hao Yin, Xin Man, Feiyu Chen, Jie Shao, Heng Tao Shen · 发表于：ACM Transactions on Multimedia Computing Communications and Applications · 年份：2026 · DOI：10.1145/3786798 · 被引用次数：2 · 研究领域：Video Surveillance and Tracking Methods、Multimodal Machine Learning Applications、Advanced Neural Network Applications

Text-to-Image Person Retrieval (TIPR) is a cross-modal matching task designed to identify the person images that best correspond to a given textual description. The key difficulty in TIPR is to realize robust correspondence between the textual and visual modalities within a unified latent representation space. To address this challenge, prior approaches incorporate attention mechanisms for implicit cross-modal local alignment. However, they lack the ability to verify whether all local features are correctly aligned. Moreover, existing methods tend to emphasize the utilization of hard negative samples during model optimization to strengthen discrimination between positive and negative pairs, often neglecting incorrectly matched positive pairs. To mitigate these problems, we propose FMFA, a cross-modal Full-Mode Fine-Grained Alignment framework, which enhances global matching through Explicit Fine-Grained Alignment (EFA) and existing implicit relational reasoning—hence the term “full-mode”—without introducing extra supervisory signals. In particular, we propose an Adaptive Similarity Distribution Matching (A-SDM) module to rectify unmatched positive sample pairs. A-SDM adaptively pulls the unmatched positive pairs closer in the joint embedding space, thereby achieving more precise global alignment. Additionally, we introduce an EFA module, which makes up for the lack of verification capability of implicit relational reasoning. EFA strengthens explicit cross-modal fine-grained i...