Deep Reversible Consistency Learning for Cross-Modal Retrieval

作者：Ruitao Pu, Yang Qin, Dezhong Peng, Xiaomin Song, Huiming Zheng · 发表于：IEEE Transactions on Multimedia · 年份：2025 · DOI：10.1109/tmm.2025.3535313 · 被引用次数：10 · 研究领域：Multimodal Machine Learning Applications、Advanced Image and Video Retrieval Techniques、Domain Adaptation and Few-Shot Learning

Cross-modal retrieval (CMR) typically involves learning common representations to directly measure similarities between multimodal samples. Most existing CMR methods commonly assume multimodal samples in pairs and employ joint training to learn common representations, limiting the flexibility of CMR. Although some methods adopt independent training strategies for each modality to improve flexibility in CMR, they utilize the randomly initialized orthogonal matrices to guide representation learning, which is suboptimal since they assume inter-class samples are independent of each other, limiting the potential of semantic alignments between sample representations and ground-truth labels. To address these issues, we propose a novel method termed Deep Reversible Consistency Learning (DRCL) for cross-modal retrieval. DRCL includes two core modules, i.e., Selective Prior Learning (SPL) and Reversible Semantic Consistency learning (RSC). More specifically, SPL first learns a transformation weight matrix on each modality and selects the best one based on the quality score as the Prior, which greatly avoids indiscriminateselection of priors learned from low-quality modalities. Then, RSC employs a Modality-invariant Representation Recasting mechanism (MRR) to recast the potential modality-invariant representations from sample semantic labels by the generalized inverse matrix of the prior. Since labels are devoid of modal-specific information, we utilize the recast features to guide the ...