IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity Alignment

作者：Taoyu Su, Jiawei Sheng, Shicheng Wang, Xinghua Zhang, Hongbo Xu, Tingwen Liu · 年份：2024 · DOI：10.1145/3664647.3680954 · 被引用次数：5 · 研究领域：Topic Modeling、Natural Language Processing Techniques、Multimodal Machine Learning Applications

Multi-modal entity alignment (MMEA) aims to identify equivalent entities between multi-modal knowledge graphs (MMKGs), where entities can be associated with related images.Most existing studies rely heavily on the automatically learned multi-modal fusion modules, which may allow redundant information such as misleading clues in the generated entity representations, impeding the feature consistency of equivalent entities.To this end, we propose a variational framework for MMEA via information bottleneck, termed as IBMEA, by emphasizing alignment-relevant information while suppressing alignment-irrelevant information in entity representations.Specifically, we first develop multi-modal variational encoders that represent modal-specific features as probability distributions.Then, we propose four modal-specific information bottleneck regularizers to limit the misleading clues in the modal-specific entity representations.Finally, we propose a modal-hybrid information contrastive regularizer to integrate modal-specific representations and ensure the similarity of equivalent entities between MMKGs to achieve MMEA.We conduct extensive experiments on 2 cross-KG and 3 bilingual MMEA datasets.Experimental results demonstrate that our model consistently outperforms previous state-of-the-art methods, and also shows promising and robust performance especially in the low-resource and high-noise data scenarios.