A Novel Energy Based Model Mechanism for Multi-Modal Aspect-Based Sentiment Analysis

作者：Tianshuo Peng, Zuchao Li, Ping Wang, Lefei Zhang, Hai Zhao · 发表于：Proceedings of the AAAI Conference on Artificial Intelligence · 年份：2024 · DOI：10.1609/aaai.v38i17.29852 · 被引用次数：23 · 研究领域：Sentiment Analysis and Opinion Mining、Advanced Computing and Algorithms、Advanced Text Analysis Techniques

Multi-modal aspect-based sentiment analysis (MABSA) has recently attracted increasing attention. The span-based extraction methods, such as FSUIE, demonstrate strong performance in sentiment analysis due to their joint modeling of input sequences and target labels. However, previous methods still have certain limitations: (i) They ignore the difference in the focus of visual information between different analysis targets (aspect or sentiment). (ii) Combining features from uni-modal encoders directly may not be sufficient to eliminate the modal gap and can cause difficulties in capturing the image-text pairwise relevance. (iii) Existing span-based methods for MABSA ignore the pairwise relevance of target span boundaries. To tackle these limitations, we propose a novel framework called DQPSA. Specifically, our model contains a Prompt as Dual Query (PDQ) module that uses the prompt as both a visual query and a language query to extract prompt-aware visual information and strengthen the pairwise relevance between visual information and the analysis target. Additionally, we introduce an Energy-based Pairwise Expert (EPE) module that models the boundaries pairing of the analysis target from the perspective of an Energy-based Model. This expert predicts aspect or sentiment span based on pairwise stability. Experiments on three widely used benchmarks demonstrate that DQPSA outperforms previous approaches and achieves a new state-of-the-art performance. The code will be released at ht...