A Review of Deep Learning for Video Captioning

作者：Moloud Abdar, Meenakshi Kollati, Swaraja Kuraparthi, Farhad Pourpanah, Daniel McDuff, Mohammad Ghavamzadeh, Shuicheng Yan, Abduallah Mohamed, Abbas Khosravi, Erik Cambria, Fatih Porikli · 发表于：IEEE Transactions on Pattern Analysis and Machine Intelligence · 年份：2024 · DOI：10.1109/tpami.2024.3522295 · 被引用次数：33 · 研究领域：Multimodal Machine Learning Applications、Subtitles and Audiovisual Media、Video Analysis and Summarization

Video captioning (VC) is a fast-moving, cross-disciplinary area of research that comprises contributions from domains such as computer vision, natural language processing, linguistics, and human-computer interaction. VC aims to understand a video and describe it through natural language descriptors. It plays a crucial role in various applications, from improving accessibility features such as low-vision navigation to advancing video question answering, video retrieval, and content generation. In this survey paper, we present a comprehensive review of deep learning-based VC methods. First, we provide an overview of VC, including the problem formulation, evaluation metrics, training losses, and attention-based architectures. Then, we categorize VC methods into several categories, including attention-based architectures graph networks, reinforcement learning, adversarial networks, and dense video captioning, and discuss each category in detail. In addition, we review existing data sets for VC methods and provide a discussion of research gaps and future research directions. We hope that this survey serves as a guide for researchers in relevant fields.