Task-Specific Transformer-Based Language Models in Health Care: Scoping Review

作者：Ha Na Cho, Tae Joon Jun, Young‐Hak Kim, Hee Jun Kang, Imjin Ahn, Hansle Gwon, Yunha Kim, Hyeram Seo, Heejung Choi, Minkyoung Kim, Jiye Han, Gaeun Kee, Seohyun Park, Soyoung Ko · 发表于：JMIR Medical Informatics · 年份：2024 · DOI：10.2196/49724 · 被引用次数：71 · 研究领域：Machine Learning in Healthcare、Artificial Intelligence in Healthcare and Education、Topic Modeling

BACKGROUND: Transformer-based language models have shown great potential to revolutionize health care by advancing clinical decision support, patient interaction, and disease prediction. However, despite their rapid development, the implementation of transformer-based language models in health care settings remains limited. This is partly due to the lack of a comprehensive review, which hinders a systematic understanding of their applications and limitations. Without clear guidelines and consolidated information, both researchers and physicians face difficulties in using these models effectively, resulting in inefficient research efforts and slow integration into clinical workflows. OBJECTIVE: This scoping review addresses this gap by examining studies on medical transformer-based language models and categorizing them into 6 tasks: dialogue generation, question answering, summarization, text classification, sentiment analysis, and named entity recognition. METHODS: We conducted a scoping review following the Cochrane scoping review protocol. A comprehensive literature search was performed across databases, including Google Scholar and PubMed, covering publications from January 2017 to September 2024. Studies involving transformer-derived models in medical tasks were included. Data were categorized into 6 key tasks. RESULTS: Our key findings revealed both advancements and critical challenges in applying transformer-based models to health care tasks. For example, models like Me...