Applying LLMs to the Classification of Cybersecurity Incident Tickets in a Few-Shot Scenario

作者：Sebastião Alves de Jesus Filho, Alvaro Santana Santos, Rafael Dias Araújo, R. Miani · 发表于：International Conference on Machine Learning and Applications · 年份：2025 · DOI：10.1109/icmla66185.2025.00144 · 研究领域：Computer Science

The growing complexity and volume of cybersecurity incidents have increasingly challenged Computer Security Incident Response Teams (CSIRTs), demanding more efficient solutions for alert triage and categorization. In this work, we investigate the use of large language models (LLMs), such as GEMINI 2.0 Flash and LLaMA 3 70B, for classifying real incident tickets based on the CERT.br taxonomy. We explore zero-shot, one-shot, and two-shot learning approaches to compare model performance under different input configurations. Our results demonstrate that incorporating just a single labeled example (one-shot) is sufficient to increase the average F1-score from approximately 45% to over 90%, highlighting the effectiveness of few-shot learning in this classification task. While the performance among models was similar, we find that the approach shows promise in significantly reducing the human effort required for alert triage. We also discuss limitations related to category ambiguity and data representativeness, and suggest future work, including expanding the dataset, refining the taxonomy, and evaluating new language models, potentially through local deployment for enhanced control and parameter tuning.