Improving Co-Decoding Based Security Hardening of Code LLMs Leveraging Knowledge Distillation

作者：Dong Li, Shanfu Shu, Meng Yan, Zhongxin Liu, Chao Liu, Xiaohong Zhang, David Lo · 发表于：IEEE Transactions on Software Engineering · 年份：2025 · DOI：10.1109/tse.2025.3591791 · 被引用次数：3 · 研究领域：Cloud Data Security Solutions、Advanced Data Storage Technologies、Security and Verification in Computing

Large Language Models (LLMs) have been widely adopted by developers in software development. However, the massive pretraining code data is not rigorously filtered, allowing LLMs to learn unsafe coding patterns. Several prior studies have demonstrated that code LLMs tend to generate code with potential vulnerabilities. The widespread adoption of intelligent programming assistants poses a significant threat to the software development process. Existing approaches to mitigating this risk primarily involve constructing secure data that are free of vulnerabilities and then retraining or fine-tuning the models. However, such an effort is resource intensive and requires significant manual supervision. When the model parameters are too large (e.g., more than 1 billion) or multiple models with the same parameter scale have the same optimization needs (e.g., to avoid outputting vulnerable code), the above work will become unaffordable. To address this challenge, in previous work, we proposed CoSec, an approach to improve the security of code LLMs with different parameters by utilizing an independent and very small parametric security model as a decoding navigator.Despite CoSec’s excellent performance, we found that there is still room for improving: 1) its ability to maintain the functional correctness of hardened targets, and 2) the security of the generated code. To address the above issues, we propose CoSec+, a hardening framework consisting of three phases: 1) Functional Correctnes...