Code Refinement with Repository Context: How Far are We?

作者：Ke Wang, Peng Lan, Jiakun Liu, Shengbing Ren, Lingfeng Bao, Junxiao Han, David Lo, Zhifang Liao · 发表于：ACM Transactions on Software Engineering and Methodology · 年份：2026 · DOI：10.1145/3820059 · 研究领域：Software Engineering Research、Software Testing and Debugging Techniques、Software System Performance and Reliability

Code refinement, a process of revising flawed code changes based on the code review comments, is a fundamental practice in software development for ensuring software quality and maintainability. Recent approaches using large language models (LLMs) show promise in automating this process, yet they typically rely only on review comments and a single code snippet, neglecting broader repository context. When a code refinement task requires invoking reviewer-unspecified identifiers (e.g., variables, functions, or classes) that are defined elsewhere in the repository, the LLM must accurately predict the exact identifier signature and usage, otherwise it could easily result in fatal errors. Prior work demonstrates that 90.59% of such knowledge-conflicting hallucinations cause all test cases to fail. Notably, over 60% of real-world code refinement scenarios in most popular Github repositories depend on repository-level knowledge, yet none of the existing code refinement works leverage this information. Moreover, current datasets do not distinguish between context-dependent and context-independent instances, contain many low-quality instances, and rely on lexical metrics (e.g., BLEU) that poorly correlate with human judgment. These limitations hinder reliable evaluation of code refinement tools in real-world, context-dependent scenarios. To address this gap, we introduce ReCoRe-Bench , a high-quality benchmark of 1,000 code refinement instances from 328 Python, Java, and JavaScript re...