CodeContests+: High-Quality Test Case Generation for Competitive Programming

作者：Zihan Wang, Siyao Liu, Yang Sun, Ming Ding, Hongyan Li · 年份：2025 · DOI：10.18653/v1/2025.findings-emnlp.299 · 被引用次数：7 · 研究领域：Software Testing and Debugging Techniques、Model-Driven Software Engineering Techniques、Logic, programming, and type systems

Competitive programming, due to its high reasoning difficulty and precise correctness feedback, has become a key task for both training and evaluating the reasoning capabilities of large language models (LLMs).However, while a large amount of public problem data, such as problem statements and solutions, is available, the test cases of these problems are often difficult to obtain.Therefore, test case generation is a necessary task for building largescale datasets, and the quality of the test cases directly determines the accuracy of the evaluation.In this paper, we introduce an LLM-based agent system that creates high-quality test cases for competitive programming problems.We apply this system to the CodeContests dataset and propose a new version with improved test cases, named CodeContests +1 .We evaluated the quality of test cases in CodeContests + .First, we used 1.72 million submissions with pass/fail labels to examine the accuracy of these test cases in evaluation.The results indicated that CodeContests + achieves significantly higher accuracy than CodeContests, particularly with a notably higher True Positive Rate (TPR).Subsequently, our experiments in LLM Reinforcement Learning (RL) further confirmed that improvements in test case quality yield considerable advantages for RL.