Identifying the Failure-Revealing Test Cases in Metamorphic Testing: A Statistical Approach

作者：Zheng Zheng, Daixu Ren, Huai Liu, Tsong Yueh Chen, T. T. Li · 发表于：ACM Transactions on Software Engineering and Methodology · 年份：2024 · DOI：10.1145/3695990 · 被引用次数：3 · 研究领域：Software Testing and Debugging Techniques、Software Reliability and Analysis Research、VLSI and Analog Circuit Testing

Metamorphic testing, thanks to its high failure-detection effectiveness especially in the absence of test oracle, has been widely applied in both the traditional context of software testing and other relevant fields such as fault localization and program repair. Its core element is a set of metamorphic relations, which are the necessary properties of the target algorithm in the form of the relationships among multiple inputs and corresponding expected outputs. When a relation is violated by the outputs of a group of test cases, namely metamorphic group of test cases, that are constructed based on the relation, a failure is said to be revealed. Traditionally, the primary task of software testing is to reveal failures. Therefore, from the perspective of software testing, it may not need to know which test case(s) in the metamorphic group cause the violation and thus the failure. However, such information is definitely helpful for other software engineering activities, such as software debugging. The current literature of metamorphic testing lacks a systematic mechanism of identifying the actual failure-revealing test cases, which hinders its applicability and effectiveness in other relevant fields. In this article, we propose a new technique for the FAILure-revealing Test case Identification in Metamorphic testing, namely FAILTIM. The approach is based on a novel application of statistical methods. More specifically, we leverage and adapt the basic ideas of spectrum-based techn...