Investigating the application of differential fuzzing to support the identification and suppression of equivalent mutants: An experimental study

Abstract

Abstract Mutation testing is widely used to assess the adequacy of test suite; however, its practical adoption is constrained by the persistent problem of equivalent mutants, i.e., mutants whose observable behavior is indistinguishable from the original program and therefore cannot be killed by any test case. Prior studies report that identifying equivalent mutants often requires substantial manual effort. Meanwhile, fuzzing is increasingly used in real-world systems, especially security-critical software. However, conventional fuzzing oracles typically detect only crash-like failures, thereby missing many behavioral changes introduced by mutants. In this paper, we investigate differential fuzzing as a practical, language-agnostic approach to support mutant classification, focusing on identifying (and confirming) equivalent mutants. We conduct a multi-project study across Bitcoin Core (C++), OpenSSL (C), LND (Go), and Arrow (Python), mutating six real functions with 1,090 valid mutants generated by the universalmutator. We compare unit/functional testing, seed-corpus-only fuzzing, time-bounded fuzzing, and two differential fuzzing configurations (seed-corpus-only and time-bounded). Our results show that conventional fuzzing yields the lowest mutation scores, while time-bounded differential fuzzing (5 minutes per mutant) achieves 98–100\% mutation score in five of six targets, and exposes cases where seed corpora contain valuable test inputs not covered by unit/functional suites. We further analyze runtime variability and observe log-normal behavior in difficult cases, providing practical guidance on stopping criteria (time/execution budgets), as well as evidence that dictionaries and parallel fuzzing can significantly improve effectiveness. Overall, our findings indicate that differential fuzzing is simple to implement and can classify mutants efficiently in practice, while also producing actionable artifacts (seed corpus inputs) to strengthen traditional test suites.