— publication, — repository


DroixBench — a collection of 24 reproducible crashes in open-source Android apps


Codeflaws — 3902 bugs from Codeforces programming competition for evaluating program repair tools across different defect classes
DBGBench — 291 (in)correct patches from real software professionals for 27 real bugs in C for the qualitative evaluation of automated repair techniques
IntroClass — automated program repair benchmark that consists of 998 defects in small student-written programming assignments
ManyBugs — automated program repair benchmark that consists of 185 defects from large popular open-source projects


Bears — an extensible Java bug benchmark for automatic program repair studies
Bugs.jar — a large-scale, diverse dataset of bugs for Java program repair
Defects4J — a database of existing faults to enable controlled testing studies for Java


BugsJS — a benchmark of 453 real, manually validated JavaScript bugs from 10 popular JavaScript server-side programs


BugSwarm — a dataset of thousands of real software bugs and their fixes
Defexts — a curated dataset of reproducible real-world bugs for modern JVM languages (Kotlin, Groovy, Scala)
QuixBugs — a parallel corpus of 40 programs in both Python and Java, each with a bug on one line


BugsInPy — a database of existing bugs in Python programs to enable controlled testing and debugging studies
Refactory — a dataset of 1783 buggy and 2442 correct student submissions for 5 Python programming assignments