— publication, — repository


DroixBench — a collection of 24 reproducible crashes in open-source Android apps


Codeflaws — 3902 bugs from Codeforces programming competition for evaluating program repair tools across different defect classes
DBGBench — 291 (in)correct patches from real software professionals for 27 real bugs in C for the qualitative evaluation of automated repair techniques
IntroClass — automated program repair benchmark that consists of 998 defects in small student-written programming assignments
ManyBugs — automated program repair benchmark that consists of 185 defects from large popular open-source projects


Bears — an extensible Java bug benchmark for automatic program repair studies
Bugs.jar — a large-scale, diverse dataset of bugs for Java program repair
Defects4J — a database of existing faults to enable controlled testing studies for Java


BugSwarm — a dataset of thousands of real software bugs and their fixes
QuixBugs — a parallel corpus of 40 programs in both Python and Java, each with a bug on one line