Skip to content

Latest commit

 

History

History

test

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
resources
resources
 
 
README.md
README.md
 
 
count_klocs.pl
count_klocs.pl
 
 
get_klocs.sh
get_klocs.sh
 
 
jobs_gen_tests.pl
jobs_gen_tests.pl
 
 
jobs_mutation.pl
jobs_mutation.pl
 
 
jobs_verify_bugs.pl
jobs_verify_bugs.pl
 
 
test.include
test.include
 
 
test_bug_mining.sh
test_bug_mining.sh
 
 
test_cp.sh
test_cp.sh
 
 
test_d4j_query.sh
test_d4j_query.sh
 
 
test_export_command.sh
test_export_command.sh
 
 
test_fix_test_suite.sh
test_fix_test_suite.sh
 
 
test_gen_tests.sh
test_gen_tests.sh
 
 
test_monitor_test.sh
test_monitor_test.sh
 
 
test_mutation_analysis.sh
test_mutation_analysis.sh
 
 
test_mutation_cmd.sh
test_mutation_cmd.sh
 
 
test_sanity_check.sh
test_sanity_check.sh
 
 
test_style.sh
test_style.sh
 
 
test_tutorial.sh
test_tutorial.sh
 
 
test_verify_bugs.sh
test_verify_bugs.sh
 
 

README.md

Testing and analysis

Scripts

  • test.include: Constants and helper functions for test scripts.

  • test_bug_mining.sh: Tests the bug-mining infrastructure.

  • test_export_command.sh: Tests the export command.

  • test_fix_test_suite.sh: Tests the fix_test_suite.pl script.

  • test_gen_tests.sh: Tests the gen_tests.pl script. Specifically, this script tests the integration of all supported test generators and the compatibility of the generated test suites with the coverage, mutation, and bug detection analyses.

  • test_mutation_analysis.sh: Tests various options for the mutation analysis.

  • test_sanity_check.sh: Checks out each buggy and fixed project version and checks for compilation and required properties.

  • test_tutorial.sh: Runs the tutorial described in the top-level README.

  • test_verify_bugs.sh: Reproduces a bug, or a set of bugs, and verifies that the observed set of triggering tests matches the expected set of triggering tests.

CI testing

See .github/workflows/ci.yml for the set of tests that currently run in CI.

Local testing

Some tests take a long time to run and usually only need to be run for major version updates (e.g., bumping the Java version or adding new defects).

Export command and exported properties

The test_export_command.sh test takes a long time as it checks out every single defect and checks whether the exported paths, files, etc. make sense.

Reproducing all bugs (parallel)

Reproducing all bugs is the most time-consuming test. To speed it up, we use GNU parallel (-j gives the number of parallel processes):

./jobs_verify_bugs.pl | shuf > jobs.txt
parallel -j20 --progress < jobs.txt