Make testing faster

[hsluytergaethje]

As mentioned here, running all tests takes very long which is a problem for CI. Possible solutions could be:

• Not running all the tests but specific ones
• Using mockups
• Accepting long runtime

[cmil]

See #20 (comment)

[cmil]

The tests should also be made more robust. The was they are designed right now the test will break if either the DraCor API software is upgraded (see #26) or if the data changes (which was the cause for the mixnmatch failure #25).

Also, the tests should probably not be run against the production site, staging.dracor.org would be a better option. Even better would be to set up a test instance with a controlled set of data.

[lehkost]

I agree, DraCor data in most corpora changes frequently (usually, new plays are added), this shouldn't break things.

For the time being, is there anything I should do or signal when new plays are added?

[hsluytergaethje]

Yes, having a test instance would be great!
By now, most tests run on RusDraCor and on the play andreyev-ne-ubiy (rus000138) – any changes there would break the tests.

[afuetterer]

I would like to offer some help here.

[afuetterer]

An initial step would be to restructure the test suite. Not all tests are slow. See #27.

============================== slowest durations ===============================
403.74s call     tests/test_play.py::TestPlayClass::test_init_play_name
402.84s call     tests/test_play.py::TestPlayClass::test_init_play_title
97.43s call     tests/test_play.py::TestPlayClass::test_init_play_id
95.47s call     tests/test_character.py::TestCharacterClass::test_init
46.13s call     tests/test_dracor.py::TestDraCorClass::test_play_id_to_play_name
44.55s call     tests/test_dracor.py::TestDraCorClass::test_play_name_to_play_id
44.21s call     tests/test_dracor.py::TestDraCorClass::test_play_id_to_play_title
43.60s call     tests/test_dracor.py::TestDraCorClass::test_play_name_to_corpus_name
43.36s call     tests/test_dracor.py::TestDraCorClass::test_play_title_to_play_id
43.26s call     tests/test_dracor.py::TestDraCorClass::test_play_title_to_corpus_name
3.86s call     tests/test_corpus.py::TestCorpusClass::test_metadata_csv
3.74s call     tests/test_corpus.py::TestCorpusClass::test_metadata
3.67s call     tests/test_play.py::TestPlayClass::test_spoken_text
3.40s call     tests/test_play.py::TestPlayClass::test_characters_csv
3.36s call     tests/test_play.py::TestPlayClass::test_get_characters
3.32s call     tests/test_play.py::TestPlayClass::test_relations_gexf
3.28s call     tests/test_play.py::TestPlayClass::test_graphml
3.26s call     tests/test_play.py::TestPlayClass::test_gexf
3.11s call     tests/test_dracor.py::TestDraCorClass::test_corpora
1.99s call     tests/test_wikidata.py::TestWikidataClass::test_mixnmatch
1.24s call     tests/test_play.py::TestPlayClass::test_spoken_text_by_character
1.13s call     tests/test_dracor.py::TestDraCorClass::test_play_info_by_id
1.07s call     tests/test_play.py::TestPlayClass::test_tei
0.90s call     tests/test_corpus.py::TestCorpusClass::test_init
0.89s call     tests/test_wikidata.py::TestWikidataClass::test_get_author_info_by_id
0.82s call     tests/test_dracor.py::TestDraCorClass::test_plays_by_character_wikidata_id
0.75s call     tests/test_play.py::TestPlayClass::test_relations_graphml
0.74s call     tests/test_play.py::TestPlayClass::test_relations_csv
0.74s call     tests/test_play.py::TestPlayClass::test_csv
0.72s call     tests/test_play.py::TestPlayClass::test_stage_directions_with_speakers
0.72s call     tests/test_dracor.py::TestDraCorClass::test_sparql
0.72s call     tests/test_play.py::TestPlayClass::test_stage_directions

(163 durations < 0.005s hidden.  Use -vv to show these durations.)

A step further could be to run the tests in parallel.

[afuetterer]

You could either:

• use mocks for all HTTP communications
• use canned responses with vcr.py
• use a dedicated test server

I think the fastest way is to record the actual responses with vcr.py and then replay them during testing. This generates HTTP responses in YAML files, that get returned in a test case, when ever a HTTP request would occur.

The responses reflect the state of the API during the recording. The recording can be overwritten once something in the API changes. But hitting the actual API multiple times during the tests, results in these slow test times.

[cmil]

Before we have restructured, more efficient revision of pydracor it doesn't make sense to improve the tests. Therefore closing this.