Concurrent Grading using asyncio

[brhoades]

I've drafted concurrent grading using asyncio coroutines (threads instead of processes). By default, it's synchronous, but if -j is specified it uses that many worker threads.

It's currently NOT printing output as I'm not sure if that's desirable / if print locks stdout until it finishes.

Implements #21

Default (unchanged) behavior:

$ bin/grader grade hw1
INFO     Grading submissions for 6960054b3387
INFO:grade-it:Time to start the tests.
============================= test session starts ==============================
platform linux -- Python 3.4.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /tmp/tmp.nH26e9oICs, inifile:
collected 0 items

========================= no tests ran in 0.00 seconds =========================
INFO     Logging 6960054b3387 output as text
INFO     Wrote to grader/assignments/hw1/results/6960054b3387.13.log
INFO     Grading submissions for 9d7fe62204ad
INFO:grade-it:Time to start the tests.
============================= test session starts ==============================
platform linux -- Python 3.4.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /tmp/tmp.dMhPCDEGYY, inifile:
collected 0 items

========================= no tests ran in 0.00 seconds =========================
INFO     Logging 9d7fe62204ad output as text
INFO     Wrote to grader/assignments/hw1/results/9d7fe62204ad.13.log
INFO     Grading submissions for a65a6d7183c2
INFO:grade-it:Time to start the tests.
============================= test session starts ==============================
platform linux -- Python 3.4.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /tmp/tmp.myqboGzXZb, inifile:
collected 0 items

========================= no tests ran in 0.00 seconds =========================
INFO     Logging a65a6d7183c2 output as text
INFO     Wrote to grader/assignments/hw1/results/a65a6d7183c2.12.log
INFO     Grading submissions for dacb24920b25
INFO:grade-it:Time to start the tests.
============================= test session starts ==============================
platform linux -- Python 3.4.3, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /tmp/tmp.s3cgnK0vx8, inifile:
collected 0 items

========================= no tests ran in 0.00 seconds =========================
INFO     Logging dacb24920b25 output as text
INFO     Wrote to grader/assignments/hw1/results/dacb24920b25.12.log

and with concurrency:

$ bin/grader grade -j2 hw1
INFO     Grading submission for 6960054b3387
INFO     Grading submission for 9d7fe62204ad
INFO     Logging 9d7fe62204ad output as text
INFO     Wrote to grader/assignments/hw1/results/9d7fe62204ad.14.log
INFO     Logging 6960054b3387 output as text
INFO     Wrote to grader/assignments/hw1/results/6960054b3387.14.log
INFO     Grading submission for a65a6d7183c2
INFO     Grading submission for dacb24920b25
INFO     Logging dacb24920b25 output as text
INFO     Wrote to grader/assignments/hw1/results/dacb24920b25.13.log
INFO     Logging a65a6d7183c2 output as text
INFO     Wrote to grader/assignments/hw1/results/a65a6d7183c2.13.log

[brhoades]

Output from threads is now, by default, printed unless suppress_output is enabled. Output is printed at the end of the thread by the thread (not as it is retrieved). This reduces the chance that two threads will print at once, but it's still possible.

Do we want to do a mutex lock on stdout here?

ie: https://docs.python.org/3/library/threading.html#condition-objects

[LinuxMercedes]

I think it'd be nice to have printing synchronous, but not a requirement.

[brhoades]

@LinuxMercedes added print lock
@michaelwisely comments?

[michaelwisely]

@brhoades i think there's a more idiomatic way to synchronize output without using a Lock(). Let me have a go at refactoring tonight.

[michaelwisely]

@brhoades is this an unbalanced parenthesis?

[brhoades]

This PR is almost as old as my career. Closing it out.