Skip to content

Commit

Permalink
Untested try at rejecting workers submitting bad data
Browse files Browse the repository at this point in the history
Marked as draft because it needs review/testing but in principle, it should be merged as soon as possible.

This addresses server-side filtering of problematic tasks caused by official-stockfish#1393. Obviously we should also prevent this worker-side problem in the first place, but in the short run this should at least prevent fishtests from being polluted by garbage as they currently are
  • Loading branch information
dubslow committed Aug 2, 2022
1 parent 899c601 commit 0e42b18
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions server/fishtest/rundb.py
Original file line number Diff line number Diff line change
Expand Up @@ -912,6 +912,13 @@ def count_games(d):
error = "Update_task: the number of games received for task {}/{} is incompatible with the SPRT batch size".format(
run_id, task_id
)
elif "stats" in task
and ((task["stats"]["time_losses"] + task["stats"]["crashes"])*100 > num_games)
# Allow workers at most 1% bad data, however patches which break TM need special handling (TODO?)
and error == "":
error = "Update task: task {}/{} has too many crashes+timelosses ({}+{})".format(
run_id, task_id, task["stats"]["crashes"], task["stats"]["time_losses"]
)

if error != "":
print(error, flush=True)
Expand Down

0 comments on commit 0e42b18

Please sign in to comment.