Use mypy in CI

[rth]

I would propose to use mypy in CI to perform static code analysis. For instance that would allow detecting invalid worker states #474 (comment)

First following issues would need to be fixed,

mypy --ignore-missing-imports ramp-database/ramp_database ramp-engine/ramp_engine ramp-frontend/ramp_frontend ramp-utils/ramp_utils
ramp-frontend/ramp_frontend/forms.py:140: error: Need type annotation for 'names_codes' (hint: "names_codes: List[<type>] = ...")
ramp-frontend/ramp_frontend/forms.py:256: error: Need type annotation for 'name_credits' (hint: "name_credits: List[<type>] = ...")
ramp-utils/ramp_utils/ramp_cli.py:15: error: Signature of "epilog" incompatible with supertype "ArgumentParser"
ramp-engine/ramp_engine/aws/api.py:29: error: "Type[Logger]" has no attribute "manager"
ramp-engine/ramp_engine/aws/api.py:93: error: Incompatible types in assignment (expression has type "Set[str]", variable has type "List[str]")
ramp-engine/ramp_engine/aws/api.py:94: error: Unsupported left operand type for - ("List[str]")
ramp-engine/ramp_engine/aws/aws_train.py:8: error: Module 'ramp_database.tools' has no attribute 'set_predictions'
ramp-engine/ramp_engine/aws/aws_train.py:9: error: Module 'ramp_database.tools' has no attribute 'set_time'
ramp-engine/ramp_engine/aws/aws_train.py:10: error: Module 'ramp_database.tools' has no attribute 'set_scores'
ramp-engine/ramp_engine/aws/aws_train.py:11: error: Module 'ramp_database.tools' has no attribute 'set_submission_state'
ramp-engine/ramp_engine/aws/aws_train.py:12: error: Module 'ramp_database.tools' has no attribute 'get_submissions'
ramp-engine/ramp_engine/aws/aws_train.py:13: error: Module 'ramp_database.tools' has no attribute 'get_submission_state'
ramp-engine/ramp_engine/aws/aws_train.py:14: error: Module 'ramp_database.tools' has no attribute 'get_submission_by_id'
ramp-engine/ramp_engine/aws/aws_train.py:15: error: Module 'ramp_database.tools' has no attribute 'set_submission_max_ram'
ramp-engine/ramp_engine/aws/aws_train.py:16: error: Module 'ramp_database.tools' has no attribute 'score_submission'
ramp-engine/ramp_engine/aws/aws_train.py:17: error: Module 'ramp_database.tools' has no attribute 'set_submission_error_msg'
ramp-engine/ramp_engine/aws/aws_train.py:18: error: Module 'ramp_database.tools' has no attribute 'get_event_nb_folds'
ramp-engine/ramp_engine/aws/ramp_aws_train.py:10: error: Module 'ramp_database.tools' has no attribute 'get_submission_by_name'
Found 18 errors in 5 files (checked 120 source files)

and the errors in ramp-engine/ramp_engine/aws/aws_train.py are actually genuine imports errors because for instance ramp_database.tools.set_predictions does not exist. I'm not sure why we are not identifying this issue with pytest during test discovery, even if there are no associated tests.

WDYT?

[lucyleeow]

FYI the files aws_train.py and aws_train_loop.py are no longer used - they are old scripts from before Joris refactored AWS RAMP. Potentially we could delete them? @glemaitre ?

[glemaitre]

Yes. It was to keep track until that we really try the worker. I think that now this is completely obsolete. Feel free to clean up.

…

On Tue, 24 Nov 2020 at 21:32, Lucy Liu ***@***.***> wrote: FYI the files aws_train.py and aws_train_loop.py are no longer used - they are old scripts from before Joris refactored AWS RAMP. Potentially we could delete them? @glemaitre <https://github.com/glemaitre> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#475 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABY32P3HHRJS7A527IDHDZLSRQJ4LANCNFSM4UBD3KOQ> .

-- Guillaume Lemaitre Scikit-learn @ Inria Foundation https://glemaitre.github.io/