Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hoist up algorithm job errors to evaluation errors for system messages only #3486

Open
jmsmkn opened this issue Aug 8, 2024 · 0 comments
Open

Comments

@jmsmkn
Copy link
Member

jmsmkn commented Aug 8, 2024

An algorithm job can fail with a system message: time out, killed due to memory, output not produced, etc. Can these be hoisted up to the evaluation to help admins?

Non-system messages (e.g. anything custom produced from the users container) MUST be avoided as that could leak information about the inputs.

These messages would have to be distinguished from any evaluation container failure which can fail for similar reasons. User can often be confused if the evaluation fails with a memory limit exception, etc, but it's only a problem with the evaluation method and not their algorithm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant