Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check Python process error handling #66

Open
johann-petrak opened this issue Oct 27, 2021 · 0 comments
Open

Check Python process error handling #66

johann-petrak opened this issue Oct 27, 2021 · 0 comments

Comments

@johann-petrak
Copy link
Collaborator

This is relevant when a pipeline is run e.g. in GCP where an exception from the python plugin will not cause the termination of the whole execution. So if the Python process started by one duplicate dies, the plugin still gets run for more documents.
Currently, we do not re-start the process automatically if it died.

  • in theory, the python process should not die if the plugin python code throws an exception
  • are there any errors that should/could terminate the python process?
  • note: syntax error should get checked even before the actual process gets started (separate process for checking that!)
  • maybe we should have a parameter to enable auto-restarting the python process? Maybe make it a number: restart that many times at most

In any case, would be good if we test this and document exactly what is happening.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant