continue-on-error in workflows, analogous to github actions

[PaulBFB]

Hi guys 👋

Big fan of weasel and used it to build a prototype recently - annotation and learning layout of PDFs (using prodigy, you're probably familiar 😉 )

One thing that came up for me: there's a workflow in which I would like to annotate data with prodigy and then directly use the data in a with prodigy db-out and transform it and train a model. However, the preferred way to shut down prodigy seems to be user interrupt (according to this prodigy issue - how to quit prodigy )

So right now if a step in a weasel workflow has a non-zero exit status, the workflow doesn't continue and this is desired behavior if I understand correctly - however, in this case it would be really nice to tell weasel to continue the command on error like it's usually done in github actions, or better yet, ignore certain exit codes (like 130)

[honnibal]

It perhaps feels a little ugly, but what I would do is write a wrapping bash script for the Prodigy command that doesn't have set -e. This script will exit with status 0, allowing the rest of the workflow to proceed.

If we want to allow continue-on-error in the definition, we'd want to be able to specify which steps should be allowed to error, and with which signals. If the script wrapping works suitably well, I'd be keen to avoid making the weasel definitions more complicated.

[PaulBFB]

thanks a lot for the answer - I should have clarified, yes that's clearly a solution. but as you said, it feel a bit ugly.

As to your point about which steps may be ignored, or which exit status may be skipped, well I suppose that's what I would've preferred since I felt it fits well within the weasel steps definition - that's of course a matter of taste I guess...

[honnibal]

I see your point, and you're right that I use exactly this in other workflow definition systems. So, we'll think about it :)

[PaulBFB]

thanks, that's great! Appreciate it! Big fan of explosion, cheers