Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Step functions #24

Merged
merged 22 commits into from
Oct 24, 2024
Merged

Step functions #24

merged 22 commits into from
Oct 24, 2024

Conversation

maxcountryman
Copy link
Owner

This introduces step functions.

Step functions are a series of linked tasks where each task takes input from the output of previous one. Said another way, step functions are a chain of dependent tasks, each depending on its immediate ancestor.

This design makes it possible to define a series of discrete tasks that can perform work which is required for further work as the series progresses. What this means is that when tasks fail and are retried, prior work is not repeated.

With this patch jobs are now composed of one or more step functions.

Enabling this interface also necessitates updating tasks to:

  1. Provide an output as an associated type and,
  2. Take a transaction as input to their execute method.

It's worth noting that various breaking changes are contained throughout and this is a significant change to the job API especially.

Note that on series 0.0.x we are NOT adhering to proper semver. Please upgrade with caution until the 0.1.x series.

@maxcountryman maxcountryman merged commit 498dcf1 into main Oct 24, 2024
4 checks passed
@maxcountryman maxcountryman deleted the step-functions branch October 24, 2024 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant