AggregateFlows and Benchmarking support for workflows with dependencies #15
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes it possible for the AggregateFlows class (and the benchmarking class) to play well with workflows that have additional dependencies (aka workflows with more than 1 input).
With these proposed changes, the
AggregateFlows
class can now accept an additional fielddependencies
that would be used (if not None) when running workflows, thus insuring that they have the correct inputs.To add support for the Benchmarking class, the only thing to add is to introduce the class
InputWithDeps
. When this type is used by the dataset (as the feature), theDefaultAggregateFlowsFactory
would automatically "unpack" the object and inject the appropriate inputs to the AggregateFlows objects. Practically, we only need to define our dataset in particular way andoctopipes
takes care of the rest (of course when theDefaultAggregateFlowsFactory
is used).Here's an example:
fixes #14