Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AggregateFlows and Benchmarking support for workflows with dependencies #15

Merged
merged 2 commits into from
Mar 21, 2024

Conversation

zeddo123
Copy link
Member

@zeddo123 zeddo123 commented Mar 21, 2024

This PR makes it possible for the AggregateFlows class (and the benchmarking class) to play well with workflows that have additional dependencies (aka workflows with more than 1 input).

With these proposed changes, the AggregateFlows class can now accept an additional field dependencies that would be used (if not None) when running workflows, thus insuring that they have the correct inputs.

flows = AggregateFlows(input, dependencies=[x, y], workflows=[wf1, wf2])

To add support for the Benchmarking class, the only thing to add is to introduce the class InputWithDeps. When this type is used by the dataset (as the feature), the DefaultAggregateFlowsFactory would automatically "unpack" the object and inject the appropriate inputs to the AggregateFlows objects. Practically, we only need to define our dataset in particular way and octopipes takes care of the rest (of course when the DefaultAggregateFlowsFactory is used).
Here's an example:

class Dataset:
    ....
    def __getitem__(self, index):
         ...
         return InputWithDeps(input=image, dependencies=[bboxes]), ground_truth

fixes #14

@zeddo123 zeddo123 added bug Something isn't working critical labels Mar 21, 2024
@zeddo123 zeddo123 self-assigned this Mar 21, 2024
Copy link

@Wala-Touati Wala-Touati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, thanks for the fix

@zeddo123 zeddo123 merged commit 1cd4c3f into master Mar 21, 2024
4 checks passed
@zeddo123 zeddo123 deleted the input-with-deps-aggregateflows branch March 21, 2024 23:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working critical
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Workflows with dependencies don't work with AggregateFlows and Benchmarks
2 participants