Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set parameters for running flow instead of any imported #1653

Closed
wants to merge 1 commit into from

Conversation

maxzheng
Copy link
Contributor

@maxzheng maxzheng commented Dec 9, 2023

This fixes the issue where parameters for multiple imported flows are displayed instead of the running one when multiple flows are in the same file with parameters.

% python /tmp/test.py run --help | head
Metaflow 2.10.8.post3+gite5c0719 executing Second for user:max
Usage: test.py run [OPTIONS]

  Run the workflow locally.

Options:
  --base-param TEXT
  --second TEXT
  --tag TEXT                Annotate this run with the given tag. You can
                            specify this option multiple times to attach
                            multiple tags in the run.

Also:

  1. Change to use Exception as bare except is bad practice / fails flake8.
  2. Switch to editable/develop install so any code changes would be effective without reinstalling. Much faster to iterate changes.

Note: I didn't add a test for this as this feature is already tested by many existing tests and they worked fine with this change, but happy to add one specifically for the mentioned issue if desired.

This fixes the issue where parameters for multiple imported flows
are displayed instead of the running one when multple flows are
in the same file with parameters.

Also:
1. Change to use Exception as bare except is bad practice / fails
flake8.
2. Switch to editable/develop install so any code changes would be
   effective without reinstalling. Much faster to iterate changes.
# This must be set before calling cli.main() below (or specifically, add_custom_parameters)
parameters.parameters = [
getattr(self, a)
for a in dir(self)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's already a function known as _get_parameters, we can probably re-use that instead?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something like: parameters.parameters = [p for _, p in self._get_parameters()]

Copy link
Collaborator

@madhur-ob madhur-ob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe @savingoyal can comment on changes to test_runner and tox.ini, LGTM otherwise...

Will prefer re-using self._get_parameters too...

Copy link
Collaborator

@saikonen saikonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of the changes seem unrelated to the proposed fix. If these are not required, could you please clean them from the PR before we go forward with it?

@@ -1,2 +1,5 @@
[metadata]
license_files = LICENSE

[flake8]
max-line-length = 120
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like an unrelated change to the PR. We're also not using flake8 currently

@@ -5,7 +5,7 @@ set -o errexit -o nounset -o pipefail
install_deps() {
for version in 2 3;
do
python$version -m pip install .
python$version -m pip install -e .
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it necessary to change to an editable install for this PR?

@@ -3,4 +3,4 @@
[testenv]
allowlist_externals = ./test_runner
commands = ./test_runner

usedevelop = True
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

required change?

@saikonen
Copy link
Collaborator

closing as this was superseded by #1691

@saikonen saikonen closed this Jan 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants