Skip to content

Commit

Permalink
[pypi][bug fix] dependency comparison operator fix
Browse files Browse the repository at this point in the history
- Earlier the micromamba package solver would just statically set `{package}=={version}`
- But some packages have a version which may not be pinned such as `boto3` / `requests`
- This can cause package resolution to fail with errors like this :

```
Bootstrapping virtual environment(s) ...
    Micromamba ran into an error while setting up environment:
    command '/home/ubuntu/.local/bin/micromamba create --yes --quiet --dry-run --no-extra-safety-checks --repodata-ttl=86400 --retry-clean-cache --prefix=/tmp/tmp46ih0kgy/prefix --channel=conda-forge requests==>=2.21.0 boto3==>=1.14.0 python==3.10.12'
```
- this commit tries to handle different scenarios where different operators maybe specified in the `version` value of `packages` dictionary.

Test Flow to verify the fix (Try running before this commit and then after this commit):
```
from metaflow import FlowSpec, step, pypi
class PypiPackageFailFlow(FlowSpec):

    @pypi(packages={"huggingface-hub":"0.16.4"})
    @step
    def start(self):
        print('Starting')
        self.next(self.end)

    @step
    def end(self):
        pass

if __name__ == "__main__":
    PypiPackageFailFlow()
```
  • Loading branch information
valayDave committed Oct 23, 2023
1 parent fd61160 commit 8869b66
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion metaflow/plugins/pypi/micromamba.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,12 @@ def solve(self, id_, packages, python, platform):
for channel in self.info()["channels"] or ["conda-forge"]:
cmd.append("--channel=%s" % channel)

available_operators = [">=", "<=", "==", ">", "<"]
for package, version in packages.items():
cmd.append("%s==%s" % (package, version))
operator = "=="
if any(op in version for op in available_operators):
operator = ""
cmd.append("%s%s%s" % (package, operator, version))
if python:
cmd.append("python==%s" % python)
# TODO: Ensure a human readable message is returned when the environment
Expand Down

0 comments on commit 8869b66

Please sign in to comment.