[pypi][bug fix] dependency comparison operator fix

[valayDave]

• Earlier the micromamba package solver would just statically set {package}=={version}
• But some packages have a version which may not be pinned such as boto3 / requests
• This can cause package resolution to fail with errors like this :

Bootstrapping virtual environment(s) ...
    Micromamba ran into an error while setting up environment:
    command '/home/ubuntu/.local/bin/micromamba create --yes --quiet --dry-run --no-extra-safety-checks --repodata-ttl=86400 --retry-clean-cache --prefix=/tmp/tmp46ih0kgy/prefix --channel=conda-forge requests==>=2.21.0 boto3==>=1.14.0 python==3.10.12'

• this commit tries to handle different scenarios where different operators maybe specified in the version value of packages dictionary.

Test Flow to verify the fix (Try running before this commit and then after this commit):

from metaflow import FlowSpec, step, pypi
class PypiPackageFailFlow(FlowSpec):

    @pypi(packages={"huggingface-hub":"0.16.4"})
    @step
    def start(self):
        print('Starting')
        self.next(self.end)

    @step
    def end(self):
        pass

if __name__ == "__main__":
    PypiPackageFailFlow()

[saikonen]

Not opposed to the changes, but wondering about the reproducibility of the issue. Is this somehow platform/architecture specific? Running your test flow on the latest master leads to no errors at least when deploying from an x86 mac to --with kubernetes.

[valayDave]

For more context : I am not running this on mac. I am running it on a ubuntu machine. One thing I noticed that this line has a packages dictionary that looks like this :

{'requests': '>=2.21.0', 'boto3': '>=1.14.0'}

Something weird happened today. I wiped all my environments and created a fresh flow to bootstrap the environment. This time, the tip of master could easily resolve the environments. Even if the command it got was :

/home/ubuntu/.local/bin/micromamba create --yes --quiet --dry-run --no-extra-safety-checks --repodata-ttl=86400 --retry-clean-cache --prefix=/tmp/tmp46ih0kgy/prefix --channel=conda-forge requests==>=2.21.0 boto3==>=1.14.0 python==3.10.12

So, it seems my crash earlier was unrelated. I can close my PR because even the tip of master worked as expected. It is feasible to bubble errors from micromamba or conda ?

[romain-intel]

For some context, micromamba/conda/mamba will work fine with something like ==<=x.y.z. We have been doing this for a while so I am not 100% sure what happened but I don't think stripping out the == is a solution to whatever problem you were having.