Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyPI release is not possible with direct references #160

Open
pjbull opened this issue Oct 27, 2021 · 4 comments
Open

PyPI release is not possible with direct references #160

pjbull opened this issue Oct 27, 2021 · 4 comments
Labels
dependencies Pull requests that update a dependency file

Comments

@pjbull
Copy link
Member

pjbull commented Oct 27, 2021

We use direct references for pytorchvideo, densepose, and yolox in our requirements:

pytorchvideo @ git+https://github.com/facebookresearch/pytorchvideo

yolox @ git+https://github.com/r-b-g-b/YOLOX.git@build-requirements

detectron2-densepose @ git+https://github.com/facebookresearch/detectron2@main#subdirectory=projects/DensePose

The official PyPI does not support direct references. Per PEP-440:

Public index servers SHOULD NOT allow the use of direct references in uploaded distributions. Direct references are intended as a tool for software integrators rather than publishers.

This means that we can't publish to PyPI until our dependencies are free of direct references. The best way to do so is to have maintainers of those packages publish on PyPI. Here's the current status:

Plan for now is to have documentation install from github, and to try to push the dependency packages forward to a happy resolution.

When fixing:

  • We have a publish to PyPI workflow in our release.yml that is commented out until we can resolve this issue
  • We have a PyPI badge that in the README.md that is commented out as well
@ejm714
Copy link
Collaborator

ejm714 commented Apr 21, 2022

yolox is now pip-installable 🎉 as of release on 2022/01/18: https://github.com/Megvii-BaseDetection/YOLOX/releases

@jayqi
Copy link
Member

jayqi commented Mar 3, 2023

I have a somewhat 🤨 idea regarding detectron2: What if we publish detectron2 to PyPI?

We could either:

  • Just claim the detectron2 name, which is open.
  • Or, name it something else like detectron2-drivendata. Searching for detectron2 on PyPI, it seems like this was the solution adopted by several other people. (1, 2, 3).

This will take a little work to maintain but should be mostly automateable with CI.

If we do claim the detectron2 name, we could just be nice about releasing it back to FAIR if they ever change their mind about publishing to PyPI.

@ejm714
Copy link
Collaborator

ejm714 commented Mar 4, 2023

yolox is now pip-installable 🎉 as of release on 2022/01/18: https://github.com/Megvii-BaseDetection/YOLOX/releases

I tried pip installing yolox and it seems like they still have an improperly handled dependency.

Collecting yolox
  Downloading yolox-0.3.0.tar.gz (79 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.0/80.0 kB 2.6 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [9 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/ft/5tzjjc1d0ysf14360lvrp3d80000gn/T/pip-install-ydc9nz72/yolox_84fe5f27c10f4653a9d63b48b65c31ff/setup.py", line 77, in <module>
          ext_modules=get_ext_modules(),
        File "/private/var/folders/ft/5tzjjc1d0ysf14360lvrp3d80000gn/T/pip-install-ydc9nz72/yolox_84fe5f27c10f4653a9d63b48b65c31ff/setup.py", line 50, in get_ext_modules
          assert TORCH_AVAILABLE, "torch is required for pre-compiling ops, please install it first."
      AssertionError: torch is required for pre-compiling ops, please install it first.
      [WARNING] Unable to import torch, pre-compiling ops will be disabled.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

@jayqi
Copy link
Member

jayqi commented Mar 4, 2023

It looks like yolox 0.2.0 (2022-01-17) had manylinux wheels on PyPI, which meant you could pip install without torch on Linux.

However, yolox 0.3.0 (2022-04-22) went back to only having source distributions, so you have to build it when you install.

@klwetstone klwetstone added docs dependencies Pull requests that update a dependency file labels Apr 17, 2024
@ejm714 ejm714 removed the docs label Sep 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

No branches or pull requests

4 participants