Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix shared library rpath once for all #7096

Merged
merged 1 commit into from
Nov 27, 2024
Merged

Fix shared library rpath once for all #7096

merged 1 commit into from
Nov 27, 2024

Conversation

larryliu0820
Copy link
Contributor

@larryliu0820 larryliu0820 commented Nov 26, 2024

Summary:
Fixes #7086
Based on this wiki:

https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling

Takeaway of reading this doc:

  • Set CMAKE_BUILD_WITH_INSTALL_RPATH to False so that we don't add install path (pip-out, cmake-out) into RPATH.
  • Set CMAKE_INSTALL_RPATH to be able to refer to libraries in the same install directory if any.
  • Set CMAKE_INSTALL_RPATH_USE_LINK_PATH to True so that we include other so files outside of the build tree can be added to RPATH.

Test Plan:

Tested on Colab:

!mkdir et; cd et; git clone https://github.com/pytorch/executorch.git
!cd et/executorch; git submodule update --init

# need to modify Codegen.cmake to make the following working
!cd et/executorch; bash install_requirements.sh
!cd et/executorch; CMAKE_INSTALL_PREFIX=/usr/local/lib/python3.10/dist-packages python setup.py bdist_wheel

Reviewers:

Subscribers:

Tasks:

Tags:

Summary

[PLEASE REMOVE] See CONTRIBUTING.md's Pull Requests for ExecuTorch PR guidelines.

[PLEASE REMOVE] If this PR closes an issue, please add a Fixes #<issue-id> line.

[PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: " label. For a list of available release notes labels, check out CONTRIBUTING.md's Pull Requests.

Test plan

[PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable.

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
Copy link

pytorch-bot bot commented Nov 26, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7096

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Cancelled Job

As of commit 8b55993 with merge base 2a292c3 (image):

CANCELLED JOB - The following job was cancelled. Please retry:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 26, 2024
@larryliu0820 larryliu0820 added topic: not user facing ciflow/binaries/all Release PRs with this label will build wheels for all python versions labels Nov 26, 2024
@larryliu0820 larryliu0820 changed the title Test Fix shared library rpath once for all Nov 26, 2024
Copy link
Contributor

@kirklandsign kirklandsign left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@kirklandsign
Copy link
Contributor

That's much cleaner!

@larryliu0820 larryliu0820 merged commit dedf77b into main Nov 27, 2024
110 of 121 checks passed
@larryliu0820 larryliu0820 deleted the test-rpath branch November 27, 2024 00:43
larryliu0820 added a commit that referenced this pull request Dec 3, 2024
Summary: This is a followup to #7096. That PR was aiming to rely on
`CMAKE_INSTALL_RPATH_USE_LINK_PATH` to set RPATH automatically. However
due to the caveat that we are not installing `_portable_lib.so` into the
correct path in CMake stage (we move the .so in setup.py after it's
installed), we are not able to add the correct RPATH to
libcustom_ops_aot_lib.so.

This PR still set the INSTALL_RPATH manually and adds TODOs to fix it
later.

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
larryliu0820 added a commit that referenced this pull request Dec 3, 2024
* [pybind] Fix rpath for custom ops

Summary: This is a followup to #7096. That PR was aiming to rely on
`CMAKE_INSTALL_RPATH_USE_LINK_PATH` to set RPATH automatically. However
due to the caveat that we are not installing `_portable_lib.so` into the
correct path in CMake stage (we move the .so in setup.py after it's
installed), we are not able to add the correct RPATH to
libcustom_ops_aot_lib.so.

This PR still set the INSTALL_RPATH manually and adds TODOs to fix it
later.

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

* Remove typo

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ciflow/binaries/all Release PRs with this label will build wheels for all python versions ciflow/binaries ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. topic: not user facing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pip install executorch doesn't install llama custom ops
3 participants