Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

emqx: upgrade to 5.1.4 and fix linkage issue #139582

Closed
wants to merge 1 commit into from

Conversation

id
Copy link
Contributor

@id id commented Aug 15, 2023

  • Have you followed the guidelines for contributing?
  • Have you ensured that your commits follow the commit style guide?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

@id id requested review from MikeMcQuaid and a team as code owners August 15, 2023 11:15
@github-actions github-actions bot added workflows PR modifies GitHub Actions workflow files automerge-skip `brew pr-automerge` will skip this pull request labels Aug 15, 2023
.github/workflows/tests.yml Outdated Show resolved Hide resolved
@id
Copy link
Contributor Author

id commented Aug 15, 2023

This library should be copied verbatim from erlang@25 install, but something overwrites load command adding 2 extra ../../ in the process.

$ otool -l $(brew --prefix emqx)/lib/crypto-5.1.4/priv/lib/crypto.so
Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 104
         name @loader_path/../../../../../../../../../opt/openssl@3/lib/libcrypto.3.dylib (offset 24)
   time stamp 2 Wed Dec 31 16:00:02 1969
      current version 3.0.0
compatibility version 3.0.0

Formula/emqx.rb Outdated Show resolved Hide resolved
@carlocab
Copy link
Member

This library should be copied verbatim from erlang@25 install, but something overwrites load command adding 2 extra ../../ in the process.

Well, not quite. The issue is that crypto.so lives in

lib/erlang/lib/crypto-5.1.4/priv/lib

while emqx copies the crypto.so from erlang@25 into

lib/crypto-5.1.4/priv/lib

which is two directories shorter (and explains the extra ../../).

You can't just relocate binaries arbitrarily and expect them to keep working.

@github-actions github-actions bot removed the automerge-skip `brew pr-automerge` will skip this pull request label Aug 15, 2023
@id
Copy link
Contributor Author

id commented Aug 15, 2023

while emqx copies the crypto.so from erlang@25 into

the error have been occurring before this patch.

@id
Copy link
Contributor Author

id commented Aug 15, 2023

@carlocab erlang's crypto.so uses absolute path to load libcrypto.3.dylib, while in emqx it somehow got changed to relative.

@carlocab
Copy link
Member

carlocab commented Aug 15, 2023

erlang's crypto.so uses absolute path to load libcrypto.3.dylib

Not in CI, it doesn't.

❯ otool -L "$(brew --prefix erlang@25)/lib/erlang/lib/crypto-5.1.4/priv/lib/crypto.so"
/usr/local/opt/erlang@25/lib/erlang/lib/crypto-5.1.4/priv/lib/crypto.so:
        @loader_path/../../../../../../../../../opt/openssl@3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

@id
Copy link
Contributor Author

id commented Aug 15, 2023

Interesting, for me it was showing this:

/opt/homebrew//Cellar/erlang@25/25.3.2.5/lib/erlang/lib/crypto-5.1.4/priv/lib/crypto.so:
        /opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib (compatibility version 3.0.0, current version 3.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

@id
Copy link
Contributor Author

id commented Aug 15, 2023

different behaviour on arm/intel?

@id
Copy link
Contributor Author

id commented Aug 15, 2023

@carlocab was there some recent change in brew which started overwriting load path to dynamic libraries to relative?

@id id force-pushed the emqx-fix-linkage-issue branch 2 times, most recently from bcbcd92 to faf9394 Compare August 15, 2023 20:06
@id
Copy link
Contributor Author

id commented Aug 15, 2023

Ok, solved it.
Please let me know guys if you like this solution or we need to do something else.

Copy link
Member

@carlocab carlocab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, one suggestion.

Also, I noticed that the build vendors a whole lot of things that Homebrew provides (e.g. lz4, rocksdb, etc.). We should change that given https://docs.brew.sh/Acceptable-Formulae#stuff-that-requires-vendored-versions-of-homebrew-formulae.

Formula/e/emqx.rb Outdated Show resolved Hide resolved
@MikeMcQuaid MikeMcQuaid removed their request for review August 16, 2023 09:04
Formula/e/emqx.rb Outdated Show resolved Hide resolved
Formula/e/emqx.rb Outdated Show resolved Hide resolved
@id
Copy link
Contributor Author

id commented Aug 16, 2023

I noticed that the build vendors a whole lot of things that Homebrew provides (e.g. lz4, rocksdb, etc.). We should change that given docs.brew.sh/Acceptable-Formulae#stuff-that-requires-vendored-versions-of-homebrew-formulae.

Thanks, brought this up with the team. It's not a trivial issue to solve, but I agree it's reasonable to expect from a library to make use of components already present in the system, if possible.

carlocab
carlocab previously approved these changes Aug 16, 2023
Copy link
Member

@carlocab carlocab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; thanks!

SMillerDev
SMillerDev previously approved these changes Aug 21, 2023
@github-actions
Copy link
Contributor

⛔ It looks like @BrewTestBot cannot push to your PR branch. For future pull requests, please allow maintainers to edit your PR to simplify the merge process.

@github-actions github-actions bot added the no push access Maintainers cannot push to this pull request. label Aug 21, 2023
@github-actions
Copy link
Contributor

🤖 An automated task has requested creation of a replacement PR.

@github-actions github-actions bot dismissed stale reviews from carlocab and SMillerDev August 21, 2023 09:10

Replacement PR dispatched

@github-actions
Copy link
Contributor

⚠️ Replacement PR creation failed. CC @carlocab

@id
Copy link
Contributor Author

id commented Aug 21, 2023

Apparently I cannot allow maintainers to edit PR because fork is owned by organization: https://github.com/orgs/community/discussions/5634.

Should I fork homebrew-core into my personal acc and create another PR?

@id
Copy link
Contributor Author

id commented Aug 23, 2023

--> #140246

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no push access Maintainers cannot push to this pull request. workflows PR modifies GitHub Actions workflow files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants