Skip to content

Update manylinux build scripts #10467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Update manylinux build scripts #10467

wants to merge 5 commits into from

Conversation

snnn
Copy link
Member

@snnn snnn commented Feb 3, 2022

Description:
Update manylinux build scripts

Motivation and Context

  • Why is this change required? What problem does it solve?

In preparation for #9782. It is just to keep the dependencies of our build environment up to date. We don't have to do it unless there are known security vulnerabilities in the tools we use(like autoconf/automake/patchelf), however, it is also not harmful if we do so. Our default CPU build always use the latest. So, changes like this just impact the other packages like CUDA/TRT/... .

I plan to add musl support and get #8754 merged after this one is done.

  • If it fixes an open issue, please link to the issue here.

@snnn snnn requested a review from pranavsharma February 4, 2022 01:02
RUN export SQLITE_AUTOCONF_ROOT=sqlite-autoconf-3360000 && \
export SQLITE_AUTOCONF_HASH=bd90c3eb96bee996206b83be7065c9ce19aef38c3f4fb53073ada0d0b69bbce3 && \
export SQLITE_AUTOCONF_DOWNLOAD_URL=https://www.sqlite.org/2021 && \
RUN export SQLITE_AUTOCONF_ROOT=sqlite-autoconf-3370200 && \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we put these variable definitions into a common place so that we don't need to update different files when the version changes?

e.g., RUN source export_sqlite_vars.sh && manylinux-entrypoint /build_scripts/build-sqlite3.sh

export_sqlite_vars.sh:

export SQLITE_AUTOCONF_ROOT=sqlite-autoconf-3370200
export SQLITE_AUTOCONF_HASH=4089a8d9b467537b3f246f217b84cd76e00b1d1a971fe5aca1e30e230e46b2d8 && \
export SQLITE_AUTOCONF_DOWNLOAD_URL=https://www.sqlite.org/2022

we'd also need to update the cgmanifest file generation

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried it. Not very good. Currently I just copy and paste the code from manylinux repo to ours. If there are too many Dockerfiles to modify, I can make a script. I already put makers in the docker files.

#Build manylinux2014 docker image begin
...
#Build manylinux2014 docker image end

If I extract the export commands to a separated file, I think it will need more human work to maintain.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can docker file include another docker file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you elaborate on how it's more work to maintain?
I counted 11 instances of SQLITE_AUTOCONF_ROOT in the changes (different Dockerfiles). Changing it in one place seems less error-prone and easier to review.
Updating them with a script is a good idea too.

@yuslepukhin
Copy link
Member

It would be nice to have a one-two lines descritpion such as advancing commits for this and that due to so it sets a frame of mind.
Some time from here we would be able to understand quickly why this was done.

@stale
Copy link

stale bot commented Apr 16, 2022

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.

@stale stale bot added the stale issues that have not been addressed in a while; categorized by a bot label Apr 16, 2022
@snnn
Copy link
Member Author

snnn commented Apr 18, 2022

Will close it. It's better to keep them up to date, which means do such things regularly. But it is not required.

@snnn snnn closed this Apr 18, 2022
@snnn snnn deleted the snnn/manylinux_update branch April 18, 2022 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale issues that have not been addressed in a while; categorized by a bot
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants