Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass through MAKEFLAGS and exclude from environment.bz2 #1364

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

zmedico
Copy link
Member

@zmedico zmedico commented Aug 5, 2024

Allow the MAKEFLAGS environment variable to pass through, in case a centralized GNU Make POSIX Jobserver is available. In order to prevent persistence of this variable in environment.bz2, exclude it when the __save_ebuild_env --exclude-init-phases argument is given.

Ultimately we may want to add support for portage to parse MAKEFLAGS and use it to allocate job tokens in various circumstances. For example, emerge could allocate a job token for each job started for emerge --jobs. This would remove a job token from the pool that is available to nested make calls, but is reasonable because nested make calls will execute jobs serially when no jobserver tokens remain.

Bug: https://bugs.gentoo.org/692576

@mgorny
Copy link
Member

mgorny commented Aug 5, 2024

Wouldn't this pass option like -k though, too?

@zmedico zmedico force-pushed the bug_692576_jobserver_makeflags_passthrough branch from af43541 to b3ea129 Compare August 5, 2024 05:01
@zmedico

This comment was marked as outdated.

@@ -598,7 +598,8 @@ def doebuild_environment(
)
mysettings.features.remove(feature)

if "MAKEOPTS" not in mysettings:
# MAKEOPTS conflicts with MAKEFLAGS, so skip this if MAKEFLAGS exists.
if "MAKEOPTS" not in mysettings and "MAKEFLAGS" not in mysettings:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The duplication of this with the GNUMAKEFLAGS + MAKEFLAGS condition below is suspicious?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that second "MAKEFLAGS" not in mysettings became redundant so I've removed it now. Thanks!

Allow the MAKEFLAGS environment variable to pass through, in case a
centralized GNU Make POSIX Jobserver is available. In order to prevent
persistence of this variable in environment.bz2, exclude it when the
__save_ebuild_env --exclude-init-phases argument is given.

Ultimately we may want to add support for portage to parse MAKEFLAGS
and use it to allocate job tokens in various circumstances. For
example, emerge could allocate a job token for each job started for
emerge --jobs. This would remove a job token from the pool that
is available to nested make calls, but is reasonable because nested
make calls will execute jobs serially when no jobserver tokens remain.

Bug: https://bugs.gentoo.org/692576
Signed-off-by: Zac Medico <[email protected]>
@zmedico zmedico force-pushed the bug_692576_jobserver_makeflags_passthrough branch from b3ea129 to f7494ea Compare August 5, 2024 05:44
@mgorny
Copy link
Member

mgorny commented Aug 5, 2024

Wouldn't this pass option like -k though, too?

Yes, in fact -k is considered a "simple" option because it has no argument, and make represents these simple options as a group of letters at the beginning of MAKEFLAGS like MAKEFLAGS="k -j4 -l4 --jobserver-auth=fifo:/tmp/GMfifo30205".

If there are no simple options then MAKEFLAGS begins with a space.

I mean, passing arbitrary options sounds dangerous and undesirable. If someone currently runs emerge via make -k as part of some bigger script, I don't think we should pass that one to make invocations within package builds.

a \fB--jobserver-auth=fifo:PATH\fR flag to specify the path of the
centralized jobserver fifo, which needs to be readable and writable by
the portage group when userpriv is enabled.
.TP
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we parse MAKEFLAGS then we can automatically add a sandbox exemption for the jobserver fifo. Otherwise users will have to do this themselves or put the fifo in an unsandboxed location like $PORTAGE_TMPDIR (they might have to do that for sesandbox too).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this pass option like -k though, too?

Yes, in fact -k is considered a "simple" option because it has no argument, and make represents these simple options as a group of letters at the beginning of MAKEFLAGS like MAKEFLAGS="k -j4 -l4 --jobserver-auth=fifo:/tmp/GMfifo30205".
If there are no simple options then MAKEFLAGS begins with a space.

I mean, passing arbitrary options sounds dangerous and undesirable. If someone currently runs emerge via make -k as part of some bigger script, I don't think we should pass that one to make invocations within package builds.

If we parse MAKEFLAGS in order to add a sandbox exemption for the fifo, I suppose we might filter out the simple options while we're there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants