Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add environment variable for tracking child processes #84

Merged
merged 2 commits into from
Aug 16, 2022

Conversation

Kobzol
Copy link
Contributor

@Kobzol Kobzol commented Aug 13, 2022

Hi! We want to add bytehound to the rustc-perf suite for profiling the Rust compiler. However, the compiler is usually invoked through cargo, which executes many rustc child processes underneath. In theory this could be resolved with some RUSTC_WRAPPER hacks, but I thought that it would be better if we could directly teach bytehound to track child processes.

I'm not sure what is the interaction of the added flag with MEMORY_PROFILER_OUTPUT, if it's set to a single, fixed filepath without placeholders, then the profile could be overwritten by the child processes? But that's probably user error.

@koute
Copy link
Owner

koute commented Aug 13, 2022

The #80 issue isn't really related; these are two entirely different use cases:

  1. Profile a process after fork without an exec. (Which is what the person in the issue wants to do, and this is really tricky to implement correctly because fork can be called essentially at any time, so all of the global state is in an unspecified state and needs to be reinitialized.)
  2. Profile a process after fork with an exec. (Which is what you're talking about, and should be relatively easy to do since execing a process will reset the global state for us in the child.)

I'm not sure what is the interaction of the added flag with MEMORY_PROFILER_OUTPUT, if it's set to a single, fixed filepath without placeholders, then the profile could be overwritten by the child processes? But that's probably user error.

Correct. This is an user error. We could probably try to detect that and print an error or something, but it's not critical.

pid_t pid = fork();
if( pid == 0 ) {
// Child
if (execl("/usr/bin/ls", "/usr/bin/ls", NULL) == -1) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you use the basic.c program here instead and in the integration test verify that:

  1. the file for the parent is created and contains the allocations we expect,
  2. the file for the child is created and contains the allocations we expect (you can just grab the checks from #[test] for the basic.c, move it into a separate functions, and then call it in two places)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, I changed the test to incorporate the basic program.

Comment on lines 187 to 188
If set to `1`, bytehound will also track the memory allocations of child processes spawned by the
profiled process.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explicitly also document that:

  1. this only applies to child processes which are forkd and execd,
  2. that the MEMORY_PROFILER_OUTPUT needs to be set with the appropriate placeholders so that the filenames of both the parent and the child are different

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@koute koute merged commit 88897fc into koute:master Aug 16, 2022
@koute
Copy link
Owner

koute commented Aug 16, 2022

LGTM; thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants