Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scamper pid leak #81

Open
yachang opened this issue Nov 2, 2020 · 3 comments
Open

scamper pid leak #81

yachang opened this issue Nov 2, 2020 · 3 comments
Labels

Comments

@yachang
Copy link
Contributor

yachang commented Nov 2, 2020

There was a pid leak detected by mlab3.fln01

https://github.com/m-lab/ops-tracker/issues/1204

After investigation of the log, the pid leak was caused by scamper after scamper-daeon failed:

Screen Shot 2020-11-02 at 3 08 51 PM

@yachang
Copy link
Contributor Author

yachang commented Nov 2, 2020

You can see the green line (scamper daemon died), then scamper was brought up, the pid leak started.

After about 11 hours, the pid leak caused the crash of the evrything.

Before we nail down the pid leak in scamper, we will replace the flag

"scamper-daemon-with-scamper-backup"

with

"scamper-daemon"

k8s-support PR following.

@yachang
Copy link
Contributor Author

yachang commented Nov 2, 2020

@stephen-soltesz
Copy link
Contributor

The PID count over the same time period as "scamper" was active in the image above.

Screen Shot 2020-11-02 at 4 29 15 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants