Create a Embeddable package to monitor and profile go tools/apps

[tarunKoyalwar]

Proposed Changes

• To easily detect and share performance issues we need a library that effectively profiles application and monitors itself without any external dependency

Tasks

Preview Give feedback

1. Env Based Input ( we can reuse existing logic in pprof package but cpuprofile might need to be improved)
2. When this package is embedded it should launch a seperate process ( with appropriate syscall attributes to make it completely indedepent of current process ) and monitor/profile and save resulting data in cache directory . similar to screen command (Ref: Detach Helper #346)
3. Detection logic should be based on below logic
4. Add Endpoint in PDTM API to support uploading and generating uuid of these profiles for effective sharing and ease of analysis

Detection Logic ( Task 3)

• continious monitering of Process MAX RSS with time as key
• profiling on every tick with time as key
• by plotting MAX RSS over time we can find inflection points and filter out IMP/CRITICAL profiles and share them instead of sharing all xGB of profiles

References

• OOM killed when using the nuclei SDK with the standard templates nuclei#4756
• memory leak fixes and optimizations nuclei#4680

[stan-threatmate]

What worked for me while debugging nuclei is this approach. It will dump the goroutines stack trace and memory allocations on a regular interval until stopped.

• pprof.WriteHeapProfile()
• runtime.ReadMemStats(&memStat)
• pprof.Lookup("goroutine").WriteTo(f, 0)

Running these on an interval will create a new dump file every X minutes which might be what you need. You can then inspect the memory and goroutine stack trace using go tool pprof file.dump