Skip to content

Latest commit

 

History

History
72 lines (53 loc) · 1.88 KB

singlethreadhighmemusage.md

File metadata and controls

72 lines (53 loc) · 1.88 KB

MemoryLeak

Our dockerized Java application JVM Heap usage sometimes increases too much We can check it from JVM Prometheus Exporter.

  1. Run Test App
docker run -it --rm  pamir/jvm-cases SingleThreadHighMemoryUsage
  1. Check, that our app is not idle(%CPU > 0.0):
top -c -p $(pgrep -d',' -f java)

Steps To Analyze

  1. Get id of container with our application(java application.jar ...):
docker ps
  1. Run container with jattach utility:
docker run --rm -it \
	--net=container:8b0970e1aa99 \
	--pid=container:8b0970e1aa99 \
	-v /tmp:/tmp \
	--privileged \
	adriantodt/alpine-zlib-jattach \
	/bin/sh
  1. Dump Heap with jattach
jattach 1 heapdump /tmp/memoryleak1.hprof
jattacj 1 heapdump /tmp/memoryleak2.hprof
  1. Be careful the dump files are not in in on jattach container. They are on java container's tmp directory
ls -lart /tmp
  1. Copy Dump files from java container to host machine
docker cp 8b0970e1aa99:/tmp/memoryleak1.hprof /tmp/memoryleak1.hprof
docker cp 8b0970e1aa99:/tmp/memoryleak2.hprof /tmp/memoryleak2.hprof

  1. Download Memory Dump Analzyer Tool

    http://www.eclipse.org/mat/

  2. Open Histogram and sort by retained heap size

  3. Investigate suspected thread which pressures memory

  4. Find out what is consuming too much memory. In this type of scenerio UsernamePassword String array can make memory pressure. This pattern is Single Thread High Memory Usage. I have seen this too much when developers forget to add where clause. We can extract the query from the heap dump in such cases

Todo

  • drop capabilities
  • Add Reference to Charlie Hunt Book Java Performance
  • Refer to Source Code
  • Convert this simple to hibernate/MYSQL scenrio