Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fatal Error has been occurring fequently #308

Open
Bhumika-Khunt opened this issue Feb 22, 2023 · 4 comments
Open

Fatal Error has been occurring fequently #308

Bhumika-Khunt opened this issue Feb 22, 2023 · 4 comments
Labels
bug Something isn't working

Comments

@Bhumika-Khunt
Copy link

Bhumika-Khunt commented Feb 22, 2023 •
edited by eastig
Loading

Hi, I am experiencing JVM crash frequently.

# A fatal error has been detected by the Java Runtime Environment:
# Native memory allocation (mprotect) failed to protect 16384 bytes for memory to guard stack pages
#  Internal Error (thread.cpp:2635), pid=3418211, tid=2090575
#  Error: memory to guard stack pages
#
# JRE version: OpenJDK Runtime Environment Corretto-11.0.18.10.1 (11.0.18+10) (build 11.0.18+10-LTS)
# Java VM: OpenJDK 64-Bit Server VM Corretto-11.0.18.10.1 (11.0.18+10-LTS, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Core dump will be written. Default location: Core dumps may be processed with "/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h %e" (or dumping to /datadrive/MDM/core.3418211)
#
# If you would like to submit a bug report, please visit:
#   https://github.com/corretto/corretto-11/issues/
#
---------------  S U M M A R Y ------------

Command Line: -XX:+HeapDumpOnOutOfMemoryError -XX:+UseGCOverheadLimit -XX:+OptimizeStringConcat -XX:+UseStringDeduplication -Xms2g -Xmn1g -Xss256K -XX:+UseAdaptiveSizePolicy -XX:-UseBiasedLocking -XX:+DisableExplicitGC -XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -XX:HeapDumpPath=shaft2/logs -Xlog:gc*,gc+ref=debug,gc+heap=debug,gc+age=trace:file=gc-%p-%t.log:tags,uptime,time,level:filecount=10,filesize=50m -Xmx1g -Djava.awt.headless=true -Xmx8g -Djava.awt.headless=true -Dsling.run.modes=oak_tar,shaft,prod -agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n shaft2.jar -p 6505 -c shaft2

Host: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz, 4 cores, 27G, Red Hat Enterprise Linux release 8.7 (Ootpa)
Time: Sun Feb 12 11:56:24 2023 UTC elapsed time: 1018438.016444 seconds (11d 18h 53m 58s)

---------------  T H R E A D  ---------------

Current thread (0x00007f901e30c1e0):  JavaThread "oak-executor-47472" daemon [_thread_new, id=2090575, stack(0x00007f8843e8a000,0x00007f8843ecb000)]

Stack: [0x00007f8843e8a000,0x00007f8843ecb000],  sp=0x00007f8843ec9a30,  free space=254k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xf4b40a]  VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x1ca
V  [libjvm.so+0xf4c2cb]  VMError::report_and_die(Thread*, char const*, int, unsigned long, VMErrorType, char const*, __va_list_tag*)+0x2b
V  [libjvm.so+0x6d6e55]  report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*, ...)+0xd5
V  [libjvm.so+0xeddb12]  JavaThread::run()+0x342
V  [libjvm.so+0xeda3ef]  Thread::call_run()+0x14f
V  [libjvm.so+0xc78006]  thread_native_entry(Thread*)+0xe6


---------------  P R O C E S S  ---------------

Threads class SMR info:
_java_thread_list=0x00007f90768e66b0, length=32558, elements={
0x00007f90c8707380, 0x00007f90c870ad70, 0x00007f90c8724180, 0x00007f90c8725980,
0x00007f90c8727360, 0x00007f90c8728ca0, 0x00007f90c872a4f0, 0x00007f90c87c7500,
0x00007f90c87cb570, 0x00007f90c87d4cd0, 0x00007f90c89ae0a0, 0x00007f90c8ed36f0,
0x00007f90c92bc4d0, 0x00007f9030032c50, 0x00007f9030034170, 0x00007f9030050a50,
0x00007f9030052ac0, 0x00007f9050001610, 0x00007f9050002b50, 0x00007f90500034c0,
0x00007f90500049c0, 0x00007f903009a590, 0x00007f9030146970, 0x00007f9030147a10,
0x00007f90301491e0, 0x00007f9050005ec0, 0x00007f90301f1580, 0x00007f90302451a0,
0x00007f90304f37c0, 0x00007f90305fa2f0, 0x00007f903060a1b0, 0x00007f90280076e0,
0x00007f9030709200, 0x00007f90307065b0, 0x00007f90306f71c0, 0x00007f903068f9a0,
0x00007f9024012c00, 0x00007f9030703e60, 0x00007f9024011a50, 0x00007f9030718ef0,
0x00007f9030705460, 0x00007f903070af40, 0x00007f903071bcd0, 0x00007f9030713540,
0x00007f9030747550, 0x00007f902c006030, 0x00007f90306f92b0, 0x00007f902c007660,
0x00007f903072aa40, 0x00007f902c008c80, 0x00007f90307278e0, 0x00007f90307720b0,
0x00007f90307ef410, 0x00007f9050018470, 0x00007f9030a98d50, 0x00007f90301ba210,
0x00007f9030ed2a80, 0x00007f90311e29b0, 0x00007f903128fa50, 0x00007f90315552d0,
0x00007f90c8029bb0, 0x00007f90ac0fd600, 0x00007f90ac1fba80, 0x00007f90ac211280,
0x00007f9088207a40, 0x00007f909000e9e0, 0x00007f90881ffde0, 0x00007f9088200900,
0x00007f9088209460, 0x00007f908820ab10, 0x00007f90ac3c7370, 0x00007f90ac472230,
0x00007f90ac48dd30, 0x00007f90ac612990, 0x00007f90ac620810, 0x00007f90ac626820,
0x00007f90ac627ea0, 0x00007f90ac64dda0, 0x00007f9070004190, 0x00007f906843ac00,
0x00007f9058004bb0, 0x00007f9058007f20, 0x00007f90580090e0, 0x00007f9058025c70,
0x00007f9068ce1d60, 0x00007f906ce9ce90, 0x00007f906ce9dc10, 0x00007f906c26a030,
0x00007f906c26b230, 0x00007f9068bbcb60, 0x00007f9084086f80, 0x00007f908402a9b0,
0x00007f90840319b0, 0x00007f9084032b70, 0x00007f906ce6cca0, 0x00007f901c01d0a0,
0x00007f9044d750b0, 0x00007f9044d97a90, 0x00007f90ca0ce3b0, 0x00007f9060d4f780,
0x00007f9060d4fc90, 0x00007f9060d54c50, 0x00007f9060d55680, 0x00007f90ca0ce8c0,

Java Threads: ( => current thread )
  0x00007f90c8707380 JavaThread "Reference Handler" daemon [_thread_blocked, id=3418220, stack(0x00007f90b00ac000,0x00007f90b00ed000)]
  0x00007f90c870ad70 JavaThread "Finalizer" daemon [_thread_blocked, id=3418221, stack(0x00007f90b006b000,0x00007f90b00ac000)]
  0x00007f90c8724180 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=3418222, stack(0x00007f9083547000,0x00007f9083588000)]
  0x00007f90c8725980 JavaThread "Service Thread" daemon [_thread_blocked, id=3418223, stack(0x00007f9083506000,0x00007f9083547000)]
  0x00007f90c8727360 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=3418224, stack(0x00007f9083405000,0x00007f9083506000)]
  0x00007f90c8728ca0 JavaThread "C1 CompilerThread0" daemon [_thread_blocked, id=3418225, stack(0x00007f9083304000,0x00007f9083405000)]
  0x00007f90c872a4f0 JavaThread "Sweeper thread" daemon [_thread_blocked, id=3418226, stack(0x00007f90832c3000,0x00007f9083304000)]
  0x00007f90c87c7500 JavaThread "JDWP Transport Listener: dt_socket" daemon [_thread_in_native, id=3418227, stack(0x00007f908307c000,0x00007f90830bd000)]
  0x00007f90c87cb570 JavaThread "JDWP Event Helper Thread" daemon [_thread_blocked, id=3418228, stack(0x00007f908303b000,0x00007f908307c000)]
  0x00007f90c87d4cd0 JavaThread "Common-Cleaner" daemon [_thread_blocked, id=3418230, stack(0x00007f9082ef8000,0x00007f9082f39000)]
  0x00007f90c89ae0a0 JavaThread "FelixDispatchQueue" [_thread_blocked, id=3418232, stack(0x00007f908288e000,0x00007f90828cf000)]
  0x00007f90c8ed36f0 JavaThread "FelixFrameworkWiring" daemon [_thread_blocked, id=3418238, stack(0x00007f9082448000,0x00007f9082489000)]
  0x00007f90c92bc4d0 JavaThread "FelixStartLevel" daemon [_thread_blocked, id=3418240, stack(0x00007f908199a000,0x00007f90819db000)]
  0x00007f9030032c50 JavaThread "CM Configuration Updater" daemon [_thread_blocked, id=3418241, stack(0x00007f9081959000,0x00007f908199a000)]
  0x00007f9030034170 JavaThread "CM Event Dispatcher" daemon [_thread_blocked, id=3418242, stack(0x00007f9081918000,0x00007f9081959000)]
  0x00007f9030050a50 JavaThread "EventAdminAsyncThread #1" daemon [_thread_blocked, id=3418243, stack(0x00007f90818d7000,0x00007f9081918000)]
  0x00007f9030052ac0 JavaThread "EventAdminAsyncThread #2" daemon [_thread_blocked, id=3418244, stack(0x00007f9081896000,0x00007f90818d7000)]
  0x00007f9050001610 JavaThread "EventAdminAsyncThread #3" daemon [_thread_blocked, id=3418246, stack(0x00007f9081814000,0x00007f9081855000)]
  0x00007f9050002b50 JavaThread "EventAdminAsyncThread #4" daemon [_thread_blocked, id=3418247, stack(0x00007f9081855000,0x00007f9081896000)]
  0x00007f90500034c0 JavaThread "EventAdminAsyncThread #5" daemon [_thread_blocked, id=3418248, stack(0x00007f90815d3000,0x00007f9081614000)]
  0x00007f90500049c0 JavaThread "EventAdminAsyncThread #6" daemon [_thread_blocked, id=3418249, stack(0x00007f90813ff000,0x00007f9081440000)]
  0x00007f903009a590 JavaThread "Activator#1" daemon [_thread_blocked, id=3418250, stack(0x00007f90813be000,0x00007f90813ff000)]
  0x00007f9030146970 JavaThread "EventAdminAsyncThread #7" daemon [_thread_blocked, id=3418251, stack(0x00007f9081592000,0x00007f90815d3000)]
  0x00007f9030147a10 JavaThread "EventAdminAsyncThread #8" daemon [_thread_blocked, id=3418252, stack(0x00007f9081551000,0x00007f9081592000)]
  0x00007f90301491e0 JavaThread "EventAdminAsyncThread #9" daemon [_thread_blocked, id=3418253, stack(0x00007f9081510000,0x00007f9081551000)]
  0x00007f9050005ec0 JavaThread "EventAdminAsyncThread #10" daemon [_thread_blocked, id=3418254, stack(0x00007f90814cf000,0x00007f9081510000)]
  0x00007f90301f1580 JavaThread "LogEntry Dispatcher" daemon [_thread_blocked, id=3418255, stack(0x00007f908148e000,0x00007f90814cf000)]
  0x00007f90302451a0 JavaThread "OsgiInstallerImpl" daemon [_thread_blocked, id=3418256, stack(0x00007f908144d000,0x00007f908148e000)]
  0x00007f90304f37c0 JavaThread "JMX OSGi Service State Event Dispatcher" daemon [_thread_blocked, id=3418260, stack(0x00007f9080c77000,0x00007f9080cb8000)]
  0x00007f90305fa2f0 JavaThread "qtp1877616181-41-acceptor-0@66457bd-ServerConnector@660b34a7{HTTP/1.1,[http/1.1]}{0.0.0.0:6505}" [_thread_in_native, id=3418263, stack(0x00007f90809b4000,0x00007f90809f5000)]
  0x00007f903060a1b0 JavaThread "Scheduler-417487992" [_thread_blocked, id=3418269, stack(0x00007f908062e000,0x00007f908066f000)]
  0x00007f90280076e0 JavaThread "EventAdminThread #1" daemon [_thread_blocked, id=3418271, stack(0x00007f90803a9000,0x00007f90803ea000)]
  0x00007f9030709200 JavaThread "EventAdminThread #2" daemon [_thread_blocked, id=3418272, stack(0x00007f9080368000,0x00007f90803a9000)]
  0x00007f90307065b0 JavaThread "EventAdminThread #3" daemon [_thread_blocked, id=3418273, stack(0x00007f9080327000,0x00007f9080368000)]
  0x00007f90306f71c0 JavaThread "EventAdminThread #4" daemon [_thread_blocked, id=3418274, stack(0x00007f90802e6000,0x00007f9080327000)]
  0x00007f903068f9a0 JavaThread "EventAdminThread #5" daemon [_thread_blocked, id=3418275, stack(0x00007f90802a5000,0x00007f90802e6000)]
  0x00007f9024012c00 JavaThread "EventAdminThread #6" daemon [_thread_blocked, id=3418276, stack(0x00007f9080264000,0x00007f90802a5000)]
  0x00007f9030703e60 JavaThread "EventAdminThread #7" daemon [_thread_blocked, id=3418277, stack(0x00007f9080223000,0x00007f9080264000)]
  0x00007f9024011a50 JavaThread "EventAdminThread #8" daemon [_thread_blocked, id=3418278, stack(0x00007f90801e2000,0x00007f9080223000)]
  0x00007f9030718ef0 JavaThread "EventAdminThread #9" daemon [_thread_blocked, id=3418279, stack(0x00007f90801a1000,0x00007f90801e2000)]
  0x00007f9030705460 JavaThread "EventAdminThread #10" daemon [_thread_blocked, id=3418280, stack(0x00007f9080160000,0x00007f90801a1000)]
  0x00007f903070af40 JavaThread "EventAdminThread #11" daemon [_thread_blocked, id=3418281, stack(0x00007f908011f000,0x00007f9080160000)]
  0x00007f903071bcd0 JavaThread "EventAdminThread #12" daemon [_thread_blocked, id=3418282, stack(0x00007f90800de000,0x00007f908011f000)]
  0x00007f9030713540 JavaThread "EventAdminThread #14" daemon [_thread_blocked, id=3418283, stack(0x00007f908009d000,0x00007f90800de000)]
  0x00007f9030747550 JavaThread "EventAdminThread #15" daemon [_thread_blocked, id=3418284, stack(0x00007f908005c000,0x00007f908009d000)]
  0x00007f902c006030 JavaThread "EventAdminThread #13" daemon [_thread_blocked, id=3418285, stack(0x00007f908001b000,0x00007f908005c000)]
  0x00007f90306f92b0 JavaThread "EventAdminThread #16" daemon [_thread_blocked, id=3418286, stack(0x00007f9023dbf000,0x00007f9023e00000)]
  0x00007f902c007660 JavaThread "EventAdminThread #17" daemon [_thread_blocked, id=3418287, stack(0x00007f9023d7e000,0x00007f9023dbf000)]
  0x00007f903072aa40 JavaThread "EventAdminThread #18" daemon [_thread_blocked, id=3418288, stack(0x00007f9023d3d000,0x00007f9023d7e000)]
  0x00007f902c008c80 JavaThread "EventAdminThread #19" daemon [_thread_blocked, id=3418289, stack(0x00007f9023cfc000,0x00007f9023d3d000)]
  0x00007f90307278e0 JavaThread "EventAdminThread #20" daemon [_thread_blocked, id=3418290, stack(0x00007f9023cbb000,0x00007f9023cfc000)]
  0x00007f90307720b0 JavaThread "Timer-1" daemon [_thread_blocked, id=3418291, stack(0x00007f9023c7a000,0x00007f9023cbb000)]
  0x00007f90307ef410 JavaThread "SCR Component Actor" daemon [_thread_blocked, id=3418302, stack(0x00007f90239f2000,0x00007f9023a33000)]
  0x00007f9050018470 JavaThread "JMX OSGi Bundle State Event Dispatcher" daemon [_thread_blocked, id=3418313, stack(0x00007f90234f9000,0x00007f902353a000)]
  0x00007f9030a98d50 JavaThread "pool-3-thread-1" [_thread_blocked, id=3418317, stack(0x00007f9023570000,0x00007f90235b1000)]
  0x00007f90301ba210 JavaThread "FileStore background tasks" daemon [_thread_blocked, id=3418341, stack(0x00007f90803ea000,0x00007f908042b000)]
  0x00007f9030ed2a80 JavaThread "ApacheSlingdefault_QuartzSchedulerThread" [_thread_blocked, id=3418346, stack(0x00007f8abac9f000,0x00007f8abace0000)]
  0x00007f90311e29b0 JavaThread "pool-8-thread-1" [_thread_blocked, id=3418356, stack(0x00007f8aba81d000,0x00007f8aba85e000)]
  0x00007f903128fa50 JavaThread "mysql-cj-abandoned-connection-cleanup" daemon [_thread_blocked, id=3418361, stack(0x00007f8aba7dc000,0x00007f8aba81d000)]
  0x00007f90315552d0 JavaThread "Tomcat JDBC Pool Cleaner[831925212:1675184556008]" daemon [_thread_blocked, id=3418366, stack(0x00007f8aba15a000,0x00007f8aba19b000)]
  0x00007f90c8029bb0 JavaThread "DestroyJavaVM" [_thread_blocked, id=3418212, stack(0x00007f90d190e000,0x00007f90d194f000)]
  0x00007f90ac0fd600 JavaThread "oak-repository-executor-1" daemon [_thread_blocked, id=3418373, stack(0x00007f8ab9617000,0x00007f8ab9658000)]
  0x00007f90ac1fba80 JavaThread "Apache Sling Resource Resolver Finalizer Thread" daemon [_thread_blocked, id=3418384, stack(0x00007f8ab9313000,0x00007f8ab9354000)]
  0x00007f90ac211280 JavaThread "Timer-4" daemon [_thread_blocked, id=3418385, stack(0x00007f8ab8f4c000,0x00007f8ab8f8d000)]
  0x00007f9088207a40 JavaThread "sling-default-1" [_thread_blocked, id=3418387, stack(0x00007f8ab8eca000,0x00007f8ab8f0b000)]
  0x00007f909000e9e0 JavaThread "JcrInstaller.3" daemon [_thread_blocked, id=3418388, stack(0x00007f8ab8f0b000,0x00007f8ab8f4c000)]
  0x00007f90881ffde0 JavaThread "sling-default-2" [_thread_blocked, id=3418395, stack(0x00007f8ab8e89000,0x00007f8ab8eca000)]
  0x00007f9088200900 JavaThread "sling-default-3" [_thread_blocked, id=3418396, stack(0x00007f8ab8e48000,0x00007f8ab8e89000)]
  0x00007f9088209460 JavaThread "sling-default-4" [_thread_blocked, id=3418397, stack(0x00007f8ab8e07000,0x00007f8ab8e48000)]
  0x00007f908820ab10 JavaThread "sling-default-5" [_thread_blocked, id=3418399, stack(0x00007f8ab8dc6000,0x00007f8ab8e07000)]
  0x00007f90ac3c7370 JavaThread "Apache Sling Scheduled Job Handler Thread" daemon [_thread_blocked, id=3418425, stack(0x00007f8ab8703000,0x00007f8ab8744000)]
  0x00007f90ac472230 JavaThread "Thread-56" daemon [_thread_blocked, id=3418426, stack(0x00007f8ab84c2000,0x00007f8ab8503000)]
  0x00007f90ac48dd30 JavaThread "Thread-57" daemon [_thread_blocked, id=3418427, stack(0x00007f8ab8481000,0x00007f8ab84c2000)]
  0x00007f90ac612990 JavaThread "Apache Sling Job Topology Listener Thread" daemon [_thread_blocked, id=3418563, stack(0x00007f8ab81ff000,0x00007f8ab8240000)]
  0x00007f90ac620810 JavaThread "discovery.connectors.common.runner.c16aec56-8b88-47ed-aa09-6635b2944f36.connectorPinger" daemon [_thread_blocked, id=3418580, stack(0x00007f8ab7bef000,0x00007f8ab7c30000)]
  0x00007f90ac626820 JavaThread "discovery.connectors.common.runner.c16aec56-8b88-47ed-aa09-6635b2944f36.discoveryLiteCheck" daemon [_thread_blocked, id=3418581, stack(0x00007f8ab7bae000,0x00007f8ab7bef000)]
  0x00007f90ac627ea0 JavaThread "Discovery-AsyncEventSender" daemon [_thread_blocked, id=3418582, stack(0x00007f8ab7b6d000,0x00007f8ab7bae000)]
  0x00007f90ac64dda0 JavaThread "pool-13-thread-1" [_thread_blocked, id=3418583, stack(0x00007f8ab7b2c000,0x00007f8ab7b6d000)]
  0x00007f9070004190 JavaThread "Scheduler-686390389" [_thread_blocked, id=3418613, stack(0x00007f90235b1000,0x00007f90235f2000)]
  0x00007f906843ac00 JavaThread "pool-15-thread-1" [_thread_in_native, id=3420184, stack(0x00007f908086f000,0x00007f90808b0000)]
  0x00007f9058004bb0 JavaThread "I/O dispatcher 1" [_thread_in_native, id=3420185, stack(0x00007f8ab9395000,0x00007f8ab93d6000)]
  0x00007f9058007f20 JavaThread "I/O dispatcher 2" [_thread_in_native, id=3420186, stack(0x00007f8ab7c71000,0x00007f8ab7cb2000)]
  0x00007f90580090e0 JavaThread "I/O dispatcher 3" [_thread_in_native, id=3420187, stack(0x00007f8ab7fb6000,0x00007f8ab7ff7000)]
  0x00007f9058025c70 JavaThread "I/O dispatcher 4" [_thread_in_native, id=3420188, stack(0x00007f8ab813c000,0x00007f8ab817d000)]
  0x00007f9068ce1d60 JavaThread "pool-16-thread-1" [_thread_in_native, id=3420358, stack(0x00007f8ab9354000,0x00007f8ab9395000)]
  0x00007f906ce9ce90 JavaThread "I/O dispatcher 5" [_thread_in_native, id=3420359, stack(0x00007f8ab7c30000,0x00007f8ab7c71000)]
  0x00007f906ce9dc10 JavaThread "I/O dispatcher 6" [_thread_in_native, id=3420360, stack(0x00007f8ab7ff7000,0x00007f8ab8038000)]
  0x00007f906c26a030 JavaThread "I/O dispatcher 7" [_thread_in_native, id=3420361, stack(0x00007f8ab7cb2000,0x00007f8ab7cf3000)]
  0x00007f906c26b230 JavaThread "I/O dispatcher 8" [_thread_in_native, id=3420362, stack(0x00007f8ab7ef3000,0x00007f8ab7f34000)]
  0x00007f9068bbcb60 JavaThread "pool-17-thread-1" [_thread_in_native, id=3420363, stack(0x00007f8ab81be000,0x00007f8ab81ff000)]
  0x00007f9084086f80 JavaThread "I/O dispatcher 9" [_thread_in_native, id=3420364, stack(0x00007f8ab7f34000,0x00007f8ab7f75000)]
  0x00007f908402a9b0 JavaThread "I/O dispatcher 10" [_thread_in_native, id=3420365, stack(0x00007f8ab7f75000,0x00007f8ab7fb6000)]
  0x00007f90840319b0 JavaThread "I/O dispatcher 11" [_thread_in_native, id=3420366, stack(0x00007f8ab8f8d000,0x00007f8ab8fce000)]
  0x00007f9084032b70 JavaThread "I/O dispatcher 12" [_thread_in_native, id=3420367, stack(0x00007f8ab817d000,0x00007f8ab81be000)]
  0x00007f906ce6cca0 JavaThread "pool-19-thread-1" [_thread_in_native, id=3431344, stack(0x00007f8ab8b85000,0x00007f8ab8bc6000)]
  0x00007f901c01d0a0 JavaThread "I/O dispatcher 13" [_thread_in_native, id=3431345, stack(0x00007f90808b0000,0x00007f90808f1000)]
  0x00007f901c0202e0 JavaThread "I/O dispatcher 14" [_thread_in_native, id=3431346, stack(0x00007f8ab80ba000,0x00007f8ab80fb000)]
  0x00007f901c01dcb0 JavaThread "I/O dispatcher 15" [_thread_in_native, id=3431347, stack(0x00007f8ab900f000,0x00007f8ab9050000)]
  0x00007f901c022120 JavaThread "I/O dispatcher 16" [_thread_in_native, id=3431348, stack(0x00007f8aba79b000,0x00007f8aba7dc000)]
  0x00007f906c1f41b0 JavaThread "pool-20-thread-1" [_thread_in_native, id=3431349, stack(0x00007f8ab9091000,0x00007f8ab90d2000)]
  0x00007f9038005fd0 JavaThread "I/O dispatcher 17" [_thread_in_native, id=3431350, stack(0x00007f8ab8038000,0x00007f8ab8079000)]
  0x00007f90380a0980 JavaThread "I/O dispatcher 18" [_thread_in_native, id=3431351, stack(0x00007f8abaee0000,0x00007f8abaf21000)]
  0x00007f90380acfd0 JavaThread "I/O dispatcher 19" [_thread_in_native, id=3431352, stack(0x00007f908270b000,0x00007f908274c000)]
  0x00007f90380ae1f0 JavaThread "I/O dispatcher 20" [_thread_in_native, id=3431353, stack(0x00007f8ab8944000,0x00007f8ab8985000)]
  0x00007f906c610f80 JavaThread "pool-21-thread-1" [_thread_in_native, id=3431354, stack(0x00007f8ab80fb000,0x00007f8ab813c000)]
  0x00007f904411bb50 JavaThread "I/O dispatcher 21" [_thread_in_native, id=3431355, stack(0x00007f8ab7aeb000,0x00007f8ab7b2c000)]
  0x00007f90442be7f0 JavaThread "I/O dispatcher 22" [_thread_in_native, id=3431356, stack(0x00007f9082689000,0x00007f90826ca000)]
  0x00007f904436e2c0 JavaThread "I/O dispatcher 23" [_thread_in_native, id=3431357, stack(0x00007f8ab8079000,0x00007f8ab80ba000)]
  0x00007f904436e7d0 JavaThread "I/O dispatcher 24" [_thread_in_native, id=3431358, stack(0x00007f90826ca000,0x00007f908270b000)]
  0x00007f90c89e58d0 JavaThread "pool-23-thread-1" [_thread_in_native, id=3431684, stack(0x00007f908274c000,0x00007f908278d000)]
  0x00007f905c02a780 JavaThread "I/O dispatcher 25" [_thread_in_native, id=3431685, stack(0x00007f8ab6da8000,0x00007f8ab6de9000)]
  0x00007f905c0061e0 JavaThread "I/O dispatcher 26" [_thread_in_native, id=3431686, stack(0x00007f90819db000,0x00007f9081a1c000)]
  0x00007f905c007430 JavaThread "I/O dispatcher 27" [_thread_in_native, id=3431687, stack(0x00007f8abac5e000,0x00007f8abac9f000)]
  0x00007f905c02f160 JavaThread "I/O dispatcher 28" [_thread_in_native, id=3431688, stack(0x00007f8ab6ce5000,0x00007f8ab6d26000)]
  0x00007f90c8dde6c0 JavaThread "pool-24-thread-1" [_thread_in_native, id=3431689, stack(0x00007f8ab6d26000,0x00007f8ab6d67000)]
  0x00007f90780bc710 JavaThread "I/O dispatcher 29" [_thread_in_native, id=3431690, stack(0x00007f8ab6d67000,0x00007f8ab6da8000)]
  0x00007f90780d5f40 JavaThread "I/O dispatcher 30" [_thread_in_native, id=3431691, stack(0x00007f8ab8fce000,0x00007f8ab900f000)]
  0x00007f90780ba110 JavaThread "I/O dispatcher 31" [_thread_in_native, id=3431692, stack(0x00007f8ab9050000,0x00007f8ab9091000)]
  0x00007f907808b470 JavaThread "I/O dispatcher 32" [_thread_in_native, id=3431693, stack(0x00007f8ab90d2000,0x00007f8ab9113000)]
  0x00007f90c8ddb150 JavaThread "pool-25-thread-1" [_thread_in_native, id=3431694, stack(0x00007f8ab8240000,0x00007f8ab8281000)]
  0x00007f909001c680 JavaThread "I/O dispatcher 33" [_thread_in_native, id=3431695, stack(0x00007f8ab69a2000,0x00007f8ab69e3000)]
  0x00007f9090024eb0 JavaThread "I/O dispatcher 34" [_thread_in_native, id=3431696, stack(0x00007f8ab6961000,0x00007f8ab69a2000)]
  0x00007f9090026470 JavaThread "I/O dispatcher 35" [_thread_in_native, id=3431697, stack(0x00007f8ab6920000,0x00007f8ab6961000)]
  0x00007f9090031a10 JavaThread "I/O dispatcher 36" [_thread_in_native, id=3431698, stack(0x00007f8ab68df000,0x00007f8ab6920000)]
  0x00007f902413f350 JavaThread "pool-27-thread-1" [_thread_in_native, id=3435463, stack(0x00007f8ab6100000,0x00007f8ab6141000)]
  0x00007f90316dc590 JavaThread "I/O dispatcher 37" [_thread_in_native, id=3435464, stack(0x00007f8ab60bf000,0x00007f8ab6100000)]
  0x00007f90317f9ca0 JavaThread "I/O dispatcher 38" [_thread_in_native, id=3435465, stack(0x00007f8ab607e000,0x00007f8ab60bf000)]
  0x00007f90317f9770 JavaThread "I/O dispatcher 39" [_thread_in_native, id=3435466, stack(0x00007f8ab603d000,0x00007f8ab607e000)]
  0x00007f90316d2790 JavaThread "I/O dispatcher 40" [_thread_in_native, id=3435467, stack(0x00007f8ab5ffc000,0x00007f8ab603d000)]
  0x00007f902427a650 JavaThread "pool-28-thread-1" [_thread_in_native, id=3435469, stack(0x00007f8ab5fbb000,0x00007f8ab5ffc000)]
  0x00007f903c00dac0 JavaThread "I/O dispatcher 41" [_thread_in_native, id=3435470, stack(0x00007f8ab5f7a000,0x00007f8ab5fbb000)]
  0x00007f903c2f0230 JavaThread "I/O dispatcher 42" [_thread_in_native, id=3435471, stack(0x00007f8ab5f39000,0x00007f8ab5f7a000)]
  0x00007f903c2efce0 JavaThread "I/O dispatcher 43" [_thread_in_native, id=3435472, stack(0x00007f8ab5ef8000,0x00007f8ab5f39000)]
  0x00007f903ca0dca0 JavaThread "I/O dispatcher 44" [_thread_in_native, id=3435473, stack(0x00007f8ab5eb7000,0x00007f8ab5ef8000)]
  0x00007f9024272f30 JavaThread "pool-29-thread-1" [_thread_in_native, id=3435474, stack(0x00007f8ab5e76000,0x00007f8ab5eb7000)]
  0x00007f90580166b0 JavaThread "I/O dispatcher 45" [_thread_in_native, id=3435475, stack(0x00007f8ab5e35000,0x00007f8ab5e76000)]
  0x00007f9058015a30 JavaThread "I/O dispatcher 46" [_thread_in_native, id=3435476, stack(0x00007f8ab5df4000,0x00007f8ab5e35000)]
  0x00007f90580154b0 JavaThread "I/O dispatcher 47" [_thread_in_native, id=3435477, stack(0x00007f8ab5db3000,0x00007f8ab5df4000)]
  0x00007f9058017430 JavaThread "I/O dispatcher 48" [_thread_in_native, id=3435478, stack(0x00007f8ab5d72000,0x00007f8ab5db3000)]
  0x00007f9040008250 JavaThread "pool-31-thread-1" [_thread_in_native, id=3435507, stack(0x00007f8ab5d31000,0x00007f8ab5d72000)]
  0x00007f90701386a0 JavaThread "I/O dispatcher 49" [_thread_in_native, id=3435508, stack(0x00007f8ab5cf0000,0x00007f8ab5d31000)]
  0x00007f9070125a20 JavaThread "I/O dispatcher 50" [_thread_in_native, id=3435509, stack(0x00007f8ab5caf000,0x00007f8ab5cf0000)]
  0x00007f9070125470 JavaThread "I/O dispatcher 51" [_thread_in_native, id=3435510, stack(0x00007f8ab5c6e000,0x00007f8ab5caf000)]
  0x00007f9070132d60 JavaThread "I/O dispatcher 52" [_thread_in_native, id=3435511, stack(0x00007f8ab5c2d000,0x00007f8ab5c6e000)]
  0x00007f904000c0d0 JavaThread "pool-32-thread-1" [_thread_in_native, id=3435512, stack(0x00007f8ab5bec000,0x00007f8ab5c2d000)]
  0x00007f9088231dd0 JavaThread "I/O dispatcher 53" [_thread_in_native, id=3435513, stack(0x00007f8ab5bab000,0x00007f8ab5bec000)]
  0x00007f90882e9d50 JavaThread "I/O dispatcher 54" [_thread_in_native, id=3435514, stack(0x00007f8ab5b6a000,0x00007f8ab5bab000)]
  0x00007f908824a400 JavaThread "I/O dispatcher 55" [_thread_in_native, id=3435515, stack(0x00007f8ab5b29000,0x00007f8ab5b6a000)]
Card table byte_map: [0x00007f90ab000000,0x00007f90ac000000] _byte_map_base: 0x00007f90a8000000

Marking Bits (Prev, Next): (CMBitMap*) 0x00007f90c806c658, (CMBitMap*) 0x00007f90c806c690
 Prev Bits: [0x00007f90a2000000, 0x00007f90aa000000)
 Next Bits: [0x00007f909a000000, 0x00007f90a2000000)

Polling page: 0x00007f90d195b000

Metaspace:

Usage:
  Non-class:    107.45 MB capacity,   101.39 MB ( 94%) used,     5.83 MB (  5%) free+waste,   236.38 KB ( <1%) overhead. 
      Class:     13.56 MB capacity,    10.71 MB ( 79%) used,     2.74 MB ( 20%) free+waste,   113.56 KB ( <1%) overhead. 
       Both:    121.01 MB capacity,   112.10 MB ( 93%) used,     8.57 MB (  7%) free+waste,   349.94 KB ( <1%) overhead. 

Virtual space:
  Non-class space:      110.00 MB reserved,     109.54 MB (>99%) committed 
      Class space:        1.00 GB reserved,      14.75 MB (  1%) committed 
             Both:        1.11 GB reserved,     124.28 MB ( 11%) committed 

Chunk freelists:
   Non-Class:  1.92 MB
       Class:  1.12 MB
        Both:  3.04 MB

MaxMetaspaceSize: unlimited
CompressedClassSpaceSize: 1.00 GB
Initial GC threshold: 20.80 MB
Current GC threshold: 207.14 MB
CDS: off

CodeHeap 'non-profiled nmethods': size=120032Kb used=60338Kb max_used=60338Kb free=59693Kb
 bounds [0x00007f90b82c0000, 0x00007f90bbdb0000, 0x00007f90bf7f8000]
CodeHeap 'profiled nmethods': size=120032Kb used=57213Kb max_used=57232Kb free=62819Kb
 bounds [0x00007f90b0d88000, 0x00007f90b46d8000, 0x00007f90b82c0000]
CodeHeap 'non-nmethods': size=5696Kb used=1500Kb max_used=1617Kb free=4195Kb
 bounds [0x00007f90b07f8000, 0x00007f90b0a68000, 0x00007f90b0d88000]
 total_blobs=30368 nmethods=29601 adapters=687
 compilation: enabled
              stopped_count=0, restarted_count=0
 full_count=0

Compilation events (20 events):
Event: 1015131.242 Thread 0x00007f90c8728ca0 45114       3       org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate$AsyncUpdateCallback::isDirty (18 bytes)
Event: 1015131.248 Thread 0x00007f90c8728ca0 nmethod 45114 0x00007f90b43fcb90 code [0x00007f90b43fcd40, 0x00007f90b43fced0]
Event: 1015131.248 Thread 0x00007f90c8728ca0 45115       3       org.apache.jackrabbit.oak.plugins.index.IndexUpdate::getUpdatedIndexPaths (11 bytes)
Event: 1015131.251 Thread 0x00007f90c8728ca0 nmethod 45115 0x00007f90b12df590 code [0x00007f90b12df760, 0x00007f90b12df9a0]
Event: 1015131.251 Thread 0x00007f90c8728ca0 45116       3       org.apache.jackrabbit.oak.plugins.index.progress.IndexingProgressReporter::getUpdatedIndexPaths (10 bytes)
Event: 1015131.251 Thread 0x00007f90c8728ca0 nmethod 45116 0x00007f90b389c610 code [0x00007f90b389c7c0, 0x00007f90b389c980]
Event: 1015131.251 Thread 0x00007f90c8728ca0 45117       3       org.apache.jackrabbit.oak.plugins.index.IndexUpdate::commitProgress (9 bytes)
Event: 1015131.251 Thread 0x00007f90c8728ca0 nmethod 45117 0x00007f90b45b7190 code [0x00007f90b45b7340, 0x00007f90b45b7480]
Event: 1015131.251 Thread 0x00007f90c8728ca0 45118       3       org.apache.jackrabbit.oak.plugins.index.IndexUpdate::getIndexingStats (8 bytes)
Event: 1015131.252 Thread 0x00007f90c8728ca0 nmethod 45118 0x00007f90b3a53610 code [0x00007f90b3a538c0, 0x00007f90b3a545d0]
Event: 1015131.252 Thread 0x00007f90c8728ca0 45119       3       org.apache.jackrabbit.oak.plugins.index.IndexUpdate$IndexUpdateRootState::getIndexingStats (30 bytes)
Event: 1015131.253 Thread 0x00007f90c8728ca0 nmethod 45119 0x00007f90b3307c10 code [0x00007f90b3307f00, 0x00007f90b3309050]
Event: 1015811.562 Thread 0x00007f90c8727360 45120       4       sun.security.ec.ECDHKeyAgreement::engineInit (34 bytes)
Event: 1015811.565 Thread 0x00007f90c8727360 nmethod 45120 0x00007f90bbda9f10 code [0x00007f90bbdaa0c0, 0x00007f90bbdaa368]
Event: 1016666.073 Thread 0x00007f90c8727360 45121       4       org.apache.jackrabbit.oak.plugins.index.lucene.directory.BufferedOakDirectory::fileLength (91 bytes)
Event: 1016666.076 Thread 0x00007f90c8727360 nmethod 45121 0x00007f90bbdaa510 code [0x00007f90bbdaa6e0, 0x00007f90bbdaa958]
Event: 1016854.848 Thread 0x00007f90c8727360 45122   !   4       javax.jcr.SimpleCredentials::getAttribute (23 bytes)
Event: 1016854.863 Thread 0x00007f90c8727360 nmethod 45122 0x00007f90bbdaac90 code [0x00007f90bbdaae60, 0x00007f90bbdab9a8]
Event: 1018437.955 Thread 0x00007f90c8727360 45123       4       org.apache.sling.resourceresolver.impl.mapping.MapEntries::doAddVanity (94 bytes)
Event: 1018437.970 Thread 0x00007f90c8727360 nmethod 45123 0x00007f90bbdabf90 code [0x00007f90bbdac180, 0x00007f90bbdac5c8]

GC Heap History (20 events):
Event: 1016471.396 GC heap before
{Heap before GC invocations=8819 (full 1):
 garbage-first heap   total 8388608K, used 6823232K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 1 survivors (2048K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1016471.638 GC heap after
{Heap after GC invocations=8820 (full 1):
 garbage-first heap   total 8388608K, used 5792971K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 8 young (16384K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1016628.511 GC heap before
{Heap before GC invocations=8821 (full 1):
 garbage-first heap   total 8388608K, used 6778059K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1016628.593 GC heap after
{Heap after GC invocations=8822 (full 1):
 garbage-first heap   total 8388608K, used 5747446K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 8 young (16384K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1016811.609 GC heap before
{Heap before GC invocations=8822 (full 1):
 garbage-first heap   total 8388608K, used 6777590K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1016811.836 GC heap after
{Heap after GC invocations=8823 (full 1):
 garbage-first heap   total 8388608K, used 5563981K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 7 young (14336K), 7 survivors (14336K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017046.123 GC heap before
{Heap before GC invocations=8823 (full 1):
 garbage-first heap   total 8388608K, used 6596173K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 7 survivors (14336K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017046.210 GC heap after
{Heap after GC invocations=8824 (full 1):
 garbage-first heap   total 8388608K, used 5563525K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 7 young (14336K), 7 survivors (14336K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017254.892 GC heap before
{Heap before GC invocations=8824 (full 1):
 garbage-first heap   total 8388608K, used 6595717K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 7 survivors (14336K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017254.968 GC heap after
{Heap after GC invocations=8825 (full 1):
 garbage-first heap   total 8388608K, used 5564210K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 8 young (16384K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017431.240 GC heap before
{Heap before GC invocations=8825 (full 1):
 garbage-first heap   total 8388608K, used 6594354K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017431.306 GC heap after
{Heap after GC invocations=8826 (full 1):
 garbage-first heap   total 8388608K, used 5565857K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 8 young (16384K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017666.370 GC heap before
{Heap before GC invocations=8826 (full 1):
 garbage-first heap   total 8388608K, used 6596001K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017666.438 GC heap after
{Heap after GC invocations=8827 (full 1):
 garbage-first heap   total 8388608K, used 5565882K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 8 young (16384K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017902.221 GC heap before
{Heap before GC invocations=8827 (full 1):
 garbage-first heap   total 8388608K, used 6596026K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1017902.283 GC heap after
{Heap after GC invocations=8828 (full 1):
 garbage-first heap   total 8388608K, used 5564846K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 8 young (16384K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1018111.303 GC heap before
{Heap before GC invocations=8828 (full 1):
 garbage-first heap   total 8388608K, used 6594990K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1018111.373 GC heap after
{Heap after GC invocations=8829 (full 1):
 garbage-first heap   total 8388608K, used 5566029K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 8 young (16384K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1018336.162 GC heap before
{Heap before GC invocations=8829 (full 1):
 garbage-first heap   total 8388608K, used 6596173K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 512 young (1048576K), 8 survivors (16384K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}
Event: 1018336.231 GC heap after
{Heap after GC invocations=8830 (full 1):
 garbage-first heap   total 8388608K, used 5569764K [0x0000000600000000, 0x0000000800000000)
  region size 2048K, 10 young (20480K), 10 survivors (20480K)
 Metaspace       used 114781K, capacity 123899K, committed 127264K, reserved 1161216K
  class space    used 10967K, capacity 13887K, committed 15100K, reserved 1048576K
}

Deoptimization events (20 events):
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT PACKING pc=0x00007f90b8448450 sp=0x00007f908009aa70
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f9080099fa8 mode 1
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT PACKING pc=0x00007f90b8688ee4 sp=0x00007f908009ab10
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f9080099fc8 mode 1
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT PACKING pc=0x00007f90b8eb0578 sp=0x00007f908009ad30
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f908009a098 mode 1
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT PACKING pc=0x00007f90b8eb0578 sp=0x00007f908009ad30
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f908009a098 mode 1
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT PACKING pc=0x00007f90b844840c sp=0x00007f908009af60
Event: 1018437.979 Thread 0x00007f9030747550 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f908009a318 mode 1
Event: 1018437.992 Thread 0x00007f90924f0f60 DEOPT PACKING pc=0x00007f90b844840c sp=0x00007f8843f4aff0
Event: 1018437.992 Thread 0x00007f90924f0f60 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f8843f4a3a8 mode 1
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT PACKING pc=0x00007f90babed824 sp=0x00007f8aa5f7b960
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f8aa5f7ada8 mode 1
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT PACKING pc=0x00007f90babed824 sp=0x00007f8aa5f7b770
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f8aa5f7abb8 mode 1
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT PACKING pc=0x00007f90bbda5f5c sp=0x00007f8aa5f7d0c0
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f8aa5f7c540 mode 1
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT PACKING pc=0x00007f90b43abb34 sp=0x00007f8aa5f7d140
Event: 1018437.994 Thread 0x00007f90240861d0 DEOPT UNPACKING pc=0x00007f90b084264a sp=0x00007f8aa5f7c690 mode 1

Classes redefined (0 events):
No events

Internal exceptions (20 events):
Event: 1015843.221 Thread 0x00007f9062250d70 Exception <a 'java/net/UnknownHostException'{0x00000007f27baf40}> (0x00000007f27baf40) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1015843.221 Thread 0x00007f9062250d70 Exception <a 'java/io/FileNotFoundException'{0x00000007f27bcda8}> (0x00000007f27bcda8) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1015888.686 Thread 0x00007f903617eba0 Exception <a 'java/io/FileNotFoundException'{0x00000007e3d9b090}> (0x00000007e3d9b090) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1015898.787 Thread 0x00007f903617eba0 Exception <a 'java/io/FileNotFoundException'{0x00000007e3dab2f0}> (0x00000007e3dab2f0) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1015961.673 Thread 0x00007f9062250d70 Exception <a 'java/io/FileNotFoundException'{0x00000007d3fb6f58}> (0x00000007d3fb6f58) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1015966.786 Thread 0x00007f9062250d70 Exception <a 'java/io/FileNotFoundException'{0x00000007d3fcb448}> (0x00000007d3fcb448) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016471.652 Thread 0x00007f90c870ad70 Exception <a 'java/net/SocketException'{0x00000007ffe871e0}> (0x00000007ffe871e0) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016471.652 Thread 0x00007f90c870ad70 Exception <a 'java/net/SocketException'{0x00000007ffe8a818}> (0x00000007ffe8a818) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016471.682 Thread 0x00007f90580c8790 Exception <a 'java/io/FileNotFoundException'{0x00000007fe87d718}> (0x00000007fe87d718) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016486.788 Thread 0x00007f90580c8790 Exception <a 'java/io/FileNotFoundException'{0x00000007fe88d118}> (0x00000007fe88d118) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016502.600 Thread 0x00007f90580c8790 Exception <a 'java/net/UnknownHostException'{0x00000007fe898310}> (0x00000007fe898310) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016654.612 Thread 0x00007f90900e25b0 Exception <a 'java/io/FileNotFoundException'{0x00000007f4f273e8}> (0x00000007f4f273e8) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016659.945 Thread 0x00007f90900e25b0 Exception <a 'java/net/UnknownHostException'{0x00000007f4f2c3b8}> (0x00000007f4f2c3b8) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016659.945 Thread 0x00007f90900e25b0 Exception <a 'java/io/FileNotFoundException'{0x00000007f4f2da28}> (0x00000007f4f2da28) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016811.849 Thread 0x00007f90c870ad70 Exception <a 'java/net/SocketException'{0x00000007ffedbd30}> (0x00000007ffedbd30) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1016811.849 Thread 0x00007f90c870ad70 Exception <a 'java/net/SocketException'{0x00000007ffef79e8}> (0x00000007ffef79e8) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1017396.904 Thread 0x00007f90900bb7e0 Exception <a 'java/io/FileNotFoundException'{0x00000007d5849bd8}> (0x00000007d5849bd8) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1017407.439 Thread 0x00007f90900bb7e0 Exception <a 'java/net/UnknownHostException'{0x00000007d58583d8}> (0x00000007d58583d8) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1017407.439 Thread 0x00007f90900bb7e0 Exception <a 'java/io/FileNotFoundException'{0x00000007d5859a48}> (0x00000007d5859a48) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]
Event: 1018437.994 Thread 0x00007f90240861d0 Exception <a 'java/io/FileNotFoundException'{0x00000007de6ad0c0}> (0x00000007de6ad0c0) thrown at [./src/hotspot/share/prims/jni.cpp, line 617]

Events (20 events):
Event: 1018437.945 Thread 0x00007f902a2efd90 Thread added: 0x00007f902a2efd90
Event: 1018437.945 Protecting memory [0x00007f8843fcf000,0x00007f8843fd3000] with protection modes 0
Event: 1018437.965 Thread 0x00007f9035fbdc50 Thread added: 0x00007f9035fbdc50
Event: 1018437.966 Thread 0x00007f90cb651000 Thread added: 0x00007f90cb651000
Event: 1018437.966 Protecting memory [0x00007f8843f8e000,0x00007f8843f92000] with protection modes 0
Event: 1018437.976 Protecting memory [0x00007f8843f4d000,0x00007f8843f51000] with protection modes 0
Event: 1018437.976 Thread 0x00007f90924f0f60 Thread added: 0x00007f90924f0f60
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck done
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck done
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck done
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck
Event: 1018437.979 loading class org/apache/sling/hc/api/HealthCheck done
Event: 1018437.986 Protecting memory [0x00007f8843f0c000,0x00007f8843f10000] with protection modes 0
Event: 1018437.997 Thread 0x00007f9086359050 Thread added: 0x00007f9086359050
Event: 1018437.998 Protecting memory [0x00007f8843ecb000,0x00007f8843ecf000] with protection modes 0
Event: 1018438.007 Thread 0x00007f901e30c1e0 Thread added: 0x00007f901e30c1e0
Event: 1018438.007 Protecting memory [0x00007f8843e8a000,0x00007f8843e8e000] with protection modes 0


Dynamic libraries:
600000000-800000000 rw-p 00000000 00:00 0 
800000000-800ebf000 rw-p 00000000 00:00 0 
800ebf000-840000000 ---p 00000000 00:00 0 
558541895000-558541896000 r-xp 00000000 fd:01 18571276                   /usr/lib/jvm/java-11-amazon-corretto/bin/java
558541a96000-558541a97000 r--p 00001000 fd:01 18571276                   /usr/lib/jvm/java-11-amazon-corretto/bin/java
558541a97000-558541a98000 rw-p 00002000 fd:01 18571276                   /usr/lib/jvm/java-11-amazon-corretto/bin/java
55854240b000-558545c7e000 rw-p 00000000 00:00 0                          [heap]
7f8843d04000-7f8843ecb000 rw-p 00000000 00:00 0 
7f8843ecb000-7f8843ecf000 ---p 00000000 00:00 0 
7f8843ecf000-7f8843f0c000 rw-p 00000000 00:00 0 
7f8843f0c000-7f8843f10000 ---p 00000000 00:00 0 
7f8843f10000-7f8843f4d000 rw-p 00000000 00:00 0 
7f8843f4d000-7f8843f51000 ---p 00000000 00:00 0 
7f8843f51000-7f8843f8e000 rw-p 00000000 00:00 0 
7f8843f8e000-7f8843f92000 ---p 00000000 00:00 0 
7f8843f92000-7f8843fcf000 rw-p 00000000 00:00 0 
7f8843fcf000-7f8843fd3000 ---p 00000000 00:00 0 
7f8843fd3000-7f8844010000 rw-p 00000000 00:00 0 
7f8844010000-7f8844014000 ---p 00000000 00:00 0 
7f8844014000-7f8844051000 rw-p 00000000 00:00 0 
7f8844051000-7f8844055000 ---p 00000000 00:00 0 
7f8844055000-7f8844092000 rw-p 00000000 00:00 0 
7f8844092000-7f8844096000 ---p 00000000 00:00 0 
7f8844096000-7f88440d3000 rw-p 00000000 00:00 0 
7f88440d3000-7f88440d7000 ---p 00000000 00:00 0 
7f88440d7000-7f8844114000 rw-p 00000000 00:00 0 
7f8844114000-7f8844118000 ---p 00000000 00:00 0 
7f8844118000-7f8844155000 rw-p 00000000 00:00 0 
7f8844155000-7f8844159000 ---p 00000000 00:00 0 
7f8844159000-7f8844196000 rw-p 00000000 00:00 0 
7f8844196000-7f884419a000 ---p 00000000 00:00 0 
7f884419a000-7f88441d7000 rw-p 00000000 00:00 0 
7f88441d7000-7f88441db000 ---p 00000000 00:00 0 
7f88441db000-7f8844218000 rw-p 00000000 00:00 0 
7f8844218000-7f884421c000 ---p 00000000 00:00 0 
7f884421c000-7f8844259000 rw-p 00000000 00:00 0 
7f8844259000-7f884425d000 ---p 00000000 00:00 0 
7f884425d000-7f884429a000 rw-p 00000000 00:00 0 
7f884429a000-7f884429e000 ---p 00000000 00:00 0 
7f884429e000-7f88442db000 rw-p 00000000 00:00 0 
7f88442db000-7f88442df000 ---p 00000000 00:00 0 
VM Arguments:
jvm_args: -XX:+HeapDumpOnOutOfMemoryError -XX:+UseGCOverheadLimit -XX:+OptimizeStringConcat -XX:+UseStringDeduplication -Xms2g -Xmn1g -Xss256K -XX:+UseAdaptiveSizePolicy -XX:-UseBiasedLocking -XX:+DisableExplicitGC -XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -XX:HeapDumpPath=shaft2/logs -Xlog:gc*,gc+ref=debug,gc+heap=debug,gc+age=trace:file=gc-%p-%t.log:tags,uptime,time,level:filecount=10,filesize=50m -Xmx1g -Djava.awt.headless=true -Xmx8g -Djava.awt.headless=true -Dsling.run.modes=oak_tar,shaft,prod -agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n 
java_command: shaft2.jar -p 6505 -c shaft2
java_class_path (initial): shaft2.jar
Launcher Type: SUN_STANDARD

[Global flags]
     intx CICompilerCount                          = 3                                         {product} {ergonomic}
     uint ConcGCThreads                            = 1                                         {product} {ergonomic}
     bool DisableExplicitGC                        = true                                      {product} {command line}
     uint G1ConcRefinementThreads                  = 4                                         {product} {ergonomic}
   size_t G1HeapRegionSize                         = 2097152                                   {product} {ergonomic}
    uintx GCDrainStackTargetSize                   = 64                                        {product} {ergonomic}
     bool HeapDumpOnOutOfMemoryError               = true                                   {manageable} {command line}
    ccstr HeapDumpPath                             = shaft2/logs                            {manageable} {command line}
   size_t InitialHeapSize                          = 2147483648                                {product} {command line}
   size_t MarkStackSize                            = 4194304                                   {product} {ergonomic}
    uintx MaxGCPauseMillis                         = 1000                                      {product} {command line}
   size_t MaxHeapSize                              = 8589934592                                {product} {command line}
   size_t MaxNewSize                               = 1073741824                                {product} {command line}
   size_t MinHeapDeltaBytes                        = 2097152                                   {product} {ergonomic}
   size_t NewSize                                  = 1073741824                                {product} {command line}
    uintx NonNMethodCodeHeapSize                   = 5830732                                {pd product} {ergonomic}
    uintx NonProfiledCodeHeapSize                  = 122913754                              {pd product} {ergonomic}
     bool OptimizeStringConcat                     = true                                   {C2 product} {command line}
    uintx ProfiledCodeHeapSize                     = 122913754                              {pd product} {ergonomic}
    uintx ReservedCodeCacheSize                    = 251658240                              {pd product} {ergonomic}
     bool SegmentedCodeCache                       = true                                      {product} {ergonomic}
     intx ThreadStackSize                          = 256                                    {pd product} {command line}
     bool UseAdaptiveSizePolicy                    = true                                      {product} {command line}
     bool UseBiasedLocking                         = false                                     {product} {command line}
     bool UseCompressedClassPointers               = true                                 {lp64_product} {ergonomic}
     bool UseCompressedOops                        = true                                 {lp64_product} {ergonomic}
     bool UseG1GC                                  = true                                      {product} {command line}
     bool UseGCOverheadLimit                       = true                                      {product} {command line}
     bool UseStringDeduplication                   = true                                      {product} {command line}

Logging:
Log output configuration:
 #0: stdout all=warning uptime,level,tags
 #1: stderr all=off uptime,level,tags
 #2: file=gc-%p-%t.log all=off,gc*=info,age*=trace,gc+ref=debug,gc+heap=debug time,uptime,level,tags filecount=10,filesize=51200K

Environment Variables:
PATH=/usr/share/Modules/bin:/sbin:/bin:/usr/sbin:/usr/bin
SHELL=/bin/bash
LANG=en_US.UTF-8
TERM=xterm

Signal Handlers:
SIGSEGV: [libjvm.so+0xf4d020], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0xf4d020], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0xf4d020], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0xc6ca00], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0xc6ca00], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0xf4d020], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR2: [libjvm.so+0xc6c8a0], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: [libjvm.so+0xc6ca50], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGINT: SIG_IGN, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGTERM: [libjvm.so+0xc6ca50], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGQUIT: [libjvm.so+0xc6ca50], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO


---------------  S Y S T E M  ---------------

OS:Red Hat Enterprise Linux release 8.7 (Ootpa)
uname:Linux 4.18.0-425.10.1.el8_7.x86_64 #1 SMP Wed Dec 14 16:00:01 EST 2022 x86_64
OS uptime: 19 days 19:50 hours
libc:glibc 2.28 NPTL 2.28 
rlimit (soft/hard): STACK 8192k/infinity , CORE 0k/infinity , NPROC 111547/111547 , NOFILE 262144/262144 , AS infinity/infinity , CPU infinity/infinity , DATA infinity/infinity , FSIZE infinity/infinity , MEMLOCK 64k/64k
load average:1.49 1.93 3.04

/proc/meminfo:
MemTotal:       28593480 kB
MemFree:          364336 kB
MemAvailable:    9921124 kB
Buffers:           14644 kB
Cached:          9863084 kB
SwapCached:       302524 kB
Active:         12260132 kB
Inactive:       11933616 kB
Active(anon):   11392064 kB
Inactive(anon):  3297828 kB
Active(file):     868068 kB
Inactive(file):  8635788 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       4194300 kB
SwapFree:        1100188 kB
Dirty:              8740 kB
Writeback:             0 kB
AnonPages:      14002996 kB
Mapped:           399544 kB
Shmem:            373848 kB
KReclaimable:     493788 kB
Slab:            1324116 kB
SReclaimable:     493788 kB
SUnreclaim:       830328 kB
KernelStack:      537968 kB
PageTables:       121264 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    18491040 kB
Committed_AS:   28869276 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
Percpu:             4128 kB
HardwareCorrupted:     0 kB
AnonHugePages:  11534336 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:     2592704 kB
DirectMap2M:    26767360 kB
DirectMap1G:     2097152 kB

/sys/kernel/mm/transparent_hugepage/enabled:
[always] madvise never
/sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter):
always defer defer+madvise [madvise] never

Process Memory:
Virtual Size: 45371148K (peak: 45402236K)
Resident Set Size: 9771140K (peak: 10159116K) (anon: 9708424K, file: 62716K, shmem: 0K)
Swapped out: 1250176K
C-Heap outstanding allocations: 1345807K, retained: 473452K (may have wrapped)
glibc malloc tunables: (default)

/proc/sys/kernel/threads-max (system-wide limit on the number of threads):
223094
/proc/sys/vm/max_map_count (maximum number of memory map areas a process may have):
65530
/proc/sys/kernel/pid_max (system-wide limit on number of process identifiers):
4194304

container (cgroup) information:
container_type: cgroupv1
cpu_cpuset_cpus: 0-3
cpu_memory_nodes: 0
active_processor_count: 4
cpu_quota: no quota
cpu_period: 100000
cpu_shares: no shares
memory_limit_in_bytes: unlimited
memory_and_swap_limit_in_bytes: unlimited
memory_soft_limit_in_bytes: unlimited
memory_usage_in_bytes: 14127796 k
memory_max_usage_in_bytes: 17845184 k
maximum number of tasks: unlimited
current number of tasks: 32573

Hyper-V virtualization detected
Steal ticks since vm start: 0
Steal ticks percentage since vm start:  0.000

CPU:total 4 (initial active 4) (4 cores per cpu, 1 threads per core) family 6 model 85 stepping 4 microcode 0xffffffff, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, tsc, bmi1, bmi2, adx, fma
CPU Model and flags from /proc/cpuinfo:
model name	: Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap clflushopt avx512cd avx512bw avx512vl xsaveopt xsavec xsaves md_clear
Online cpus:
0-3
Offline cpus:

BIOS frequency limitation:
<Not Available>
Frequency switch latency (ns):
<Not Available>
Available cpu frequencies:
<Not Available>
Current governor:
<Not Available>
Core performance/turbo boost:
<Not Available>

Memory: 4k page, physical 28593480k(364336k free), swap 4194300k(1100188k free)

vm_info: OpenJDK 64-Bit Server VM (11.0.18+10-LTS) for linux-amd64 JRE (11.0.18+10-LTS), built on Jan 10 2023 20:20:12 by "jenkins" with gcc 7.3.1 20180303 (Red Hat 7.3.1-5)

END.
@Bhumika-Khunt Bhumika-Khunt added the bug Something isn't working label Feb 22, 2023
@eastig
Copy link
Member

eastig commented Mar 9, 2023

Hi @Bhumika-Khunt,
The JVM crashed because it failed to setup a stack for a Java thread. JVM reported out of memory.

Your process might have reached /proc/sys/vm/max_map_count which 64k. You can check the process memory with pmap.
Could you try to increase /proc/sys/vm/max_map_count to 128k or 256k?

@eastig
Copy link
Member

eastig commented Mar 9, 2023

I also see you might have a lot of Java threads: 32558

_java_thread_list=0x00007f90768e66b0, length=32558, elements={

If it's true, this is too many for your host. It has only 4 cores.

@simonis
Copy link
Contributor

simonis commented Mar 14, 2023

I agree with @eastig in that you probably ran out of memory mappings. Every Java thread will consume two memory mappings (for the stack itself and for its guard pages). You can see this in the memory mappings of your error file (which I suppose is truncated):

...
7f8844096000-7f88440d3000 rw-p 00000000 00:00 0 
7f88440d3000-7f88440d7000 ---p 00000000 00:00 0 
...

0x7f88440d3000 - 0x7f8844096000 = 249856
0x7f88440d7000 - 0x7f88440d3000 =  16384
----------------------------------------
                                  266240 = 260k

This corresponds to your 265k stack size (i.e. -Xss256K)

32558 * 2 gives 65116 mappings just for the threads and is already quite close to your 64k mappings limit.

@Bhumika-Khunt
Copy link
Author

Hi @eastig and @simonis,
Thank you for your valuable inputs.
Basis on the your inputs, I will increase the /proc/sys/vm/max_map_count to 128k or 256k and keep it on observation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@simonis @eastig @Bhumika-Khunt