JVM level memory tuning parameters (InitialRAMPercentage MaxRAMPercentage and MinRAMPercentage) are not supported for APIM 4.1.0 in Kubernetes environments.

[Menuka-Senevirathne]

Description

Hi All

The goal is to dynamically change the memory allocation for APIM in the Kubernetes environment. But the pod is failing with an OOMKIlled error in the startup once the percentages are configured instead of Xmx and Xms.

Steps to Reproduce

Followed the steps here and deployed single APIM 4.1.0 instance in kubernetes

helm install --dependency-update apimtest /home/azureuser/new4.1.0/kubernetes-apim/simple/am-single --namespace apim
Initially, we kept the default Xmx and Xms (-Xms1024m -Xmx1024m) values and monitored the pods behavior for a while.

In the pod the resource allocation was as below.

    requests:
      memory: "3Gi"
      cpu: "2000m"
    limits:
      memory: "3Gi"
      cpu: "3000m"

The pods were running smoothly without an issue. Please note that there was no load on the APIM servers.
Then we edited the deployment and removed the default Xmx and Xms values.

k edit deploy wso2am-single-node-am-deployment -n apim
And it was replaced with percentage values as below.

spec:
  containers:
  - env:
    - name: NODE_IP
      valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: status.podIP
    - name: JVM_MEM_OPTS
      value: -XX:InitialRAMPercentage=75.0 -XX:MaxRAMPercentage=75.0 -XX:MinRAMPercentage=75.0

helm upgrade --dependency-update apimtest /home/azureuser/new4.1.0/kubernetes-apim/simple/am-single --namespace apim --create-namespace

Afterwards, we could see the pod keep crashing with OOM error. And we can see the percentage values are applied properly when the pod is described.

Environment:
NODE_IP: (v1:status.podIP)
JVM_MEM_OPTS: -XX:InitialRAMPercentage=75.0 -XX:MaxRAMPercentage=75.0 -XX:MinRAMPercentage=75.0

As you can see 75% is higher than the original Xmx and Xms values it had, since we didn't change anything with requests, the behavior is the same when percentages are dropped to 50%.

We tried by applying the parameters as -XX:+UnlockExperimentalVMOptions -XX:+UseContainerSupport -XX:MaxRAMFraction=1 -XX:InitialRAMPercentage=75.0 -XX:MaxRAMPercentage=75.0 -XX:MinRAMPercentage=75.0. But still the pod fails to start with an OOMKilled issue. Sharing the output after describing the pod.

Containers:

  wso2am:
    Container ID:   docker://b4517c857765f128c3d5e4bce1342bcdbc7f9dadfe54f7bc57a8a9ea825e0259
    Image:          docker.wso2.com/wso2am:4.1.0.0
    Image ID:       docker-pullable://docker.wso2.com/wso2am@sha256:d656003b53f38ed8693bc4a7f2b17999d55f9e2e1d4a0f81d5a8750a24ccd4bb
    Ports:          8280/TCP, 8243/TCP, 9763/TCP, 9443/TCP, 9711/TCP, 9611/TCP, 5672/TCP
    Host Ports:     0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP, 0/TCP
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    0
      Started:      Tue, 29 Oct 2024 14:39:19 +0000
      Finished:     Tue, 29 Oct 2024 14:41:05 +0000
    Ready:          False
    Restart Count:  1
    Limits:
      cpu:     3
      memory:  3Gi
    Requests:
      cpu:      2
      memory:   3Gi
    Liveness:   http-get http://:9763/services/Version delay=180s timeout=1s period=10s #success=1 #failure=3
    Readiness:  http-get http://:9763/services/Version delay=180s timeout=1s period=10s #success=1 #failure=3
    Environment:
      NODE_IP:        (v1:status.podIP)
      JVM_MEM_OPTS:  -XX:+UnlockExperimentalVMOptions -XX:+UseContainerSupport -XX:MaxRAMFraction=1 -XX:InitialRAMPercentage=75.0 -XX:MaxRAMPercentage=75.0 -XX:MinRAMPercentage=75.0

Affected Component

APIM

Version

4.1.0

Environment Details (with versions)

Kubernetes envrionment

JDK version is as below
openjdk version "11.0.14" 2022-01-18
OpenJDK Runtime Environment Temurin-11.0.14+9 (build 11.0.14+9)
OpenJDK 64-Bit Server VM Temurin-11.0.14+9 (build 11.0.14+9, mixed mode)

Relevant Log Output

No response

Related Issues

No response

Suggested Labels

No response