Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MPSolver: Java memory leakage problem #4292

Open
longhuihy opened this issue Jun 29, 2024 · 23 comments
Open

MPSolver: Java memory leakage problem #4292

longhuihy opened this issue Jun 29, 2024 · 23 comments
Assignees
@lperron
Labels
Lang: Java Java wrapper issue Solver: Linear Solver Related to all Linear Solver (GLOP, BOP, CBC etc...)

Comments

@longhuihy
Copy link

longhuihy commented Jun 29, 2024 •
edited
Loading

What version of OR-Tools and what language are you using?
Version: 9.10.4067

Language: C++/Java/Python/C#
Java

Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP, Gurobi)
SCIP
MPSolver solver = new MPSolver("SCIP", MPSolver.OptimizationProblemType.SCIP_MIXED_INTEGER_PROGRAMMING);

What operating system (Linux, Windows, ...) and version?
NAME="Oracle Linux Server"
VERSION="8.5"

What did you do?
The model I am solving is not very large in scale, but I will serially call to solve dozens to hundreds of models of similar scale, resulting in an increase in memory. I will investigate if it is not the JVM's heap memory and out of heap memory because I have printed out the heap memory and out of heap memory, which is likely caused by or-tools memory leakage

What did you expect to see

What did you see instead?

Make sure you include information that can help us debug (full error message, model Proto).

Anything else we should know about your project / environment
@longhuihy
Copy link
Author

After performing full GC, both heap and non heap memory decreased to a very low level, but container monitoring showed that there was still a large amount of memory that was not reclaimed, ultimately leading to exceeding the container's limits of memory

@lperron
Copy link
Collaborator

lperron commented Jun 29, 2024

can you try with ModelBuilder ?

@longhuihy
Copy link
Author

2024-06-30_074921 This is the memory change curve, during which a container automatic restart occurred

@lperron
Copy link
Collaborator

lperron commented Jul 1, 2024

If the memory returns to low level, then most likely there are no memory leak.
Just the GC buckling under pressure.

There are some parameters to alleviate this. Or maybe just for a garbage collect after each solve.

@longhuihy
Copy link
Author

image

@longhuihy
Copy link
Author

If the memory returns to low level, then most likely there are no memory leak. Just the GC buckling under pressure.

There are some parameters to alleviate this. Or maybe just for a garbage collect after each solve.

Thanks for your reply. After performing full GC, both heap and non heap memory decreased to a very low level，But the total memory usage of pods is still very high.

Previously, I only called MPSolver. delete() after the solution was completed, but the memory could not be released. I tried calling MPVariable. delete() and MPConstraint. delete() one by one, but encountered an error during execution.

@lperron
Copy link
Collaborator

lperron commented Jul 1, 2024

this has happened before, and the solution was to tweak the GC parameters on the jvm.
It is still possible there is a leak. We do now own the SCIP code. But currently, there is no indication this is the case.

@lperron
Copy link
Collaborator

lperron commented Jul 1, 2024 via email •
edited
Loading

@longhuihy
Copy link
Author

variables and constraints are owned by the solver. you cannot delete them yourself.
…
--Laurent

I tried this after reading what was written in this article
https://groups.google.com/g/or-tools-discuss/c/KtrRGXkc8Bo

@lperron
Copy link
Collaborator

lperron commented Jul 2, 2024 via email

@longhuihy
Copy link
Author

Sorry, I am not familiar with how to use Model Builder to model mixed integer programming problems, and I have not found any relevant examples online. Could you please recommend a document or article? Thank you.

@lperron
Copy link
Collaborator

lperron commented Jul 3, 2024 via email

@longhuihy
Copy link
Author

Thank you. I'll give it a try

@longhuihy
Copy link
Author

I tried using ModelBuilder's modeling method, but there are still memory leakage issues. The example I am referring to is the ortools/linear_solver/samples/SimpleMipProgramMb. I used the command jcmdGC. run in the container to reclaim memory. The heap memory and out of heap memory have both been reduced to very low levels, but the container memory usage is still high.

@longhuihy
Copy link
Author

[Heap Memory Usage: init = 4294967296(4194304K) used = 66685936(65122K) committed = 4294967296(4194304K) max = 12884901888(12582912K)] ## ''
[Non-Heap Memory Usage: init = 7667712(7488K) used = 149457328(145954K) committed = 152567808(148992K) max = -1(-1K)] ## ''

@lperron
Copy link
Collaborator

lperron commented Jul 4, 2024 via email

@lperron
Copy link
Collaborator

lperron commented Jul 4, 2024 via email

@longhuihy
Copy link
Author

The maximum memory limit of the container is set to 18G, and the maximum heap memory size is set to 10G. Although heap memory can be reclaimed, as the number of calls to the solver increases, the total memory continues to increase, ultimately exceeding the container's memory limit and causing the container to restart.This is also the reason why I suspect a memory leak has occurred.

@lperron
Copy link
Collaborator

lperron commented Jul 4, 2024 via email

@GloryJie
Copy link

GloryJie commented Sep 5, 2024

@longhuihy

The maximum memory limit of the container is set to 18G, and the maximum heap memory size is set to 10G. Although heap memory can be reclaimed, as the number of calls to the solver increases, the total memory continues to increase, ultimately exceeding the container's memory limit and causing the container to restart.This is also the reason why I suspect a memory leak has occurred.

Has the problem been solved？how？

@longhuihy
Copy link
Author

@GloryJie
I haven't completely solved it yet. I am currently releasing memory through scheduled restarts

@lperron
Copy link
Collaborator

lperron commented Sep 18, 2024 via email

@Mizux Mizux added Lang: Java Java wrapper issue Solver: Linear Solver Related to all Linear Solver (GLOP, BOP, CBC etc...) labels Oct 22, 2024
@Mizux Mizux changed the title memory leakage problem MPSolver: Java memory leakage problem Oct 22, 2024
@phiedw
Copy link

phiedw commented Oct 28, 2024 •
edited
Loading

@longhuihy
Are you easily able to try using jemalloc? While investigating a possible memory leak on our app (may be the same issue you were facing), we tried using it and it considerably reduced the memory usage. (In fact, simply setting MALLOC_MMAP_THRESHOLD_=65536 with the the default allocator seems to do the trick)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lperron lperron

Labels
Lang: Java Java wrapper issue Solver: Linear Solver Related to all Linear Solver (GLOP, BOP, CBC etc...)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Mizux @lperron @longhuihy @phiedw @GloryJie