Originally posted by: SergeyPolyakovskiy

Dear All,

I have implemented my custom constraint as an extension of IloCustomConstraint in Java for CPoptimizer 12.6 or 12.6.1 which I run on Windows and Linux machine. The constraint is propagated many times in the searching process but the problem I solve is not really large - just 250 binary variables.

So, many times leafs are reached in branching while the searching log reports me that "Memory usage = 8.6 MB". However, even the computational cluster with allocated 30Gb RAM I use is out of memory (or the garbage collector reports an exception). The interesting fact is that memory leakage exists even when the "execute" procedure has an empty body, thus no actions are executed really. I get rid of leakage when the custom constraint is not used.

Is there any explanations or suggestions on how can I proceed?

All the best,

Sergey

 

public class PrecedenceConstraint_EXT extends IloCustomConstraint

{

                private Item itemJ;

                private IloIntVar weight;

                private int index;

 

                public PrecedenceConstraint_EXT(IloCP cp, IloIntVar weight, Item itemJ, int index) throws IloException

                {

                                super(cp);

                                this.weight = weight;

                                this.itemJ = itemJ;

                                this.index = index;

                                addVar(weight);

                                addVar(itemJ.chosen);

                }

 

                public void execute()

                {

                 }

}

 

 

#CPOptimizer
#DecisionOptimization

Originally posted by: GGR

Hi Sergei

 

Have you try to trace the call to the constructor of the constraint (or the one of itemJ?)

 

Hope that helps

 

 

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hello Rogerie,

 

I am curious about your guess. What do you mean exactly? I have traced all my program code. I do not see any cause that may generate leakage so far. If I comment both addvar(…) entries, the problem goes away. So looks like the leakage arises when the constraint is propagated as an action to the change of the variables' states. Do you have any ideas?

 

Sergey

#CPOptimizer
#DecisionOptimization

Originally posted by: GGR

Hi Sergey

Sorry I was not clear.

My point was about a malfunctioning of the engine. For example, it repeatedly constructs the constraint ans so allocates huge amount of memory.

 

Cheers

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hi Rogerie,

I add all the constraints with cp.add(new PrecedenceConstraint_EXT(…)) function prior to start the search. So, I do not have any multiple creations of similar objects or something that may generate leakage. Note that the leakage starts once I start the search, not earlier that time. The execute() procedure is empty and the leakage takes place only when I add variables in the creator to point when to propagate the constraint. So it looks like the leakage is caused by some internal causes during the search. I do not know how to catch the things that happen internally in the searching process. So far, it looks like some internal objects are created during the search and kept. The garbage collector does not find anything to free as a result.

Thank you,

 

Sergey

#CPOptimizer
#DecisionOptimization

Originally posted by: GGR

Hi Sergey

I do agree with you the leakage should be in the internal code.

Can you try your code with the CP Optimizer parameter workers set to 1?

Cheers

 

 

 

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hi Rogerie,

This is that I am exactly doing. I attach the log.

! ----------------------------------------------------------------------------

 ! Maximization problem - 165 variables, 1782 constraints, 114 phases

 ! Presolve      : 1625 extractables eliminated

 ! LogPeriod            = 1000000

 ! SearchType           = DepthFirst

 ! Workers              = 1

 ! OptimalityTolerance  = 0

 ! RelativeOptimalityTolerance = 0

 ! Initial process time : 0.00s (0.00s extraction + 0.00s propagation)

 !  . Log search space  : 854.0 (before), 854.0 (after)

 !  . Memory usage      : 2.2 MB (before), 2.2 MB (after)

 ! Using sequential search.

 ! ----------------------------------------------------------------------------

 !          Best Branches  Non-fixed            Branch decision

 * 5595.87045057       33 0.01s                   0  = in city48

 * 5715.62021263       34 0.01s                   1  = in city48

 * 6122.53885894       36 0.01s                   0  = in city48

 *  6242.0497242       37 0.01s                   1  = in city48

 * 6342.51946918       38 0.01s                   1  = in city38

 * 6390.78479514       41 0.01s                   0  = in city43

 * 6796.82082065       43 0.01s                   0  = in city48

 * 6916.09271092       44 0.01s                   1  = in city48

 * 7016.15426479       45 0.01s                   1  = in city38

 * 7067.82399632       48 0.01s                   0  = in city38

 * 7167.81702792       49 0.01s                   1  = in city38

 * 7215.66629595       52 0.01s                   0  = in city43

 * 7620.64739454       54 0.01s                   0  = in city48

 * 7739.63318532       55 0.01s                   1  = in city48

 * 7839.20690345       56 0.01s                   1  = in city38

 * 7865.27495064       59 0.01s                   0  = in city38

 * 7964.82553473       60 0.01s                   1  = in city38

 *  8016.3587808       63 0.01s                   0  = in city38

 * 8115.83884346       64 0.01s                   1  = in city38

 * 8163.57712754       67 0.01s                   0  = in city43

 ! Time = 0.01s, Average fail depth = 31, Memory usage = 4.0 MB

#CPOptimizer
#DecisionOptimization

Originally posted by: GGR

Hi Seigei

 

Actually the engine memory seems correct. Have you a minimal code you cand send us that illustrate the problem.

 

Cheers

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Dear Rogerie,

In attachment you will find a simple example which implements the 0-1 knapsack problem. Adding a customer constraint causes memory leakage as described above. It works fine when the line that adds the constraint is commented. Perhaps, I miss something, but it would be nice to determine the cause of the problem.

 

All the best,

 

Sergey

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Dear Sergei,

Thank you for the example.

I ran it and do not see any leakage, but stability in the memory consumption by the JVM :

I did tests using JDK 6 and 7, usingCOS 12.6, 126.1 ad 12.6.2 looking at the process virtual memory peak after 20s :

CPLEX_Studio126 with ibm-java-x86_64-60 vm peak: 1681876
CPLEX_Studio1261 with ibm-java-x86_64-60 vm peak: 1683732
CPLEX_Studio1262 with ibm-java-x86_64-60 vm peak: 1683936
CPLEX_Studio126 with jdk1.7.0_40-x86-64 vm peak: 5638828
CPLEX_Studio1261 with jdk1.7.0_40-x86-64 vm peak: 5621164
CPLEX_Studio1262 with jdk1.7.0_40-x86-64 vm peak: 5638828

One can notice that the jdk6 consumes much less than the jdk7.

I hope this helps.

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hi Rogerie,

But have you tried to uncomment the line

cp.add(new Constraint_(cp, x[i]));

in the code, and run it then for a comparison? Actually, it consumes much more memory in this case, and this consumption monotonically grows over time. Just run it not for 20 seconds but longer, and have a look whether the memory consumed by the process grows. I attach the version where the mentioned line is active, so it creates the problem case on my machine and our server. The previous version does not do this.

I am using JDK 8.45 but not sure whether it should influence. I can test it on an earlier version to check the difference.

 

All the best,

 

Sergey

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Hi sergei,

with a 20 seconds timeout , jdk8 shows a peak of 2gb, actually less than the the jdk7. I get these measurements when looking at the process VM size after 20 seconds, and with the Constraint_ enabled :

CPLEX_Studio126 with ibm-java-x86_64-80 timeout 20 vm-peak 2831380
CPLEX_Studio1261 with ibm-java-x86_64-80 timeout 20 vm-peak 2767664
CPLEX_Studio1262 with ibm-java-x86_64-80 timeout 20 vm-peak 2767864
CPLEX_Studio126 with ibm-java-x86_64-60 timeout 20 vm-peak 1681876
CPLEX_Studio1261 with ibm-java-x86_64-60 timeout 20 vm-peak 1683732
CPLEX_Studio1262 with ibm-java-x86_64-60 timeout 20 vm-peak 1683936
CPLEX_Studio126 with jdk1.7.0_40-x86-64 timeout 20 vm-peak 5750272
CPLEX_Studio1261 with jdk1.7.0_40-x86-64 timeout 20 vm-peak 5817632
CPLEX_Studio1262 with jdk1.7.0_40-x86-64 timeout 20 vm-peak 5686760

 

 strangest of all, the logs generated by these runs produce memory measurements for the solver < 1Mb :

KP-CPLEX_Studio1261-ibm-java-x86_64-60-timeout-20.log: ! Time = 16.88s, Average fail depth = 45, Memory usage = 868.7 kB
KP-CPLEX_Studio1261-ibm-java-x86_64-80-timeout-20.log: ! Time = 19.21s, Average fail depth = 43, Memory usage = 868.7 kB
KP-CPLEX_Studio1261-jdk1.7.0_40-x86-64-timeout-20.log: ! Time = 19.40s, Average fail depth = 43, Memory usage = 868.7 kB
KP-CPLEX_Studio1262-ibm-java-x86_64-60-timeout-20.log: ! Time = 12.81s, Average fail depth = 46, Memory usage = 868.8 kB
KP-CPLEX_Studio1262-ibm-java-x86_64-80-timeout-20.log: ! Time = 18.04s, Average fail depth = 43, Memory usage = 868.8 kB
KP-CPLEX_Studio1262-jdk1.7.0_40-x86-64-timeout-20.log: ! Time = 19.55s, Average fail depth = 43, Memory usage = 868.8 kB
KP-CPLEX_Studio126-ibm-java-x86_64-60-timeout-20.log: ! Time = 15.97s, Average fail depth = 45, Memory usage = 786.4 kB
KP-CPLEX_Studio126-ibm-java-x86_64-80-timeout-20.log: ! Time = 18.45s, Average fail depth = 43, Memory usage = 786.4 kB
KP-CPLEX_Studio126-jdk1.7.0_40-x86-64-timeout-20.log: ! Time = 18.78s, Average fail depth = 43, Memory usage = 786.4 kB

I am currently running longer tests 20 30 40 60 secs and will let you know the outcome.

A few questions:

  - are you using the IBM jvms ? If not, it'd be worth a try.

  - are you sure you are actually using the COS 12.6 or greater binaries? is LD_LIBRARY_PATH correctly set?

  - how many cores has your machine?

Cheers,

 

   Renaud

 

 

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Hi Sergei,

Here are several plots that shows how various jvms react regarding the vm size with your program. I'll run longer tests to see if it stablizes for jdk7 and 8.

#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hi Renaud,

- I have a windows-based laptop with 4 cores and Oracle JDK 8.45. I tried to solve the problem with CPOptimizer 12.6., 12.6.1 and 12.6.2. For all the versions the situation is like you see now. I am running the code under the Eclipse environment. So I feel LD_LIBRARY_PATH correct. At least, you have the same situation and I believe your paths are correct.

- I also run the same code on our Linux-based cluster with 48 cores and OpenJDK Runtime Environment version "1.7.0_75". It produces the same situation. But sure I am using just a single core everywhere.

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Hi Sergei,

 

It seems that JNI-related data  allocation by the JVM changes between Java6 and 7. Technically, a possible explanation is that the JVM fragments memory.

We are currently working to fix this problem. Please contact support to ask for a service request.

In the meantime, if you can use the IBM JDK6 for your experiments, it should work fine.

Yours sincerely,

 

         Renaud

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hi Renaud,

Thank you for your response.

As I understand ibm-java-x86_64-60 is the Linux version, isn't it? I have downloaded a version for Windows within some Rational package (if I am right with the name). So the installation package is called ibm-java-16sr14-win-x86_64.zip. It works much better, but looks like some leakage is still there. Starting with 32 Kb, after 7 min of computation the Resource Monitor reports just about 73 Kb of memory used. However, when the line with adding of the customer constraint is commented, consumed memory is stable and around 32 Kb. So, it still monotonically grows over time. As I understand, it must not grow at all. So, just to let you know, that the problem might be related not only to different versions of JVM.

 

For a comparison, Oracle 8.45 JDK uses on my side 2038 Kb in 7 min.

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Hi Sergei,

yes, it is the linux version. Please try with the latest version of ibm java6.

Thank you for the info, I'll check with longer runtimes.

Cheers,

 

 

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Sergei,

 

I confirm a slow increase using JDK6 on the long run. Please contact support, we need a service request to provide a fix for  this problem.

Cheers,

 

       Renaud

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hi Renaud,

I do not mind to send a service request. However, it looks like my CPLEX is not registered because of the general academic licence I use (because of IBM Academic Initiative program indeed). I am trying this https://www-947.ibm.com/support/servicerequest/newServiceRequest.action reference, but have no customer ID number to input to send a request. Do you know how I can do this?

Another fact about the problem, just note that more variables one adds with addVar() for propagation, the faster is leakage. So it seems like some data is created each time the execute() procedure is performed and then kept in the system forever.

Cheers,

Sergey

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Hi Sergei,

 

Thank you for the info. We are working on a fix for this problem.

In the meantine, as there is no obvious workaround for this bug, could you please detail your model to see if we could replace the custom constraint using built-in ones?

Best regards,

 

     Renaud

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Hi Renaud,

 

Unfortunately, I really need a customized constraint in my search. But have you tried ibm-java-x86_64-60 for Linux for a long run? Perhaps, I can use our server if this particular JVM works.

 

All the best,

 

Sergey

#CPOptimizer
#DecisionOptimization

Originally posted by: rdumeur

Dear Sergey,

 

Yes we tried 60 on a longer run and it slowly leaks.  The problem will be fixed in the next version of CPO.

Without having a service request, it seems we have no way to provide a hotfix, unfortunately.

 

Best regards,

 

         Renaud

#CPOptimizer
#DecisionOptimization

Originally posted by: SergeyPolyakovskiy

Can I make a service request being an academic and having the corresponding licence?

#CPOptimizer
#DecisionOptimization