Originally posted by: SystemAdmin

Hi All,

I need to solve a problem, which requires a certain separation problem to be called after every solution is found and checked if it is valid. In essence what I am doing is,
// Separation Problem

validate() {

% solve a problem here as well %

}

main() {

% define the master problem, solve it and pass the solution to validate and check if it is valid, runs in a loop %

}

The deal was that I hadn't come across this issue, until this point, where the memory usage shot up drastically while execution of the program. I have narrowed it down to that because I am initializing an instance of IloCplex everytime during validate, it allocates some memory which wasn't getting freed. However even after calling the IloCplex::end(), the issue persists. As a simple exercise I tried this,

cout << "Before Init." << endl;
getch();
IloEnv env;
IloCplex cplex(env);

cout << "After Init." << endl;
getch();

cplex.end();
env.end();
cout << "After End." << endl;
getch();

I check the memory usage by the program at each of the 3 checkpoints, namely before initialization of IloCplex instance, after initialization of IloCplex instance, and after calling IloCplex::end(). I was under an impression that the memory usage before initialization and after calling IloCplex::end() should be the same. However no memory seems to be getting free. Am I doing something wrong here? If not,is this an issue.. Any help in the regard would be really appreciated.

Thanks,
Avinash
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Which CPLEX version are you using? And which platform?
If you run your simple test program in a loop, do you see another memory increase after the second, third, ... execution of cenv.end()? Or is it only the first loop iteration that consumes some memory?

Tobias
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I am using the latest CPLEX v12.3.

For the second question, indeed, the memory usage by application starts increasing which is the point of concern in the first place for me.
Thanks,
Avinash
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Which port are you using? In particular, are you running on Linux? There is one difference for CPLEX on Linux to all other operating systems that could explain the behavior...
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

No I am running it on Win 7, x64-86.

I had tested the same on a Linux x86-64 as well, the behavior was the same though. I would be interested in knowing the difference.

-Avinash
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Concerning your little test program

IloEnv env;
IloCplex cplex(env);
cplex.end();
env.end();

You say that memory increases when you run this once. Did you try to run this in a loop? If you run this sequence of commands multiple times in the same code, do you observe further increase in memory usage after the first iteration? In other words, is this memory increase for your test code a one-time effect that occurs at initialization or is it really a leak?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Did you try to run this in a loop? If you run this sequence of commands multiple times in the same code, do you observe further increase in memory usage after the first iteration?

I tried running this in a loop. The memory increase seems to be a one time effect. However for that one time no memory gets freed.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: EdKlotz

Can you elaborate on what you mean when you say memory usage shot up drastically? How much memory was
in use before the problematic call to your validate() function, and by how much did it increase? Also,
what did you use to monitor memory usage on X64 Windows? Task Manager? And, how big was
your master problem?

Also, I have attached a modified version of the ilomipex2.cpp that tries to mimic the operations that
you have described. It essentially solves a master problem, then calls a subproblem routine that
just creates and destroys an IloEnv and IloCplex object. So, unlike true decomposition, the
master problem remains unchanged from one pass through the loop to the next. I tried to reproduce the behavior you
reported on both x64 Windows and Linux, but in both cases I could not. I see no significant increase in memory during the first or any other call to the subproblem routine. Furthermore, running the program under Valgrind indicates no memory leakage at all.

Now, this program is probably simpler than yours in that the "master" problem remains unchanged from
one pass in the loop; it just requires an MPS, LP or SAV file of a model to solve and the number of times to loop between solving the "master" problem and calling the subproblem routine. But,
perhaps you can use this program as a starting point, then incrementally modify it so it
performs some of the additional tasks in your own program. Or, move in the other direction,
systematically commenting out more and more of your own program until it resembles this program.
Hopefully this will enable you to isolate the specific part in your program that results in
the spike in memory usage. Perhaps you can start by running the attached program as is on a SAV file
exported via IloCplex::exportModel from your program.

Ed
#CPLEXOptimizers
#DecisionOptimization