Originally posted by: amindehghanian

Hi,

I implemented a branch and cut using callable library.
For one easy instance, when my cuts are not puregable, I can find optimal solution in 5 seconds!

On the other side, if I let my cuts be purgeable, I will get the out of memory message after almost 2 minutes( I have assigned 2 Gb RAM to CPLEX). Actually, It gets stuck in the root node, it seems it generates and removes the same cut over and over. So, it makes sense to run out of memory.
But my question is that it looks unreasonable for me to run out of memory while I have just one node and about 500 cuts?

I also believe that my code should not have memory leakage since I can solve some hard instances after an hour on the same machine and the same settings.

Also according to what Tobias wrote here :
http://www.ibm.com/developerworks/forums/thread.jspa?threadID=416567
we should be conservative when we have non-purgeable cuts. How my observation can be justified?
Should we force our cuts or let them be purgeable?

Thanks,
Amin
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: amindehghanian

Also, I am using 64 bit version of CPLEX. Do I have to use syntax starting with CPXX rather than CPX to make use of 64 bit methods? what is the difference?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

In general, you should use CPXX methods no matter what.
On a 64bit architecture the situation is as follows:
Internally CPLEX works with 32bit variable indices and 64bit non-zero counts. This allows you to have up to 2^31-1 variables and up to 2^63-1 non-zeros.
The API offered by the CPX methods only works with 32bit non-zero counts. So if you use this API then CPLEX has to transform the 32bit non-zero counts to 64bit non-zero counts. This may introduce some overhead during model build time or when adding cuts (although the overhead is probably not too big).
The API provided by the CPXX methods uses types CPXDIM for variable indices and CPXNNZ for non-zero counts. These types expand to the actual types CPLEX uses internally. So if you consistently use CPXDIM, CPXNNZ and CPXX methods then no data transformation will occur.
Moreover, on a 32bit architecture the library internally uses 32bit non-zero counts only. The CPXX API takes care of this fact and if you use CPXDIM, CPXNNZ and CPXX methods consistently your code will not produce any data transformation, regardless of the architecture you compiler for is 32bit or 64bit.

To cut a long story short: If you use the CPX methods then data transformation may need to happen between your code and the core engine, if you use the CPXX methods no such transformation will be required.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

What you observe might actually be expected: If CPLEX purges a cut that you added and you re-add the cut then CPLEX may purge this cut again. If you keep adding this cut and CPLEX keeps purging it then you may observe an infinite loop.
CPLEX keeps a copy of the purged cut so adding the same cut again and again will eventually exhaust your memory.

However, that you run out of memory after 500 cuts sounds weird. How dense are these cuts? How many non-zeros do they have? Is it possible that your separation routine leaks memory? Could you check memory consumption at the beginning and the end of your separation routine?
Can you use tools like purify or valgrind to check whether the report any leaks?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: amindehghanian

Thanks Daniel for your detailed answer!
In order to make sure I understood what you said, I can use CPXX even on 32 bit systems, and the CPXX API will take care of everything, right?

As for the running out of memory issue, like you what is really surprising for me is that I am running out of memory after adding exactly 72 user cuts while I just have 121 binary and 2 continuous variables!!!!

I think my separation procedure does not have memory leak since for all other hard instances, my code works totally fine (after an hour, and adding almost 37000 cuts).

In addition, thanks Kai for your idea too! I already took care of that issue.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

> amindehghanian wrote:
> Thanks Daniel for your detailed answer!
> In order to make sure I understood what you said, I can use CPXX even on 32 bit systems, and the CPXX API will take care of everything, right?
>
Correct. Note that when using CPXX function you should use CPXDIM, CPXNNZ and CPXCNT instead of plain int.

> As for the running out of memory issue, like you what is really surprising for me is that I am running out of memory after adding exactly 72 user cuts while I just have 121 binary and 2 continuous variables!!!!
>
OK, that really sounds weird. Do you have a chance to run a memory profiler (valgrind, purify) on that? Or maybe use GetProcessMemoryInfo at the beginning and the end of your callback to double check that it is not your callback that is leaking memory.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: blaarg

Hello Amin,

do you check whether your cuts really cut off something or not?
In the past I had an issue similar to yours and I solved it by just adding a tolerance epsilon to the check whether a cut cuts off something or not, like:

IloNum eps = cplex.getParam(IloCplex::EpRHS);
if ( lhs > rhs + eps )
add(cut).end();
else
cut.end();

If you don't do so, you may add a cut that only violates the current relaxated solution a bit, like 10^-15, then cplex purges your cut, the relaxated solution doesn't change, you find the same cut again, and so on and so forth.

Best regards
Kai
#CPLEXOptimizers
#DecisionOptimization