Originally posted by: pfaut

Hi,

I am using the python api (with IDLE as user interface) of cplex and run into something kind of unexpected. I have a MIP which I need to solve several times with different objective functions.
Now i encounter the following issue:
Every time the problem is solved cplex takes another ~ 300-400 Megabiyte of ram and will not free it upon deletion of the instance. Is there some way to free up whatever memory was used by the solver? I would risk slowing down solving of further problems (at leasts thats my guess were all this memory is going to, storing earlier solutions for further solving attempts).

Thanks a lot.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Eumpfenbach

I'm just a user, no expert, but I believe this probably relates to your problem:

https://www-304.ibm.com/support/docview.wss?uid=swg1RS00149

Change the objective using c.objective.set_linear() rather than creating a new problem instance (if that is what you are doing. I am guessing).
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: pfaut

Unfortunately this isnt the problem.

I am just changing the objective of the same cplex instance each time (i.e. I do what you suggested). Which is why I am so puzzled.

But thanks for the idea.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

What version of CPLEX do you use?
Are you running on multiple threads? If you do, does it help to set the thread count to 1? Does it help to set environment variable MKL_DISABLE_FAST_MM to 1?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: pfaut

I am using CPLEX 12 (academic initiative version).

However, I think I found the leak somewhere outside cplex... did not check for some particular thing.

Sorry for the trouble.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: pfaut

Need to correct myself again.

It has nothing to do with the actual solver but something with access to it.

More precisely:
the variabes.get_names() function seems to leave some strange references intact.

A very small code example to show this would be something along the lines:

vars = []
for i in range(2000):
   vars.append(str(i))
 
c = cplex.Cplex()
cplex.variables.add(names=vars)
 
for v in vars:
    temp = c.variables.get_names()
    del temp

This will cause about 200 Mb of additional memory usage each time the last for loop is run.
My guess at the moment is some kind of leak in the get_names() function, which should be avoidable.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I confirm that there seems to be a memory leak in the get_names() function. We will investigate this and get back to you if we have more information.
Thank you for reporting the problem.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

OK, I think I found the problem. Unfortunately, there is no way to work around it other than not calling the function at all.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: amindehghanian

I hope that this issue appears just in Paython not in C?
Because I also need to use a loop in order to solve my different instances. I mean I want CPLEX to free the allocated memory at the end of each iteration.

Thanks,
Amin
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

It is only in the Python connector and only if you repeatedly call get_names().
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: pfaut

Well, for the time being I am working around it by keeping track of the variable names outside cplex. But I can't imagine that this issue can't be fixed.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

It will definitely be fixed in the next version of CPLEX.
#CPLEXOptimizers
#DecisionOptimization