Originally posted by: Robert Vincent

Hi All,

I'm running CPLEX 12.7.1 via Matlab 2018a on Ubuntu 18.04.  I limit memory usage using Param.workmem < Param.mip.limits.treememory.  Notwithstanding, memory usage exceeds both these limits!

Any help would be greatly appreciated,

Robert.

P.S.  I've attached the output from Matlab, showing the problem:

CPXPARAM_TimeLimit                               3600
CPXPARAM_WorkDir                                 Dir
CPXPARAM_WorkMem                                 16384
CPXPARAM_Threads                                 12
CPXPARAM_Parallel                                1
CPXPARAM_Read_APIEncoding                        "*"
CPXPARAM_Output_CloneLog                         1
CPXPARAM_MIP_Strategy_File                       0
CPXPARAM_MIP_Limits_TreeMemory                   24576
CPXPARAM_MIP_Strategy_Probe                      3

...

...

Elapsed time = 2997.94 sec. (2258693.23 ticks, tree = 27927.53 MB, solutions = 119)
 645449 627993   386103.0491   272   386586.1552   386098.6562 11749795    0.13%
 655447 637802   386098.8782   235   386586.1552   386098.6569 11922346    0.13%
 656467 639779   386099.6474   309   386586.1552   386098.6569 11956262    0.13%
 659908 643640   386098.9822   157   386586.1552   386098.6571 12030754    0.13%
 682638 664756   386099.6189   190   386586.1552   386098.6581 12411365    0.13%
 709759 692359   386098.8778   296   386586.1552   386098.6589 12916847    0.13%
 713537 696912   386098.9573   281   386586.1552   386098.6591 12996813    0.13%
 716357 699818   386099.0813   329   386586.1552   386098.6593 13048836    0.13%
 725497 708642   386098.9278   287   386586.1552   386098.6597 13212438    0.13%
 726851 709738   386099.1531   290   386586.1552   386098.6597 13232847    0.13%
 729063 710868   386098.8999   270   386586.1552   386098.6597 13252552    0.13%
 732138 714320   386099.0702   274   386586.1552   386098.6598 13313211    0.13%
 735935 718962   386098.6851   189   386586.1552   386098.6600 13394555    0.13%
 738684 721220   386098.7912   183   386586.1552   386098.6601 13432271    0.13%
 740764 722610   386098.7529   202   386586.1552   386098.6602 13455572    0.13%
 742055 724965   386099.5250   227   386586.1552   386098.6603 13491752    0.13%
 742965 725360   386099.1883   211   386586.1552   386098.6603 13498439    0.13%
Clique cuts applied:  92
Cover cuts applied:  135
Flow cuts applied:  1
Mixed integer rounding cuts applied:  8
Zero-half cuts applied:  69
Gomory fractional cuts applied:  17

Root node processing (before b&c):
  Real time             =   25.23 sec. (21740.49 ticks)
Parallel b&c, 12 threads:
  Real time             = 3574.92 sec. (2707331.23 ticks)
  Sync time (average)   =  685.52 sec.
  Wait time (average)   =    0.08 sec.
                          ------------
Total (root+branch&cut) = 3600.15 sec. (2729071.72 ticks)
CPXPARAM_TimeLimit                               3600
CPXPARAM_WorkDir                                 Dir
CPXPARAM_WorkMem                                 16384
CPXPARAM_Threads                                 12
CPXPARAM_Parallel                                1
CPXPARAM_Read_APIEncoding                        "*"
CPXPARAM_Output_CloneLog                         1
CPXPARAM_MIP_Strategy_File                       0
CPXPARAM_MIP_Limits_TreeMemory                   24576
CPXPARAM_MIP_Strategy_Probe                      3
118 of 119 MIP starts provided solutions.
MIP start 'm119' defined initial solution with objective 386586.1552.
Retaining values of one MIP start for possible repair.
 744245 727257   386098.8564   216   386586.1552   386098.6604 13535123    0.13%
Elapsed time = 6.21 sec. (3781.04 ticks, tree = 32525.91 MB, solutions = 119)
...

...
Elapsed time = 941.49 sec. (704081.04 ticks, tree = 41371.02 MB, solutions = 119)

*Ctrl^C* 

Sorry, I thought I had replied earlier but either I did not or the forum swallowed my answer.

It is expected that CPLEX may overshoot limits since checking for exact limit hits incurs too much runtime overhead. In particular, the tree size limit may be overshot significantly since computing the total tree size in a multi-threaded run is complicated.

However, I agree that in your case the amount by which CPLEX overshoots the limit looks too much. Can you share the model or code to reproduce this issue?

Originally posted by: Robert Vincent

Thanks for the reply.  So by the sounds, it's a feature and not a bug!  

I'm under an NDA with the company funding my PhD, so I won't be able to share the code, unfortunately.

My solution so far has been to simply reduce the runtime sufficiently such that it never overflows.  Fortunately, I can do this because I need a 'good' solution rather than the absolute 'optimal' (given a somewhat arbitrary cost function).  I am slightly surprised that it takes so much CPU to calculate the tree-size and yet it manages to display this figure periodically: is the addition of an {if x > y; end} type statement somewhere in the source code too much to ask?

Anyway, thanks again.  And hopefully if this happens to anyone else they too can limit the runtime to compensate!

The reason that this is complicate is not checking x>y. The problem is computing the current tree size. In a multi-threaded solve, each thread only has a partial view of the whole tree. Computing the current tree size exactly would require stopping all threads and merging the sizes of all these partial views. That is where the overhead comes from.

Note that there is indeed a bug with the tree size limit parameter, see my comment in this thread.