Originally posted by: claud10

Hi,

I have a code that is working on CPLEX 10.0 using the C++ Concert Technology callable library. Now, I'm testing the academic license of version 12.1 and I m having problems running my programs. I get to compile and link the code but at the moment of running I get the exception

*** glibc detected *** ./clrp: free(): invalid pointer: 0x00000000075bfb08 ***

I have run dbg on my file and the problem seems to be in the middle of a cutcallback when calling the function IloCutCallbackI::add().

Is there any recommendation for migrating old code? Any ideas?

Thanks in advance,

Claudio
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

As a first check, you should compile and run the C++ examples that contain a cut callback. If these work fine (and they should), you should compare your code to the example implementation.

One issue could be multi-threading. Are you using parallel MIP (can be seen in the log file; the number of threads is reported)? Is your cut callback thread safe?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: claud10

Thanks Tobias,

I have detected where the problem occurs but still do not know why. I have a global variable called

IloRangeArray Cuts;

This variables is declared out of the main as static. It is intended to keep the cuts found so far by the cut callback. While in the CutCallback, what I do is try to access that variable and adding cuts using the function

IloRangeArray::add(IloRange rng)

and this is where the programs crashes. I compared my program against the example files but it is not that easy since it is a big code with a lot of external libraries. Even though I can not see any major difference. I also tried the examples and they all compile, link and run flawlessly. Finally, I verified that the program runs as single-threaded (CPLEX passes the message Parallel mode: none, using 1 thread.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: claud10

Forgot to say that if I deactivate the callbacks the program continues normally (and of course returns a solution that does not make much sense).
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Laci Ladanyi

Having a global variable makes your code non thread safe. If you need to save the cuts in a single pool then any time you write into it (and depending on how you use it at reads as well) you need to lock it so other threads would be forced to wait (if they want to access the global var) until the writing thread is done writing. However, acquiring a lock is relatively expensive (time-wise), so it should be done as infrequently as possible. E.g., collect in a local var as much of what you want to write, then do it in one go.

--Laci
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Laci,

this was also my first guess, but as you can see in the earlier post, the code does not run in parallel mode. Therefore, the reason for the crash must be different.
Claudio: is it possible that you construct a short example that shows the crash? This would help us a lot with debugging or pointing you to a mistake in how you use Concert, whatever applies.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Laci Ladanyi

I guess I misunderstood the sentence: "Finally, I verified that the program runs as single-threaded (CPLEX passes the message Parallel mode: none, using 1 thread." I interpreted that as the code runs correctly in single-threaded mode but fails in multi-threaded mode. If the code fails in single threaded mode then you are right, the problem is elsewhere...

Claudio, the short example is the best, but if that's not an option, could you run the code through valgrind and post the backtrace?

--Laci
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: claud10

Thanks for your help guys. Here is the output of valgrind

IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
Tried aggregator 1 time.                                                                                                         
MIP Presolve eliminated 361 rows and 362 columns.                                                                                
Reduced MIP has 263 rows, 530 columns, and 1795 nonzeros.                                                                        
Reduced MIP has 215 binaries, 210 generals, 0 SOSs, and 0 indicators.                                                            
Presolve time =    0.72 sec.                                                                                                     
Warning: Control callbacks may disable some MIP features.                                                                        
Probing time =    0.22 sec.                                                                                                      
Clique table members: 126.                                                                                                       
MIP emphasis: best bound.                                                                                                        
MIP search method: traditional branch-and-cut.                                                                                   
Parallel mode: none, using 1 thread.                                                                                             
Tried aggregator 1 time.                                                                                                         
No LP presolve or aggregator reductions.                                                                                         
Presolve time =    0.08 sec.                                                                                                     
Initializing dual steep norms . . .                                                                                              
 
Iteration log . . .
Iteration:     1   Dual objective     =            42.118763
Root relaxation solution time =    0.98 sec.                
 
        Nodes                                         Cuts/ 
   Node  Left     Objective  IInf  Best Integer     Best Node    ItCnt     Gap         Variable B NodeID Parent  Depth
 
      0     0      291.1728    11                    291.1728       42         
==11341== Invalid free() / delete / delete[]                                   
==11341==    at 0x4C24D68: free (vg_replace_malloc.c:325)                      
==11341==    by 0xC081B5: IloConcertAlloc::free(void*, unsigned long) const (in /home/ccontard/cvsworkdir/lrp/src/clrp)
==11341==    by 0xBF61D3: IloArrayI::grow(long) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                            
==11341==    by 0xC66506: IloExtractableArray::add(IloExtractable) (in /home/ccontard/cvsworkdir/lrp/src/clrp)         
==11341==    by 0x4E5119: IloRangeArray::add(IloRange) (ilolinear.h:332)                                               
==11341==    by 0x4B60C6: NoCutsI::main() (f2.cpp:2808)                                                                
==11341==    by 0x518D03: IloCplexCallbackManager::call(IloCplex::CallbackI*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)
==11341==    by 0x519DEA: cutcallback (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                        
==11341==    by 0x7ADB28: _8c13b877e24ea2c917aa6aba8adfc9da (in /home/ccontard/cvsworkdir/lrp/src/clrp)                  
==11341==    by 0x737FFC: _2d5477278040a7aec7eb2a1a71d258d7 (in /home/ccontard/cvsworkdir/lrp/src/clrp)                  
==11341==    by 0x732FBD: _30e44abd6402add8902a05197fddbbb5 (in /home/ccontard/cvsworkdir/lrp/src/clrp)                  
==11341==    by 0x72B484: _3a03ddf8943f2f4879d8a97634f5eb40 (in /home/ccontard/cvsworkdir/lrp/src/clrp)                  
==11341==  Address 0xafce6c8 is 3,992 bytes inside a block of size 4,008 alloc'd 
==11341==    at 0x4C255E4: operator new[](unsigned long) (vg_replace_malloc.c:264)                                       
==11341==    by 0xBD88A7: IloFixedSizeAllocatorI::allocateBlock(IloMemory*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)  
==11341==    by 0x5373B4: IloLPExtractorManager::makeData(void*, int, int) (in /home/ccontard/cvsworkdir/lrp/src/clrp)   
==11341==    by 0x537DD4: IloLPExtractorManager::doMakeIndex(IloNumExprI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)
==11341==    by 0x53FFC5: IloDefaultLPExtractor::extract2Row(int*, IloRangeI::LinearIterator, double) (in /home/ccontard/cvsworkdir/lrp/src/clrp)
==11341==    by 0x5402FC: IloDefaultLPExtractor::extractRange(IloRangeI const*, int**) (in /home/ccontard/cvsworkdir/lrp/src/clrp)               
==11341==    by 0x537598: IloLPExtractorManager::doExtract(IloExtractableI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                   
==11341==    by 0x53BA9C: IloDefaultLPExtractor::extractModel(IloModelI const*, void**) (in /home/ccontard/cvsworkdir/lrp/src/clrp)              
==11341==    by 0x537598: IloLPExtractorManager::doExtract(IloExtractableI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                   
==11341==    by 0x4FAAA7: IloCplexI::extract(IloExtractableI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                 
==11341==    by 0xBD667B: IloAlgorithmI::load(IloModelI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                      
==11341==    by 0xBD3298: IloAlgorithm::extract(IloModel) const (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                      
==11341==                                                                                                                                        
==11341== Invalid free() / delete / delete[]                                                                                                     
==11341==    at 0x4C24D68: free (vg_replace_malloc.c:325)                                                                                        
==11341==    by 0xC081B5: IloConcertAlloc::free(void*, unsigned long) const (in /home/ccontard/cvsworkdir/lrp/src/clrp)                          
==11341==    by 0xBF6252: IloArrayI::grow(long) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                                      
==11341==    by 0xC66506: IloExtractableArray::add(IloExtractable) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                   
==11341==    by 0x4E5119: IloRangeArray::add(IloRange) (ilolinear.h:332)                                                                         
==11341==    by 0x4B60C6: NoCutsI::main() (f2.cpp:2808)                                                                                          
==11341==    by 0x518D03: IloCplexCallbackManager::call(IloCplex::CallbackI*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                        
==11341==    by 0x519DEA: cutcallback (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                                                
==11341==    by 0x7ADB28: _8c13b877e24ea2c917aa6aba8adfc9da (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                          
==11341==    by 0x737FFC: _2d5477278040a7aec7eb2a1a71d258d7 (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                          
==11341==    by 0x732FBD: _30e44abd6402add8902a05197fddbbb5 (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                          
==11341==    by 0x72B484: _3a03ddf8943f2f4879d8a97634f5eb40 (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                          
==11341==  Address 0xafe90d8 is 3,992 bytes inside a block of size 4,008 alloc'd 
==11341==    at 0x4C255E4: operator new[](unsigned long) (vg_replace_malloc.c:264)                                                               
==11341==    by 0xBD88A7: IloFixedSizeAllocatorI::allocateBlock(IloMemory*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                          
==11341==    by 0x5373B4: IloLPExtractorManager::makeData(void*, int, int) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                           
==11341==    by 0x5374F1: IloLPExtractorManager::pushConData(long, int, int, void*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                  
==11341==    by 0x5377A5: IloLPExtractorManager::doExtract(IloExtractableI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                   
==11341==    by 0x53BA9C: IloDefaultLPExtractor::extractModel(IloModelI const*, void**) (in /home/ccontard/cvsworkdir/lrp/src/clrp)              
==11341==    by 0x537598: IloLPExtractorManager::doExtract(IloExtractableI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                   
==11341==    by 0x4FAAA7: IloCplexI::extract(IloExtractableI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                 
==11341==    by 0xBD667B: IloAlgorithmI::load(IloModelI const*) (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                      
==11341==    by 0xBD3298: IloAlgorithm::extract(IloModel) const (in /home/ccontard/cvsworkdir/lrp/src/clrp)                                      
==11341==    by 0x511C8C: IloCplex::IloCplex(IloModel) (in /home/ccontard/cvsworkdir/lrp/src/clrp)
==11341==    by 0x49E02D: branchandcut(std::basic_ifstream<char, std::char_traits<char> >*, std::basic_ofstream<char, std::char_traits<char> >*, std::basic_ofstream<char, std::char_traits<char> >*, std::basic_ofstream<char, std::char_traits<char> >*, std::basic_ofstream<char, std::char_traits<char> >*, std::basic_ofstream<char, std::char_traits<char> >*, std::basic_ofstream<char, std::char_traits<char> >*, int, bool) (f2.cpp:6878)

The program indeed continues and does not throw any other error. Here is the next part of it.

0     0      379.6107    31                    Cuts: 33      108                                                                                                                          
      0     0      390.6456    50                    Cuts: 56      152                                                                                                                          
      0     0      395.4344    61                    Cuts: 35      201                                                                                                                          
      0     0      402.7973    67                    Cuts: 85      281                                                                                                                          
      0     0      405.3898    66                    Cuts: 48      306                                                                                                                          
      0     0      407.3443    64                    Cuts: 50      336                                                                                                                          
      0     0      407.9150    75                    Cuts: 35      374                                                                                                                          
      0     0      408.3075    70                    Cuts: 39      402                                                                                                                          
      0     0      408.6212    70                   Fract: 36      420                                                                                                                          
      0     0      408.7422    74                     Cuts: 5      434                                                                                                                          
      0     0      408.7454    75                     User: 1      439                                                                                                                          
 
Repeating presolve.
Tried aggregator 1 time.
Represolve time =    0.43 sec.
Probing time =    0.08 sec.   
Clique table members: 126.    
MIP emphasis: best bound.     
MIP search method: traditional branch-and-cut.
Parallel mode: none, using 1 thread.          
Root relaxation solution time =    0.26 sec.  
 
        Nodes                                         Cuts/ 
   Node  Left     Objective  IInf  Best Integer     Best Node    ItCnt     Gap         Variable B NodeID Parent  Depth
 
      0     0      408.7454    75                    408.7454      582         
      0     0      409.0834    68                    Cuts: 11      607         
      0     0      409.2751    74                    Cuts: 12      626         
      0     0      409.2834    77                     User: 1      630         
      0     0      409.2845    77                     User: 2      631         
      0     2      409.2845    77                    409.2845      631                                 0             0
Elapsed real time =  47.03 sec. (tree size =  0.00 MB, solutions = 0)                                                 
*    39     0      integral     0      424.8991      424.8991     2168    0.00%           id599 D     39     37      8
                                                    User: 127                                                         
 
Zero-half cuts applied:  4
Gomory fractional cuts applied:  7
User cuts applied:  281           
Number of nodes = 40              
Lower Bound = 424.899             
Upper Bound = 424.899             
----------------------------------------------------------------
TOTAL CUTS AND TIME SPENT IN SEPARATION                         
Cover Inequalities                              : 0     0:0:0:0 
Vehicle Capacity Constraints                    : 48    0:0:9:390
Z-Vehicle Capacity Constraints                  : 219   INCLUDED IN VEHICLE CAPACITY SEPARATION
Simple Route Cuts                               : 0     0:0:0:0                                
Chain Barring Constraints                       : 14    0:0:4:400                              
Effective Facility Capacity Constraints         : 0     0:0:0:0                                
Path Inequalities I                             : 0     0:0:0:0 0:0:0:0                        
Path Inequalities II                            : 0     0:0:0:0 0:0:0:0                        
Facility Capacity Constraints                   : 0     0:0:0:0                                
Location-Routing Combs                          : 0     0:0:0:0                                
Framed Capacity Inequalities                    : 0     0:0:0:0                                
Location GLM                                    : 0     0:0:0:0                                
MultiStar Inequalities                          : 0     0:0:0:0                                
Generalized Large MultiStar Inequalities        : 0     0:0:0:0                                
Strengthened Comb Inequalities                  : 0     0:0:0:0                                
Hypotour Inequalities                           : 0     0:0:0:0                                
----------------------------------------------------------------                               
CUTS AT ROOT NODE:                                                                             
Cover Inequalities                              : 0                                            
Vehicle Capacity Constraints                    : 26                                           
Z-Vehicle Capacity Constraints                  : 128                                          
Simple Route Cuts                               : 0                                            
Chain Barring Constraints                       : 0                                            
Effective Facility Capacity Constraints         : 0                                            
Path Inequalities I                             : 0                                            
Path Inequalities II                            : 0                                            
Facility Capacity Constraints                   : 0                                            
Location-Routing Combs                          : 0                                            
Framed Capacity Inequalities                    : 0                                            
Location GLM                                    : 0                                            
MultiStar Inequalities                          : 0                                            
Generalized Large MultiStar Inequalities        : 0                                            
Strengthened Comb Inequalities                  : 0                                            
Hypotour Inequalities                           : 0                                            
----------------------------------------------------------------                               
ADDITIONAL (CPLEX 10.0) CUTS                                                                   
Cliques                                         : 0                                            
Covers                                          : 0                                            
Disjunctive                                     : 0                                            
Flow Covers                                     : 0                                            
Flow Paths                                      : 0                                            
Fractional Cuts                                 : 0                                            
GUB Covers                                      : 0                                            
Implied Bounds                                  : 0                                            
Mixed-Integer Rounding                          : 0                                            
----------------------------------------------------------------                               
CPU TIME                                                                                       
Total   : 00:01:39:719 (hh:mm:ss:mss)                                                          
Root    : 00:00:46:899 (hh:mm:ss:mss)                                                          
----------------------------------------------------------------                               
NUMBER OF NODES : 40                                                                           
LOWER BOUND      : 424.899
UPPER BOUND      : 424.899
----------------------------------------------------------------

The problem trying to solve correponds to an instance of the capacitated location routing problem.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: CPXfanatic

I see in your log that some CPLEX cuts are omitted. If your intention is to enable/disable/control certain CPLEX cuts, you should take into account that 2 new cuts were added in CPLEX 11 and 12: ZeroHalf and MultiCommodityFlow.

ADDITIONAL (CPLEX 10.0) CUTS
Cliques : 0
Covers : 0
Disjunctive : 0
Flow Covers : 0
Flow Paths : 0
Fractional Cuts : 0
GUB Covers : 0
Implied Bounds : 0
Mixed-Integer Rounding : 0
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: claud10

Thank you all. In fact the trick suggested worked perfectly but now I m stucked into something else related. Indeed, after the optimization finishes (i.e., after having run the IloCplex::solve() function) I need to run another optimization using the same variables and constraints, so I simply define it in the same environment and use the function IloCplex::addCuts(IloConstraintArray) in order to add the cuts stored during the cut callback, getting the same error. Any ideas?

Thanks,

Claudio
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: claud10

I have modified one of the examples (iloadmipex5) in order to illustrate the problem.
Here is attached the file. You should be able to compile and link it just as if it was another example.

Thanks for your feedback guys.
Cheers,

Claudio
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: RWunderling

Hi Claudio,

I could reproduce the problem you are seeing, and identified two sources for it:

1. This is probably just a copy-past issue:

cut = (lhs[i] <= xrhs);
CUTS.add(cut);
add(cut).end(); // <- You delete the cut you want to save?!
rhs[i] = IloInfinity;

2. This one is very subtle and we need to investigate further. When running in a
solve(), a different memory manager gets used. By saving the cuts generated during
the solve in a container created before mixes both memory managers. You can work
around this issue like this:

IloRangeArray CUTS = 0;

// Use a user cut callback when the added constraints strengthen the
// formulation. Use a lazy constraint callback when the added constraints
// remove part of the feasible region. Use a cut callback when you
// are not certain.

ILOUSERCUTCALLBACK3(CtCallback, IloExprArray, lhs, IloNumArray, rhs, IloNum, eps) {
IloInt n = lhs.getSize();
if ( CUTS.getImpl() == 0 )
CUTS = IloRangeArray(getEnv());
for (IloInt i = 0; i < n; i++) {
IloNum xrhs = rhs[i];
if ( xrhs < IloInfinity && getValue(lhs[i]) > xrhs + eps ) {
IloRange cut;
try {
cut = (lhs[i] <= xrhs);
CUTS.add(cut);
// add(cut).end();
rhs[i] = IloInfinity;
}
catch (...) {
cut.end();
throw;
}
}
}
}

Hope this helps, and thank you for pointing out the issue. We will investigate
further why and when the change was introduced.

• Roland

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: claud10

Hi Roland,

Thank you very much for your feedback. Indeed,what you suggest seems to work and solve the issue, at least for the example. Now I have to work on getting my code working again. Anyway, I would really appreciate if you can check what happened in btw 10.0 and 12.1 for this to occur.

Thanks again and have a nice weekend,

Claudio
#CPLEXOptimizers
#DecisionOptimization