Originally posted by: bindasmariner

I am trying to solve reasonable sized MIPs using the Python API of CPlex. The problem is that it suddenly stops execution in between on trying to solve slightly larger instances. These instances can be solved in the Cplex IDE/ OPL easily. Please find attached the crash report.
Please suggest me a remedy to this problem.

Regards
Saurabh
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

What version of CPLEX do you use?
Do you solve many models in a loop or do you only solve one single model and that fails?
Could you export the offending model to a SAV file and attach it here?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: bindasmariner

• I use CPLEX optimization studio Academic research edition 12.3
• I am trying to solve only single MILP model in this case.
• Please find attached the SAV file for the optimization problem.

Note: On trying the same MILP code with a smaller data set, it solves to optimality.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

How did you generate the SAV file? The file is in MPS and not in SAV file format. Moreover, the MPS file is corrupted as it contains duplicate rows (for example row r6).
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: bindasmariner

I am sorry. I was using a wrong extension for saving the SAV file. Now I have used the following command to save a SAV file in my Python code, after writing all my constraints.

c.linear_constraints.add(lin_expr = rows,senses = my_senses, rhs = my_rhs,names=my_rownames)
c.write("ModelCheck.sav")
c.solve()

PFA the new SAV file for the problem.

thanks and regards
Saurabh
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Your problem is invalid, it contains duplicate entries. You can see this in the interactive as follows:

CPLEX> set read datacheck y
New value for data consistency check indicator: yes
CPLEX> read ModelCheck.sav
CPLEX Error  1222: Duplicate entry.
No file read.

Please set the data check parameter to 1 right after instantiating the Cplex class. This will raise an exception at the point at which you create the duplicate entry. You then need to fix that.
Duplicate entries in rows/columns can cause any kind of trouble, including hangs or crashes.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: bindasmariner

Ok I understood this problem.
The problem data in my case is coming from a separate data file and duplicate rows or columns may arise when these are converted to a mathematical formulation in LP or other format. It is really difficult to avoid data duplication at the initial stage.
Can you please suggest me if there is a way to remove duplicate rows and columns before processing with the solver.

with regards
thanks
Saurabh
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

The question is what the meaning of the duplicate entries are.
For example, if you have a row in which a certain variable x appears twice, say once with coefficient 1 and once with coefficient 2. What would be the correct coefficient for x? Is it 1, 2, or 3?

The Python API is based on the low-level C API. For this reason, there is no automatic merging of duplicate variables in a constraint. The C++ API (or any other Concert API) would do this and produce a coefficient of 3x in the example above. With Python, you would need to do the merging yourself, for example by using a hash map to identify duplicate entries.
Tobias
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Do you read the data into the Python script and then create the model from within Python or do you turn the data directly into an LP file and then read that into Python.
In the first case you could just clean up things before submitting them to CPLEX as in this code snippet (note that 'ind' has duplicate names):

import cplex;
 
def cleanup(ind,val):
    clean = dict()
    for (i,v) in zip(ind,val):
        if i in clean:
            clean[i] = clean[i] + v
        else:
            clean[i] = v
    return cplex.SparsePair(clean.keys(), clean.values())
 
ind = [ "x1", "x2", "x3", "x1" ]
val = [ 1.0, 2.0, 3.0, 4.0 ]
 
c = cplex.Cplex()
c.variables.add(names = [ "x1", "x2", "x3", "x4" ])
c.linear_constraints.add(lin_expr = [ cleanup(ind, val) ],
                         senses = "L",
                         rhs = [ 5.0 ])
c.write("test.lp")

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: bindasmariner

Thanks a lot! This code snippet helped.

Actually I am trying to develop an iterative metaheuristic/LP algorithm for a very complex problem. At each iteration a part of the overall problem is solved using the metaheuristic and the values generated from this step are fed into a Linear Program to solve the complete problem (+The overall problem is a BLP consisting of binary vars and linear vars. The values of binary vars are found using metaheuristic and the remaining variables are found using LP+). Surprisingly these crashes occur for some problem instances, after running some iterations of the algorithm. Also for some other problem instances there are no crashes at all.

For the problem instances with crash reports, I manually searched for data duplication errors in my LP entries for rows and columns and could not find one so far. Although incorporating this code helped.

B/Regards
Saurabh
#CPLEXOptimizers
#DecisionOptimization