Originally posted by: Trino

Hi all.

Since my model has one exponential set of constraints I opted to use a Branch&Cut and add them as they are needed. I have a separation procedure which works on both, integers and fractional solutions to check if one or more constraints of this set are being violated.

If in the separation procedure I found one or more violated constraints and add them into the model, feasible integer solutions may be removed from the feasible space, so I by definition I should use the LazyConstraintCallbackI, but this callback is only called when candidate feasible (integer) solutions are found. I wanted to also being able to analyze the fractional solutions found along the way.

By definition I shouldn't use UserCutCallbackI since the cut would remove part of the feasible space of solutions, so which callback should I use to implement this?
Thanks.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Trino

Sorry about the title, I was going to ask something else and I forgot to change the title...
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

The major issue with lazy constraints is that certain dual reductions (and maybe some other stuff) have to be turned off to use them safely. My understanding is that you can add both a lazy constraint callback (which could have an empty body) and a user cut callback (which would actually add the constraints). The presence of the lazy constraint callback will cause CPLEX to turn off the things that need to be turned off.

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Trino

I tried to add a lazy constraint callback with an empty body and put my code only inside the user cut callback as you suggested, but if I do this, for some instances, cplex does not call the user cut callback, it just finish solving.

I also tried disabling some cplex parameters like the following:

cplex.setParam(IloCplex::Reduce, 0); //This is disabled when I add lazy constraints
 cplex.setParam(IloCplex::PreLinear, 0); //also disabled when I add lazy constraints
 cplex.setParam(IloCplex::PreInd, 0); //Disable presolve

But even disabling these options cplex finishes the execution before having the chance to enter the callback. The only way I could put the code to work was searching for cuts in both, the lazy constraint callback and the user cut callback, but I think this is a bad workaround I didn't want to rely on this.

I've attached a log of cplex using the empty lazy contraint callback and the user cut callback. Inside the user cut callback for this log I just put a

cout << "#####" << endl;

to indicate that cplex has called the callback, but as you can see it hasn't...
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

If presolve determines the optimal solution, the user cut callback is not called. I'm not sure, but this might also be true if a heuristic at the root node generates a feasible solution matching the LP bound. What I would do is put the "guts" of the callback in a separate function and call that function from both a lazy constraint callback and a user cut callback.

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Trino

In this case it wasn't that the presolve was finding the optimal solution, since even if I disable it, it keeps skiping the user cut callback.

Anyway, I think I found out why this is happening. For these small instances cplex is either able to find the optimal (for the model withouth the exponential set) by solving the root node or it only generates integer feasible solutions along the way. It seems like the user cut callback is not called when cplex finds a feasible integer solution, instead it calls either lazy constraints callback or incumbent callback.

Taking this into consideration if I only use a user cut callback, it's understandable why cplex skips it. In my case what I did in the end was adding both callbacks like @Paul suggested. The procedure in the user cut callback is able to analyse fractional and integer nodes, however for the lazy constraint callback I took advantage of the fact that it's called only for integer nodes and implemented a faster procedure for this case.

Please, if my conclusion about the user cut callback is wrong, correct me, but it's the only thing that made sense.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

The user cut callback is called at every node (possibly excepting the root node - I'm not sure about that) immediately after CPLEX gets done solving the node LP, provided the node is not pruned. Clearly there is no point in calling it if the node is infeasible. I suspect it is also not called if the node LP solution is integer-feasible with an objective value that meets the termination criteria relative to the best bound (which is automatically the case at the root). If the user cut callback was not being entered, that's probably what happened.

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I think instead of adding a lazy constraint callback with an empty body you should add a lazy constraint callback and a user cut callback that both implement the same separation procedure. For an integral node there is no need to invoke a cut callback and only the lazy constraint callback will be invoked. If that does never separate any cuts then you may end up accepting infeasible solutions.
There are several ways to easily write the separator in a way so that it can be used from two different callbacks. The most elegant way IMO is to write the separator as a template class for which the super class is a template argument. This way you can instantiate the template for a lazy constraint callback and a user cut callback and need to implement the separation code only once.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Trino

Thank you @Paul and @Daniel for the answers. I used both callbacks and for the Lazy I took advantage of the fact that all solutions would be integer, used a faster separation procedure and let the other one in the cut callback. Notice that both of them generates the same type of cut.

After that everything was working fine, until I tested one instance and found a case where cplex would throw the error 1217 telling that no solution exists. Of course the first thing I did was checking if I was adding invalid cuts, but in fact I didn't fing such cuts.

I,then, reduced the code to a minimum where the problem would still happened, in order to be easier to analyse. In this case I only have the lazy constraint callback, which is called once, add a total of 9 cuts, then cplex continues to solve and gives me this error. I have the optimal solution for this instance and none of the added cuts make this solution infeasible, so I don't know why cplex can't find it. I've attached the cplex log and in the end I added the optimal solution for this instance and (again) the cuts added by my separation procedure to be easier to analyse. Notice that whenever the lazy constraint callback is called I print the node and also the cuts added.
I tried, but I can't find out why this is happening, so if you could give my some insights I would appreciate. Thanks.

ps: I'm using cplex 12.5
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Add the nine cuts you generated to your test model and export it as a SAV file. It would be a good idea also to export your parameter settings (if you have changed any). In the interactive optimizer, read in both the SAV file and the parameter file, and if necessary turn off dual reductions. (I'm not sure if the changes CPLEX makes when it sees the lazy constraint callback show up in the parameter file.) Run the conflict refiner, and see if it identifies a problem. If not, try solving the MIP model while collecting kappa statistics (to check for numerical instability).

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Trino

But, you see, when I take these same cuts and add them directly into the model and don't use branch&cut, cplex finds a solution. If the cuts were really infeasible the result should be the same. I think it might be the case of some precision error when adding the cut in cplex inside the callback, but since in concert I can't export the model with the cuts (or so I read) I can't be sure. I would need to chance all my code for the callable library that as far as I know is the only one that suports exporting the model including the cuts.

Any ideas?

#CPLEXOptimizers
#DecisionOptimization

I once wrote a code that takes a Concert-C++ model (with only linear constraints, no logical or indicator constraints) and solves it using the callable library, including a cut callback. I think you could use that to actually dump the model including cuts to disk. If you want to have this code (it is only 350 lines or so) then please contact me: daniel(dot)djunglas(at)de(dot)ibm(dot)com.

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Trino

I've send you an email Daniel, thanks.

#CPLEXOptimizers
#DecisionOptimization

Sorry, that email will never reach me because I mistyped my address :-( The correct address is daniel(dot)junglas(at)de(dot)ibm(dot)com (no 'd' after the first dot).

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Trino

No problem, I've re-sent the email to this address...let me know if you don't get it.

 

Thanks.

#CPLEXOptimizers
#DecisionOptimization