Originally posted by: nr83

Hi, I'm new on this forum, I hope my question is interesting for everybody.

I'm using cplex 12.5.1 java libraries, my code looks like this:

    ...
    IloCplex model = new IloCplex();

    //add objective function and  constraints in model
    ...
    
    model.setOut(null);
    model.solve();
    final double objValue = model.getObjValue();
    model.end();
    return objValue;

I do not know why the returned object value is 0, while I know it should be >0.
If I modify the code as:

    ...
    IloCplex model = new IloCplex();

    //add objective function and  constraints in model
    ...
    
    model.exportModel("model.lp");
    model.end();
    IloCplex m2 = new IloCplex();
    m2.importModel("model.lp");
    m2.setOut(null);
    m2.solve();
    final double objValue = m2.getObjValue();
    m2.end();
    return objValue;

it works properly.
Anybody knows what is happening?

With Concert, there is this concept of extraction ... it could be related to your issue.  For the first case, where it doesn't work as you expect, can you provide a complete example?  Ideally, it would be something very simple that isolates the issue.

Originally posted by: nr83

I don't think the problem is the extraction, because is actually the extraction that makes the program working properly.

The problem is that if I don't extract the model in a text file and then import it in a IloCplex object, the program seems to work as a different one, i.e., as it would not have all the constraints I generated before calling model.slve().

The constraints are generated with the methods IloCplex.Ge(), IloCplex.Le() and IloCplex.Eq().

I tried to solve a very simple linear program and it works, but with big programs I need to do the trick "export and import" otherwise it does not work as expected.

below part of the code I use to build objective function and constraints

                 final IloCplex cplex = new IloCplex();
            IloIntVar xVar;
            IloIntVar zVar;
            IloIntVar qVar;
            
            final IloLinearIntExpr objective = cplex.linearIntExpr();
            final List<IloIntExpr> sumXvars18 = new ArrayList<IloIntExpr>();
            final List<IloIntExpr> sumZvars17_bs = new ArrayList<IloIntExpr>();
            
            for(final Resource r : resources)
            {
                final String id = r.getId();
                xVar = cplex.boolVar("x"+id);
                objective.addTerm(r.getEC(), xVar);//objective function
                sumXvars18.add(cplex.prod(r.getCost(), xVar));//c18 sum
                
                for(final Location l : r.getDom())
                {
                    zVar = cplex.boolVar("z"+l.getId()+id);
                    cplex.addEq(cplex.diff(zVar, xVar), 0);//c5
                    sumZvars17_bs.add(zVar);
                }
                
                int type = r.getType();
                
                if(type==3 || type==4)
                    for(final Node n : r.getNodes())
                    {
                        qVar = cplex.boolVar("q"+n.getLocation().getId());
                        zVar = cplex.boolVar("z"+n.getLocation().getId()+r.getId());
                        cplex.addLe(cplex.sum(zVar, qVar), 1);//c16
                    }
            }
            

            //convert the List<IloIntExpr> into an array IloIntExpr[]
            IloIntExpr[] sumZvarsArray_bs = sumZvars17_bs.toArray(new IloIntExpr[sumZvars17_bs.size()]);
            cplex.addGe(cplex.sum(sumZvarsArray_bs), res.keySet().size());//c17
            
            cplex.addMinimize(objective);//objective function
            IloIntExpr[] sumXvarsArray = sumXvars18.toArray(new IloIntExpr[sumXvars18.size()]);
            cplex.addLe(cplex.sum(sumXvarsArray), 100);//c18

The memory (binary) -> .lp (ASCII/UTF-8/whatever) -> memory (binary) chain introduces small changes to model coefficients. Try saving to .sav rather than .lp and reloading. If the objective value for the .sav version reverts to 0, you likely have numerical stability issues in your model. If the .sav version produces the same positive objective value as the .lp version, then something else is going on. If it differs from both the in-memory and .lp versions, I would again suspect numerical issues.

Originally posted by: nr83

something very weird happened.

I saved to .sav instead of .lp and I reloaded it as you suggested. The .sav version produced a different objective value than the .lp version. Then I tried to run the in memory version again, without any export-import operation, and it produced an objective value equal to that was previously produced by the .sav version. It seems that after I exported and imported the .sav version something changed and now the in-memory version works as expected.

To summarize what happened in chronological order:

1. I ran the in-memory version IloCplex model and the objective value was 0, which is a wrong result;
2. I exported the in-memory version model in a .lp file and imported the same file in a new IloCplex instance model2. Then, model2.solve() produced an objective value =x (>0);
3. I exported the in-memory version model in a .sav file and imported the same file in a new IloCplex instance model3. Then, model3.solve() produced an objective value =y (>0);
4. I ran the in-memory version IloCplex model and the objective value was y. From now on every time I run the in-memory version model, it produce an objective value =y.

​I guess that the right result is y, but I cannot check it since when I try to obtain the values of the variables, java returns a nullPointerException. The code I use to get the variables values is the following:

                  model.solve();

            List<String> activeVariables= new ArrayList<String>();
        IloLPMatrix lp = (IloLPMatrix)model.LPMatrixIterator().next();
  
        IloNumVar[] var = lp.getNumVars();//<-----------------nullPointerException
            
            for(int i=0; i<var.length; i++)
            {
                IloNumVar iloNumVar = var[i];
                String name = iloNumVar.getName();
                
                if(name.charAt(0)=='x')
                    if(model.getValue(iloNumVar)==1)
                        activeVariables.add(name);

            }

The x variables are those involved in the objective function. The nullPointerException is given by lp.getNumVars(), indicated by the arrow.

Have you an idea about why the in-memory version is now working? And have you an idea about why java returns this exception?

I don't use the matrix constructors, so I may be off here, but I think the reason for the null pointer exception is that 'model' does not contain any LPMatrix instances, since you built it by adding IloRanges. Just store the variables in a global vector during construction and refer to that when you want to call getValues().

Regarding whether "y" is the correct objective value or not, I suggest collecting and checking kappa statistics first. When the small differences between .lp and .sav versions of the same model produce notably different results, my first inclination is to suspect  numerical instability.

As to why the in-memory version might work after running the .sav version, I can only speculate. It may be that CPLEX hot-started the solution of the root LP in the post-SAV run of the in-memory model using an LP basis left over from the SAV solution, and that might have led it down a different path than when it first ran the in-memory version. You could test that theory by turning off the advanced start switch before doing the runs, but I think checking kappa stats would be a higher priority.

Like Paul said, the null pointer exception most likely occurs because lp is null. And this is because you don't have any IloLPMatrix instances in your model, you only add IloRange instances.

As for the import/export mystery: Do I understand correctly that now the model solves correctly even if you do not import/export anything?

Originally posted by: nr83

I fixed the nullPointerException very quickly, thanks Paul for your help.

Regard the export/import mystery, it is still a mistery for me, but yes now the model solves correctly even if I do not use the trick.

I tried to get the kappa statistics as suggested by Paul:

            model.solve();
            System.out.println(model.getQuality(IloCplex.QualityType.Kappa));

and the output is the following:

           0.0
           -1.0

The measures you want are IloCplex.QualityType.KappaSuspicious and IloCplex.QualityType.KappaUnstable. I'm not sure what IloCplex.QualityType.Kappa gets you (and the docs are rather unhelpful here).

Originally posted by: nr83

I tried with:

            model.solve();
            System.out.println(model.getQuality(IloCplex.QualityType.KappaSuspicious));

and

            model.solve();
            System.out.println(cplex.getQuality(IloCplex.QualityType.KappaUnstable));

and in both cases COPLEX sends the exception:

            ilog.cplex.CpxException: CPLEX Error  1269: No kappa statistics are available.

Sorry, you need to set the parameter loCplex.Param.MIP.Strategy.KappaStats to 0, 1 or 2. (The default, -1, records no statistics.) Higher values essentially increase the sample size but possibly slow down the solver.

Originally posted by: nr83

with

           model.setParam(IloCplex.Param.MIP.Strategy.KappaStats , 0);

CPLEX sends the same exception:

           ilog.cplex.CpxException: CPLEX Error  1269: No kappa statistics are available.

while with

           model.setParam(IloCplex.Param.MIP.Strategy.KappaStats , 1);

and                  model.setParam(IloCplex.Param.MIP.Strategy.KappaStats , 2);

both 

           model.getQuality(IloCplex.QualityType.KappaUnstable));

and                  model.getQuality(IloCplex.QualityType.KappaSuspicious));

return 0.

Just to be clear, you set the KappaStats parameter before solving the model and called getQuality after solving the model, correct?

Originally posted by: nr83

yes, I set the KappaStats parameters then I solve the model and then I call getQuality,

otherwise if I set the parameters after solving the model CPLEX sends the exception

ilog.cplex.CpxException: CPLEX Error  1269: No kappa statistics are available.

Well, zero suspicious or unstable bases is quite good, and makes it rather unlikely that poor scaling or other numerical issues are the culprits. I wonder if something occurs during presolve that would explain it?

Originally posted by: nr83

do you mean i should set IloCplex.Param.Preprocessing.Presolve to false to be sure to have the exact result?

I actually did that and the objective function has the same value. Obviously the time taken was 20 times larger.

Yes, I meant set Presolve to false, but not to confirm the objective value. At this point, I'm willing to believe that "y" is correct. I was thinking in terms of figuring out why the initial run was wrong -- maybe presolve got something wrong that it got correct after the .sav business. If you cannot reproduce the original error, though, then there's no way to test this.
 

Originally posted by: nr83

thanks a lot for you help, the important is that finally I got it working.