Originally posted by: BastianMeier

Hello,

i am using Java to generate/solve my problem. I wonder if it is possible to get a list of all variables of the active model before/after it is solved.

I haven't found any "IloCPlex.getVariables()"-function. By iterating through "IloCplex.getModel().iterate()" i only get CpxRange-Objects (which seems to contain some but not all of the restrictions of my optimization problem). Using "IloCplex.lpmatrix()" cplex is generating an empty matrix.

On the other side when importing the optimzation problem through "IloCplex.import(file.lp)" i can iterate through "IloCplex.getModel().iterate()" which contains the IloLpMatrix-Object. This object allows me to access all variables of the loaded model using "getNCols()" and "IloLpMatrix.getNumVar(1..getNCols())". But i do not want to export/import my model because it is taking a lot of time (file > 1GB).

Of course i can collect all variables in a g global list when generating them but having a list generated by Cplex would be much easier for me.

So far i wasn't able to answer my question by searching the internet and the Jave Reference manual.

Is it possible to get access to all variables (or the IloLpMatrix) of the active model?
Thanks for your time,
Bastian
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

As you discovered, there is no API function for this. I put some Java code for extracting variables in this blog post, but it comes with no guarantees.

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: BastianMeier

Hi Paul,

as much as i understand your code you are iterating through the restrictions, objective etc. of the active model to extract the variables used in those. I wonder why i did not found this page via Google.

Thank you very much for your help (again).

Bastian
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

> BastianMeier wrote:

> as much as i understand your code you are iterating through the restrictions, objective etc. of the active model to extract the variables used in those.

Yes, that is correct. What it lacks in elegance it makes up for in cumbersomeness. :-)

> I wonder why i did not found this page via Google.

Good question. Sometimes I think Google (which owns Blogspot) is embarrassed by my blog.

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Paul, I think in the code in your blog you are missing some (corner) cases:
1. An IloRange may encode a quadratic expression, in which case you need to iterate over the quadratic part of the expression itself.
2. 'thing' may be an instance of IloNumVar. This can happen if you explicitly cplex.add(x) for an IloNumVar instance x.
Also I would recommend to throw an exception in case 'thing' has an unexpected type, just to be sure ...
In general I would also recommend to sort the resulting array (for example by variable name) so that each run of the code returns the variables in the same order. Sorting only by hash codes may not return the variables in the same order every time.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Daniel,

Thanks for the suggestions.

> dju358 wrote:
> 1. An IloRange may encode a quadratic expression, in which case you need to iterate over the quadratic part of the expression itself.

I've tested the code against several quadratic examples, including an example where one variable appears only in the quadratic part of the objective and another appears only in the quadratic part of a cone constraint. It seems to be working correctly, but if you come across a counterexample, please let me know.

> 2. 'thing' may be an instance of IloNumVar. This can happen if you explicitly cplex.add(x) for an IloNumVar instance x.

I've added coverage for that.

> Also I would recommend to throw an exception in case 'thing' has an unexpected type, just to be sure ...

Good idea. Done.

> In general I would also recommend to sort the resulting array (for example by variable name) so that each run of the code returns the variables in the same order. Sorting only by hash codes may not return the variables in the same order every time.

Hmm. I thought it was more deterministic than that (assuming of course that the model did not change between runs). On the other hand, sorting's not a problem; so I did as you suggested and sorted the vector alphabetically by variable name. The one small annoyance is that IloX11 comes between IloX1 and IloX2, etc. I'm way too lazy to screw with regex just to fix that. :-)

I also moved the code to a static method in a class of its own (with a customized exception class). The new code can be downloaded from Google Docs (in case anybody finds this thread).

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

> > 1. An IloRange may encode a quadratic expression, in which case you need to iterate over the quadratic part of the expression itself.
>
> I've tested the code against several quadratic examples, including an example where one variable appears only in the quadratic part of the objective and another appears only in the quadratic part of a cone constraint. It seems to be working correctly, but if you come across a counterexample, please let me know.
>
The code below does not find x and y unless you set CONSIDER_QUAD_EXPRS to true (and if it does find them with CONSIDER_QUAD_EXPR=false then I would consider that a bug).

import ilog.cplex.*; 

import ilog.concert.*;   

public 

final 

class ListQuadExpr 
{ 

private 

static 

final 

boolean CONSIDER_QUAD_EXPRS = 

false;   

private 

static IloNumVar[] parse(IloCplex cplex) 

throws IloException 
{ java.util.HashSet<IloNumVar> vars = 

new java.util.HashSet<IloNumVar>(); java.util.Iterator it = cplex.iterator(); IloLinearNumExpr expr; IloLinearNumExprIterator it2; 

while (it.hasNext()) 
{ IloAddable thing = (IloAddable) it.next(); 

if (thing 

instanceof IloRange) 
{ expr = (IloLinearNumExpr) ((IloRange) thing).getExpr(); it2 = expr.linearIterator(); 

while (it2.hasNext()) 
{ vars.add(it2.nextNumVar()); 
} 

if ( CONSIDER_QUAD_EXPRS ) 
{ IloQuadNumExpr qexpr = (IloQuadNumExpr) ((IloRange) thing).getExpr(); IloQuadNumExprIterator qit = qexpr.quadIterator(); 

while (qit.hasNext()) 
{ qit.next(); vars.add(qit.getNumVar1()); vars.add(qit.getNumVar2()); 
} 
} 
} 

else 

if (thing 

instanceof IloObjective) 
{ expr = (IloLinearNumExpr) ((IloObjective) thing).getExpr(); it2 = expr.linearIterator(); 

while (it2.hasNext()) 
{ vars.add(it2.nextNumVar()); 
} 
} 

else 

if (thing 

instanceof IloSOS1) 
{ vars.addAll(java.util.Arrays.asList(((IloSOS1) thing).getNumVars())); 
} 

else 

if (thing 

instanceof IloSOS2) 
{ vars.addAll(java.util.Arrays.asList(((IloSOS2) thing).getNumVars())); 
} 

else 

if (thing 

instanceof IloLPMatrix) 
{ vars.addAll(java.util.Arrays.asList(((IloLPMatrix) thing).getNumVars())); 
} 
} IloNumVar[] varray = vars.toArray(

new IloNumVar[1]); 

return varray; 
}   

public 

static 

void main(String[] args) 
{ 

try 
{ IloCplex cplex = 

new IloCplex(); IloNumVar x = cplex.numVar(0, 1, 
"x"); IloNumVar y = cplex.numVar(0, 1, 
"y"); cplex.addLe(cplex.prod(x, y), 1); System.out.println(
"Variables:"); 

for (IloNumVar v : parse(cplex)) System.out.println(
"\t" + v); 
} 

catch (IloException e) 
{ System.err.println(e.getMessage()); System.exit(-1); 
} 
} 
}

> > ...
> > In general I would also recommend to sort the resulting array (for example by variable name) so that each run of the code returns the variables in the same order. Sorting only by hash codes may not return the variables in the same order every time.
>
> Hmm. I thought it was more deterministic than that (assuming of course that the model did not change between runs).
>
I am not exactly sure here. I think unless you override the hash function in a subclass the hash value may depend on the memory address of the object and that may be non-deterministic? And therefore you may depend on correct overriding of this function. But that really goes beyond my Java expertise :-(

> On the other hand, sorting's not a problem; so I did as you suggested and sorted the vector alphabetically by variable name. The one small annoyance is that IloX11 comes between IloX1 and IloX2, etc. I'm way too lazy to screw with regex just to fix that. :-)
>
Question is: what will happen if the variables have no names?
In C++ I usually sort on the extractable id but no such thing exists in Java :-(
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

> dju358 wrote:

> The code below does not find x and y unless you set CONSIDER_QUAD_EXPRS to true (and if it does find them with CONSIDER_QUAD_EXPR=false then I would consider that a bug).

You guys are killing me! I was testing my code against models read in from LP or SAV files. I confirmed that your little model causes my code indigestion. On the other hand, if I export your model to an LP file and then import that into an IloCplex object, my code parses it correctly. Other than assigning names to the variables (which should not affect the parsing of the model), I don't know what export-import changes, but apparently it changes something. Sheesh!

Anyway, I updated my code, and it now tests correctly against your model as well as my previous test cases.

> > On the other hand, sorting's not a problem; so I did as you suggested and sorted the vector alphabetically by variable name. The one small annoyance is that IloX11 comes between IloX1 and IloX2, etc. I'm way too lazy to screw with regex just to fix that. :-)
> >
> Question is: what will happen if the variables have no names?
> In C++ I usually sort on the extractable id but no such thing exists in Java :-(

This was another non-problem with imported models (since unnamed variables are given IloX... names), but it's a problem with models built internally. I resolved it by assigning an arbitrary (and moderately ugly) name to each variable. The moderately ugly part both avoids conflicts with anything a reasonable user might have used elsewhere as a name and also perhaps encourages the user to assign more aesthetically pleasing names. :-)

While I was at it, I put a license (EPL) in my code in case anybody actually wants to use it, God help them.

Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

> Paul Rubin wrote:
> > dju358 wrote:
>
> > The code below does not find x and y unless you set CONSIDER_QUAD_EXPRS to true (and if it does find them with CONSIDER_QUAD_EXPR=false then I would consider that a bug).
>
> You guys are killing me! I was testing my code against models read in from LP or SAV files. I confirmed that your little model causes my code indigestion. On the other hand, if I export your model to an LP file and then import that into an IloCplex object, my code parses it correctly. Other than assigning names to the variables (which should not affect the parsing of the model), I don't know what export-import changes, but apparently it changes something. Sheesh!
>
I was not aware of that either but the explanation is simple: When CPLEX reads a problem file then it stores the linear constraints in an instance of IloLPMatrix. Even if there are no linear constraints, it still stores all the variables in this matrix (so you end up with a 0 by n matrix).
Your code handled IloLPMatrix instances correct, so after importing you correctly found all the variables.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Well, the explanation makes sense, although I would never have guessed it.

Thanks,
Paul

Mathematicians are like Frenchmen: whenever you say something to them, they translate it into their own language, and at once it is something entirely different. (Goethe)
#CPLEXOptimizers
#DecisionOptimization