Originally posted by: SystemAdmin

Hi guys,

I installed an IBM ILOG CPLEX Optimization Studio Preview Edition, which is still good for 83 more days. I am using concert technology for C++ to solve an optimization problem (referred to as a one-step opt. problem as it is re-optimized for 8640 times throughout the simulation). Each one-step optimization problem is of 394 variables and 197 constraints (IloRange). The code structure is as followings:

for(

int i = 0; i < 8640; i++)
{ IloEnv env; IloModel model(env, 
"resalloc");   IloIntVarArray n[10]; IloIntVarArray fl[10];   
//Initialization of 394 variables   
//Define the optimization objective IloExpr obj_expr(env); model.add(obj);   
//Define 197 constraints IloRangeArray range[10]; 

for(

int i = 0; i < APP_NUM; i++)
{ ... 

for(

int j = 0; j < application[i].hosting_server_num; j++)
{ ... 
} model.add(range[i]); 
} 
}   IloCplex cplex(env); cplex.extract(model);   

try
{ cplex.solve();  
//this is where the exception occurs ... 
}

catch (IloException& e) 
{ std::cerr << 
"IloException: " << e << std::endl; 
} 

catch (std::exception& e) 
{ std::cerr << 
"Standard exception: " << e.what() << std::endl; 
}   obj_expr.end(); env.end();

The error message reads "IBM ILOG CPLEX Optimization Studio Preview Edition good for 83 more days. The CPLEX Optimizers will solve problems up to 500 variables and 500 constraints."

I did put the statements:

obj_expr.end(); env.end();

at the end of each loop. Don't they clear all created instances of IloIntVarArray and IloRange? I was hoping that at the beginning of each loop, all modeling objects would be created all over again, so that the number of variables (resp. that of constraints) wouldn't accumulate. I am not sure if this exception was thrown because of the outer loop over i. Shall I clear all variables and constraints explicitly with the following statements?

for(

int i = 0; i < 10; i++)
{ n[i].clear(); fl[i].clear(); 
}

Will the above statements remove all created modeling objects so that within each loop, the one-step opt. problem would be totally independent and reconstructed?

Thanks in advance!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

P.S. The var objects are created using:

for(int i = 0; i < APP_NUM; i++){
   n[i] = IloIntVarArray(env, application[i].hosting_server_num);
   fl[i] = IloIntVarArray(env, application[i].hosting_server_num);
   for(int j = 0; j < application[i].hosting_server_num; j++){
      n[i][j] = IloIntVar(env, 1, 3, "number");
      fl[i][j] = IloIntVar(env, 1, 6, "freq_lvl");
   }
}

There are totally 394 IloIntVars created, and each variable comes with a constraint (upper/lower bounds). Are the lower and upper bounds of each variable counted as a constraint, too? If so, the total number of constraints would be 394 + 197 (IloRange objects specified in the first post) = 591, exceeding the limit 500. I am not sure if this is the crux of the prob tho. Thanks!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

It looks like something is wrong with the code you posted: At the point where you instantiate IloCplex neither model nor env are in the scope (they are defined in the loop).
Can you try to cplex.exportModel("model.lp") before you call cplex.solve()? Then look at the created model.lp file and check whether it contains unexpected variables/constraints.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Thanks for the reply, Daniel! I instantiated IloCplex inside the scope of both model and env. It was a typo in the first post. Sorry about that. I added the statement cplex.exportModel("model.lp") before calling cplex.solve(), but failed to find where the file is. Is it under the same directory of the VS project?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: T_O

It should be in the actual working path (wherever this is...).

Just provide the full path: "C:\\my\\path\\model.lp". Heed the 2 backslashes.

Best regards,
Thomas
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Thanks Thomas! It's odd tho, that an exception occurs before the statement "cplex.exportModel("C:\\model.lp")" being executed. I fixed the typo in the first post, the code structure is as follows:

for(

int i = 0; i < 8640; i++)
{ IloEnv env; IloModel model(env, 
"resalloc"); IloIntVarArray n[10]; IloIntVarArray fl[10]; 
//Initialization of 394 variables 
//Define the optimization objective IloExpr obj_expr(env); model.add(obj); 
//Define 197 constraints IloRangeArray range[10]; 

for(

int i = 0; i < APP_NUM; i++)
{ ... 

for(

int j = 0; j < application[i].hosting_server_num; j++)
{ ... 
} model.add(range[i]); 
}   IloCplex cplex(env); cplex.extract(model); 

try
{ cplex.solve();  
//this is where the exception occurs ... 
}

catch (IloException& e) 
{ std::cerr << 
"IloException: " << e << std::endl; 
} 

catch (std::exception& e) 
{ std::cerr << 
"Standard exception: " << e.what() << std::endl; 
} obj_expr.end(); env.end();   
}

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Hey Daniel, I put the statement

cplex.export("C:\\model.lp");

outside the 'try' brackets (see the following code), then there comes "Unhandled exception at 0x000007fefdbc9e5d in resautoscaling.exe: Microsoft C++ exception: IloCplex::Exception at memory location 0x0013ef30.."

for(int i = 0; i < 8640; i++){
    ....
  IloCplex cplex(env);
  cplex.extract(model);
 
  cplex.exportModel("C:\\model.lp");
 
  try{
    cplex.solve();
      ...
  }catch (IloException& e) {
     std::cerr << "IloException: " << e << std::endl;
  } catch (std::exception& e) {
     std::cerr << "Standard exception: " << e.what() << std::endl;
  } 
}

If I move cplex.export("C:\\model.lp") inside the try bracket right before cplex.solve(), then the original exception was thrown: "IBM ILOG CPLEX Optimization Studio Preview Edition good for 83 more days. The CPLEX Optimizers will solve problems up to 500 variables and 500 constraints."
Any idea where the prob is? thanks!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

It is expected that the original exception is still thrown.
The question is: does exportModel() succeed and actually create the file or does this method throw the "problem size limits exceeded" exception as well?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Hey Daniel, exportModel("C:\\model.lp") didn't succeed. There's no model.lp file created under C:\, and it throws the "problem size limits exceeded" exception when the statement is placed within try{}. This is the final step to finish the one-step opt. problem, but exceptions just keep coming up;( Hope u could kindly help me out with this problem. thanks!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

OK, there is one more thing you can try. Add this code right before cplex.solve():

IloXmlContext ctx(cplex.getEnv());
ctx.writeModel(cplex.getModel(), "C:\\model.xml");
ctx.end();

and include header ilconcert/iloxmlcontext.h in your source code.
This will write out the Concert model (not the model used by the engine) to file model.xml in XML format. You can either try to inspect that file and figure out if everything is expected or post the file here so that we can take a look at it with an unlimited version of CPLEX. From this we may be able to figure out where the undesired constraints and/or variables come from.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Thanks a lot, Daniel! I added the code you suggested before cplex.solve() and the model.xml file was generated. Then I inspected the xml file and the number of variables seems right: 394 (at least in the linearpart of the xml file). If x and y are two variables I defined, and in the objective function there are terms x*x, y*y, x*y (as is the case in the objective expression I defined). Are they (+x*x, y*y, x*y+) also counted as variables? I am not sure about the number of constraints because I don't know the exact meaning of different IDs in the xml file. The skeleton of the xml file is attached as follows (the original file is too long to be copied and pasted here):

*<?xml version="1.0"?>
-<Root>
-<Modeler version="1.0a" name="resalloc">
-<objective sense="Minimize" Id="1">
-<numScalProd Id="2">
-<linearpart>
-<numLinearVar coef="0.283801693019633">
-<intVar name="number" Id="3">
<intDomain upperbound="3" lowerbound="1"/>
</intVar>
</numLinearVar>

........
numLinearVar coef="-0.0000946005643398777">
-<intVar name="freq_lvl" Id="396">
<intDomain upperbound="6" lowerbound="1"/>
</intVar>
</numLinearVar>
</linearpart>
-<nonlinearpart>
-<numExpr coef="1.0">
-<numQuadExpr Id="397">
-<numQuadExprTerm coef="0.0000473002821699389">
-<var1> <intVar RefId="4"/>
</var1>
-<var2>
<intVar RefId="4"/>
</var2>
</numQuadExprTerm>
.........
-<numQuadExprTerm coef="0.0000473002821699389">
-<var1>
<intVar RefId="396"/>
</var1>
-<var2>
<intVar RefId="396"/>
</var2>
</numQuadExprTerm>
</numQuadExpr>
</numExpr>
</nonlinearpart>
<constantpart> 0 </constantpart>
</numScalProd>
</objective>
-<range name="range" Id="398" upperbound="8" lowerbound="7">
-<expr>
-<numLinTerm Id="399">
-<linearpart>
-<numLinearVar coef="6">
<intVar RefId="3"/>
</numLinearVar>
-<numLinearVar coef="1">
<intVar RefId="4"/>
</numLinearVar>
</linearpart>
<constantpart> 0 </constantpart>
</numLinTerm>
</expr>
</range>
.............
-<range name="range" Id="790" upperbound="8" lowerbound="7">
-<expr>
-<numLinTerm Id="791">
-<linearpart>
-<numLinearVar coef="6">
<intVar RefId="395"/>
</numLinearVar>
-<numLinearVar coef="1">
<intVar RefId="396"/>
</numLinearVar>
</linearpart>
<constantpart> 0 </constantpart>
</numLinTerm>
</expr>
</range>
</Modeler>
</Root>*

I am lost now. How many variables and constraints are there actually? The range ID spans from 398 to 790 with a step of 2, i.e., 197 constraints, which is the number of constraints i thought there would be. However, the numLinTerm ID spans from 399 to 791 with the same step of 2, that makes another 197 'I-dunno-what-it-is'(constraint?). And what about the RefId?

This xml file does provide some info about the model i created tho. Hope the information is enough for u. I can post the whole xml file here if it's necessary for u to read more info from it.

Zillions of thanks!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

If the XML file is too large to copy and paste, can you just attach it? That would be the easiest way for us to look at your model. Or do you prefer not to reveal the details about your model?
In general the different things in the XML file mean:

• <boolVar .. Id="...">, <intVar ... Id="...">, <numVar ... Id="..."> all define a variable of the respective type. Each variable is assigned a unique id.
• <range ... Id="..."> defines a ranged constraint. Each range is assigned a unique id.
• <intVar RefId="..."> is a reference to the variable with the specified id.

To count the number of variables and constraints you just need to count the number of the first two items.
However, extraction of a model into the engine may create new variables/constraints, depending on the type of your constraints. It may be that this is the problem you are facing here. Would be easy to tell if we could look at the complete XML file.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Hey Daniel, thanks for the detailed explanation. I think I calculated the number of variables and constraints right in the first place, both of which are within 500. I applied for the IBM Academic Initiative program and downloaded a Cplex full version. It works fine now. I do have another question tho. Do I need to explicitly erase all variables/ranges I created within each loop to avoid memory leakage? The code structure is as follows:

for(

int i = 0; i < 8640; i++)
{ IloEnv env; IloModel model(env, 
"resalloc"); IloIntVarArray n[10]; IloIntVarArray fl[10]; 
//Initialization of 394 variables 
//Define the optimization objective IloExpr obj_expr(env); model.add(obj); 
//Define 197 constraints IloRangeArray range[10]; 

for(

int i = 0; i < APP_NUM; i++)
{ ... 

for(

int j = 0; j < application[i].hosting_server_num; j++)
{ ... 
} model.add(range[i]); 
}   IloCplex cplex(env); cplex.extract(model); 

try
{ cplex.solve();  
//this is where the exception occurs ... 
}

catch (IloException& e) 
{ std::cerr << 
"IloException: " << e << std::endl; 
} 

catch (std::exception& e) 
{ std::cerr << 
"Standard exception: " << e.what() << std::endl; 
} obj_expr.end(); env.end(); 
}

I called obj_expr.end() and env.end() at the end of each loop over i. Will that release the memory allocated to all modeling objects within each loop? Or should I explicitly use the following statements to do it, so that at the next loop, all modeling objects will be created from scratch?

for(

int i = 0; i < 8640; i++)
{ IloEnv env; IloModel model(env, 
"resalloc"); .... IloIntVarArray n[10]; IloIntVarArray fl[10]; IloExpr obj_expr(env); IloRangeArray range[10]; .... 
//cplex.solve();   

for(

int i = 0; i < 10; i++)
{ n[i].endElements(); n[i].end(); fl[i].endElements(); fl[i].end(); range[i].endElements(); range[i].end(); 
} obj_expr.end(); env.end(); 
}

looking forward to your reply;)
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

The statement env.end() will release everything that was allocated for this environment. In your case it will delete all modeling objects, so you don't need to delete them explicitly. Even obj_expr.end() is not required.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

By the way, if you are still curious: Now that you have an unlimited license you could try the cplex.exportModel("C:\\model.lp") again and look at the generated file to figure out how many variables/constraints it actually contains.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

Thanks for your kind help all these days, Daniel! It's the first time I've used CPLEX to solve optimization problems, quite a struggle, but you've made it so much easier;) I exported the model to a .lp file, and you were right about it, there were additional constraints added. That's why the problem size limits were exceeded.

All the best!
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

another weird thing is When I used the following code for IloRange creation:

for(int i = 0; i < 10; i++){
   range[i] = IloRangeArray(env, application[i].hosting_server_num);
                                
   for(int j = 0; j < application[i].hosting_server_num; j++){
     IloInt lower_bound =  Max(1, (application[i].cpu[j].number - 1)*6 + application[i].cpu[j].freq_level - 1); //in debugging mode, it takes the value of 1 when i = 0, j = 0;
     IloInt upper_bound =  Min(18, (application[i].cpu[j].number - 1)*6 + application[i].cpu[j].freq_level + 1);//in debugging mode, it takes the value of 2 when i = 0, j = 0;
     range[i][j] = IloRange(env, lower_bound, 6*(n[i][j]-1) + fl[i][j], upper_bound, "range");
   }
   model.add(range[i]);
}

, for i = 0, j = 0, lower_bound and upper_bound are shown to be 1 and 2 in debugging mode, respectively, which is correct being what I expected, whereas they are changed to 7, 8 respectively in the xml file. I don't understand why the values of the lower and upper bounds are changed? Is this because the range expression is equivalent to 6*n[i][j] + fl[i][j] - 6, and the -6 was added to the lower and upper bounds? I think this would be the case.

Looking forward to your reply.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I think you are right and the range is "normalized" before output to not contain any constants in the expression.
#CPLEXOptimizers
#DecisionOptimization