Decision Optimization

Hi,

before CPLEX 12.8, in order to run many models in parallel we needed to call OPL from an external language as can be seen in the example

CPLEX_Studio128\opl\examples\opl_interfaces\java\ConcurrentProcessing

But now with IloOplExec we can do that in an OPL main!

Let me give you an example:

Let us change slightly the example scalableWarehouse.mod|View Details

I changed the top

int n=...;
int nbthreads=...;
int Fixed        = 400+n;
int NbWarehouses = 400;
int NbStores     = 800+n;

so that the model needs some parameters.

Then

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=-1 c:\\scalableWarehouse.mod",true );
}
 
}

will execute 20 models one after the other whereas

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );
}
}

will run 20 models in parallel.

 

NB:

With my 8 cores machine, the first non parallel runs take 16 minutes whereas the parallel runs take 4 minutes!

Yes parallel can help.

 

Alex Fleischer

PS:

Many how to with OPL at https://www.linkedin.com/pulse/how-opl-alex-fleischer/

Many examples from a very good book : https://www.linkedin.com/pulse/model-building-oplcplex-alex-fleischer/

Making optimization simple : https://www.linkedin.com/pulse/making-decision-optimization-simple-alex-fleischer/

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: qtbgo

Dear Alex, I tried your method, but get some confusing result.

 

Here is my "scalableWarehouseParallel.mod":

int n = ...;
int nbthreads = ...;

int Fixed        = 400 + n;
int NbWarehouses = 400 ;
int NbStores     = 800 + n;

assert( NbStores > NbWarehouses );

range Warehouses = 1..NbWarehouses;
range Stores     = 1..NbStores;
int Capacity[w in Warehouses] = 
  NbStores div NbWarehouses + 
  w % ( NbStores div NbWarehouses );
int SupplyCost[s in Stores][w in Warehouses] = 
  1 + ( ( s + 10 * w ) % 100 );
dvar int Open[Warehouses] in 0..1;
dvar float Supply[Stores][Warehouses] in 0..1;
dexpr int TotalFixedCost = sum( w in Warehouses ) Fixed * Open[w];
dexpr float TotalSupplyCost = sum( w in Warehouses, s in Stores )  SupplyCost[s][w] * Supply[s][w];
minimize TotalFixedCost + TotalSupplyCost;

subject to {
  forall( s in Stores )
    ctStoreHasOneWarehouse: 
      sum( w in Warehouses ) 
        Supply[s][w] == 1;
  forall( w in Warehouses )
    ctOpen:
      sum( s in Stores ) 
        Supply[s][w] <= Open[w] * Capacity[w];
}

 

and here is my main flow "parallelMain.mod":

 

when I run parallelMain.mod, there is nothing in engine log, why? my  machine has 2 cores.

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: qtbgo

Dear, Alex,

 I cannot download your mod, why?

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi

so let me show you the content:

int n=...;
int nbthreads=...;
int Fixed        = 400+n;
int NbWarehouses = 400;
int NbStores     = 800+n;

assert( NbStores > NbWarehouses );

range Warehouses = 1..NbWarehouses;
range Stores     = 1..NbStores;
int Capacity[w in Warehouses] =
  NbStores div NbWarehouses +
  w % ( NbStores div NbWarehouses );
int SupplyCost[s in Stores][w in Warehouses] =
  1 + ( ( s + 10 * w ) % 100 );
dvar int Open[Warehouses] in 0..1;
dvar float Supply[Stores][Warehouses] in 0..1;
dexpr int TotalFixedCost = sum( w in Warehouses ) Fixed * Open[w];
dexpr float TotalSupplyCost = sum( w in Warehouses, s in Stores )  SupplyCost[s][w] * Supply[s][w];
minimize TotalFixedCost + TotalSupplyCost;

subject to {
  forall( s in Stores )
    ctStoreHasOneWarehouse:
      sum( w in Warehouses )
        Supply[s][w] == 1;
  forall( w in Warehouses )
    ctOpen:
      sum( s in Stores )
        Supply[s][w] <= Open[w] * Capacity[w];
}

execute
{
var o=new IloOplOutputFile("c:\\res"+n+".txt");
date=new Date();
o.writeln(date);
o.writeln(Open);
o.close();
}

regards

 

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: qtbgo

Thank you, Alex,

So, where do you put the following code? There is no "main" block, just "execute" block. So I guess it'not a main script.

 

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );
}
}

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi,

the code

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );
}
}

would be in an additional .mod

which is the model you run to call scalableWarehouse.mod many times in parallel

regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: 88Simon88

Hi Alex,

What does this part mean? 

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );

What are the arguments of "IloOplExec"?

Best regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi,

you have some documentation at https://www.ibm.com/support/knowledgecenter/en/SSSA5P_12.9.0/ilog.odms.ide.help/refjsopl/html/OPL_functions.html#IloOplExec

IloOplExec

{string} IloOplExec(command)

Exec command

Parameters:

command - The command to be executed.

Returns:

status.

 

The string command is the line that will be run and the boolean parameter is about waiting or not for the end of that command

regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: 88Simon88

Hi,

I have read that before and didn't understand. 

What are each element of the commnad?

"C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "

" -Dn="

" -Dnbthreads=1 c:\\scalableWarehouse.mod"

How should I customize it for my model? 

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi,

maybe you could start by having a look at https://www.ibm.com/developerworks/community/forums/html/topic?id=4bef7847-9ac0-4402-bd3d-74eba89a03f8&ps=25

and then have a look at oplrun documentation

oplrun is a command line to call OPL and "-D" is a way to give parameters to the .mod

See https://www.ibm.com/support/knowledgecenter/en/SSSA5P_12.9.0/ilog.odms.ide.help/OPL_Studio/refoplrun/topics/oplrun_TOP.html

regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

PS:

In CPLEX 12.8 with IloOplExec had a setting where we could call a program without waiting, now in CPLEX 12.10 IloOplExec waits so let me share a new way to run many OPL jobs in //

Again let us take the same model scalablewarehouse.mod and have it in a temp directory

Then serial:

range r=1..20;
    execute
    {
    d=new Date();

    writeln(d);

    for(i in r)
    {
    writeln(i);

    IloOplExec("C:\\ILOG\\CPLEX_Studio1210\\opl\\bin\\x64_win64\\oplrun.exe "+
    " -Dn="+i+" -Dnbthreads=1 c:\\temp\\scalableWarehouse.mod",false );
    }
    
    
    var d2=new Date();
    writeln("total time : ",(d2-d)/1000);
    }

 

takes 489 s

whereas parallel

range r=1..20;
    execute
    {
    d=new Date();

    writeln(d);
    
    var f=new IloOplOutputFile("c:\\temp\\parallel.bat");
    
    for(i in r)
    {
      writeln(i);
      f.writeln("start C:\\ILOG\\CPLEX_Studio1210\\opl\\bin\\x64_win64\\oplrun.exe "+
    " -Dn="+i+" -Dnbthreads=1 c:\\temp\\scalableWarehouse.mod ");
    }   
    
    
    f.close();

    IloOplExec("c:\\temp\\parallel.bat");
    
    
    
    var d2=new Date();
    writeln("total time : ",(d2-d)/1000);
    
    }

takes 79 s

 

To run in // we generate a .bat file and then call that file

This works on windows but on linux one needs to generate equivalent shell script

regards

 

#DecisionOptimization
#OPLusingCPLEXOptimizer