Hi,

before CPLEX 12.8, in order to run many models in parallel we needed to call OPL from an external language as can be seen in the example

CPLEX_Studio128\opl\examples\opl_interfaces\java\ConcurrentProcessing

But now with IloOplExec we can do that in an OPL main!

Let me give you an example:

Let us change slightly the example scalableWarehouse.mod|View Details

I changed the top

int n=...;
int nbthreads=...;
int Fixed        = 400+n;
int NbWarehouses = 400;
int NbStores     = 800+n;

so that the model needs some parameters.

Then

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=-1 c:\\scalableWarehouse.mod",true );
}
 
}

will execute 20 models one after the other whereas

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );
}
}

will run 20 models in parallel.

 

NB:

With my 8 cores machine, the first non parallel runs take 16 minutes whereas the parallel runs take 4 minutes!

Yes parallel can help.

 

Alex Fleischer

PS:

Many how to with OPL at https://www.linkedin.com/pulse/how-opl-alex-fleischer/

Many examples from a very good book : https://www.linkedin.com/pulse/model-building-oplcplex-alex-fleischer/

Making optimization simple : https://www.linkedin.com/pulse/making-decision-optimization-simple-alex-fleischer/

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: qtbgo

Dear Alex, I tried your method, but get some confusing result.

 

Here is my "scalableWarehouseParallel.mod":

int n = ...;
int nbthreads = ...;

int Fixed        = 400 + n;
int NbWarehouses = 400 ;
int NbStores     = 800 + n;

assert( NbStores > NbWarehouses );

range Warehouses = 1..NbWarehouses;
range Stores     = 1..NbStores;
int Capacity[w in Warehouses] = 
  NbStores div NbWarehouses + 
  w % ( NbStores div NbWarehouses );
int SupplyCost[s in Stores][w in Warehouses] = 
  1 + ( ( s + 10 * w ) % 100 );
dvar int Open[Warehouses] in 0..1;
dvar float Supply[Stores][Warehouses] in 0..1;
dexpr int TotalFixedCost = sum( w in Warehouses ) Fixed * Open[w];
dexpr float TotalSupplyCost = sum( w in Warehouses, s in Stores )  SupplyCost[s][w] * Supply[s][w];
minimize TotalFixedCost + TotalSupplyCost;

subject to {
  forall( s in Stores )
    ctStoreHasOneWarehouse: 
      sum( w in Warehouses ) 
        Supply[s][w] == 1;
  forall( w in Warehouses )
    ctOpen:
      sum( s in Stores ) 
        Supply[s][w] <= Open[w] * Capacity[w];
}

 

and here is my main flow "parallelMain.mod":

 

when I run parallelMain.mod, there is nothing in engine log, why? my  machine has 2 cores.

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: qtbgo

Dear, Alex,

 I cannot download your mod, why?

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi

so let me show you the content:

int n=...;
int nbthreads=...;
int Fixed        = 400+n;
int NbWarehouses = 400;
int NbStores     = 800+n;

assert( NbStores > NbWarehouses );

range Warehouses = 1..NbWarehouses;
range Stores     = 1..NbStores;
int Capacity[w in Warehouses] =
  NbStores div NbWarehouses +
  w % ( NbStores div NbWarehouses );
int SupplyCost[s in Stores][w in Warehouses] =
  1 + ( ( s + 10 * w ) % 100 );
dvar int Open[Warehouses] in 0..1;
dvar float Supply[Stores][Warehouses] in 0..1;
dexpr int TotalFixedCost = sum( w in Warehouses ) Fixed * Open[w];
dexpr float TotalSupplyCost = sum( w in Warehouses, s in Stores )  SupplyCost[s][w] * Supply[s][w];
minimize TotalFixedCost + TotalSupplyCost;

subject to {
  forall( s in Stores )
    ctStoreHasOneWarehouse:
      sum( w in Warehouses )
        Supply[s][w] == 1;
  forall( w in Warehouses )
    ctOpen:
      sum( s in Stores )
        Supply[s][w] <= Open[w] * Capacity[w];
}

execute
{
var o=new IloOplOutputFile("c:\\res"+n+".txt");
date=new Date();
o.writeln(date);
o.writeln(Open);
o.close();
}

regards

 

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: qtbgo

Thank you, Alex,

So, where do you put the following code? There is no "main" block, just "execute" block. So I guess it'not a main script.

 

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );
}
}

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi,

the code

range r=1..20;
execute
{
d=new Date();

writeln(d);

for(i in r)
{
writeln(i);

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+
" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );
}
}

would be in an additional .mod

which is the model you run to call scalableWarehouse.mod many times in parallel

regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: 88Simon88

Hi Alex,

What does this part mean? 

IloOplExec("C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "+" -Dn="+i+" -Dnbthreads=1 c:\\scalableWarehouse.mod",false );

What are the arguments of "IloOplExec"?

Best regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi,

you have some documentation at https://www.ibm.com/support/knowledgecenter/en/SSSA5P_12.9.0/ilog.odms.ide.help/refjsopl/html/OPL_functions.html#IloOplExec

IloOplExec

{string} IloOplExec(command)

Exec command

Parameters:

command - The command to be executed.

Returns:

status.

 

The string command is the line that will be run and the boolean parameter is about waiting or not for the end of that command

regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

Originally posted by: 88Simon88

Hi,

I have read that before and didn't understand. 

What are each element of the commnad?

"C:\\ILOG\\CPLEX_Studio128\\opl\\bin\\x64_win64\\oplrun.exe "

" -Dn="

" -Dnbthreads=1 c:\\scalableWarehouse.mod"

How should I customize it for my model? 

#DecisionOptimization
#OPLusingCPLEXOptimizer

Hi,

maybe you could start by having a look at https://www.ibm.com/developerworks/community/forums/html/topic?id=4bef7847-9ac0-4402-bd3d-74eba89a03f8&ps=25

and then have a look at oplrun documentation

oplrun is a command line to call OPL and "-D" is a way to give parameters to the .mod

See https://www.ibm.com/support/knowledgecenter/en/SSSA5P_12.9.0/ilog.odms.ide.help/OPL_Studio/refoplrun/topics/oplrun_TOP.html

regards

#DecisionOptimization
#OPLusingCPLEXOptimizer

PS:

In CPLEX 12.8 with IloOplExec had a setting where we could call a program without waiting, now in CPLEX 12.10 IloOplExec waits so let me share a new way to run many OPL jobs in //

Again let us take the same model scalablewarehouse.mod and have it in a temp directory

Then serial:

range r=1..20;
    execute
    {
    d=new Date();

    writeln(d);

    for(i in r)
    {
    writeln(i);

    IloOplExec("C:\\ILOG\\CPLEX_Studio1210\\opl\\bin\\x64_win64\\oplrun.exe "+
    " -Dn="+i+" -Dnbthreads=1 c:\\temp\\scalableWarehouse.mod",false );
    }
    
    
    var d2=new Date();
    writeln("total time : ",(d2-d)/1000);
    }

 

takes 489 s

whereas parallel

range r=1..20;
    execute
    {
    d=new Date();

    writeln(d);
    
    var f=new IloOplOutputFile("c:\\temp\\parallel.bat");
    
    for(i in r)
    {
      writeln(i);
      f.writeln("start C:\\ILOG\\CPLEX_Studio1210\\opl\\bin\\x64_win64\\oplrun.exe "+
    " -Dn="+i+" -Dnbthreads=1 c:\\temp\\scalableWarehouse.mod ");
    }   
    
    
    f.close();

    IloOplExec("c:\\temp\\parallel.bat");
    
    
    
    var d2=new Date();
    writeln("total time : ",(d2-d)/1000);
    
    }

takes 79 s

 

To run in // we generate a .bat file and then call that file

This works on windows but on linux one needs to generate equivalent shell script

regards

 

#DecisionOptimization
#OPLusingCPLEXOptimizer