OPL - Reading in problem parameters from multiple files in a loop

6. RE: OPL - Reading in problem parameters from multiple files in a loop

Like

CPLEX User

Posted Tue July 13, 2021 01:04 PM

Thanks for the code, Christiane. This helps me understand the OPL way of doing things.

------------------------------
CPLEX User
------------------------------

Original Message

Original Message:
Sent: Tue July 13, 2021 12:53 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You should use ".dat" files which must follow the right format.

For example, here is what can be declared in .mod file:

int k = ...;float d = ...;{int} is = ...;string s = ...;{string} ss = ...;int a[1..2] = ...;int a2[1..2][1..3] = ...;

and the corresponding values in the .dat file:

k = 2701;d = 0.5;is = { 37, 73 };s = "FOO";ss = { "foo", "bar" };a = [ 37, 73 ];a2 = [ [ 11, 12, 13], [21, 22, 23] ];

I modified your files to make them run and attached them to this message.

I hope this helps,

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 02:29 PM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Thanks Christiane, Alex:

I am running into errors on trying to implement the ideas presented in this thread.

My workspace directory/project has the following file (all of which are attached with this message as well):

opl.mod

int NORUNS = 2;//Number of LPs to solve
int NOVARS = 3;//Number of variables in each LP
int NOCONS = 2;//Number of constraints in each LP
string IFILEPREFIX = "input";//Prefix of input file to read
string OFILEPREFIX = "output";//Prefix of output files to generate

main{
var modelname="iterate.mod";
var source = new IloOplModelSource(modelname);
var def = new IloOplModelDefinition(source);
var number_of_problems = NORUNS;
for(var probno = 1; probno <= number_of_problems; probno++){
var cplex = new IloCplex();
var theModel = new IloOplModel(def, cplex);
var ifile = IFILEPREFIX + probno + ".txt";
//How exactly and where to read input file ?
var data = new IloOplDataSource(ifile);
theModel.addDataSource(data);
theModel.generate();

var result;
var time0 = new Date();
if(cplex.solve()){
result = cplex.getObjValue();
var time1 = new Date();
var ofilename = OFILEPREFIX + probno + ".txt";
var ofile = new IloOplOutputFile(ofilename);
ofile.open(ofilename);
ofile.writeln("Objective is ", result);
ofile.writeln("solve time : ", (time1-time0)/1000);
ofile.close();
}
else {
writeln("Input " + ifile + " : Fail to solve.");
}
theModel.end();
cplex.end();
data.end();
}
def.end();
source.end();
};

iterate.mod:

int NOVARS = ...; // does this not work as extern int NOVARS; would in C/C++?
int NOCONS = ...; //How can I pass information/data from opl.mod to iterate.mod?
int c[1..NOVARS] = ...;
int A[1..NOCONS][1..NOVARS] = ...;
int b[1..NOCONS] = ...;

dvar float+ X[j in 1..NOVARS];

maximize sum(j in 1..NOVARS) c[j]*X[j];

subject to{
A[1][1] * X[1] + A[1][2] * X[2] + A[1][3] * X[3] <= b[1];
A[2][1] * X[1] + A[2][2] * X[2] + A[2][3] * X[3] <= b[2];
};

input1.txt

2 4 7 //Maximize 2 x1 + 4 x2 + 7 x3
1 3 2 4 s.t. x1 + 3 x2 + 2 x3 <= 4
2 3 3 2 2 x1 + 3 x2 + 3 x3 <= 2

input2.txt

7 1 4 //Maximize 7 x1 + x2 + 4 x3
-3 1 4 9 s.t. -3 x1 + x2 + 4 x3 <= 9
2 4 2 6 2 x1 + 4 x2 + 2 x3 <= 6

When I try to debug / run this, it fails with error message that `NORUNS` is an unknown variable. I come from a C/C++ workflow and hence OPL is new to me. So, I am struggling with some things that work in C++ such as a global variable which is what I hoped everything defined prior to `main` would be, but apparently not. Also, after defining `ifile` in `opl.mod`, how exactly is this data passed onto the model residing in `iterate.mod`? In other words, how should I actually read the text input files and which data structures should I populate (and how should this be done) before calling `cplex.solve()`?

Thank you.

------------------------------
CPLEX User

Original Message:
Sent: Mon July 12, 2021 01:01 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You can find different sample of flow control in scripting part in the OPL distribution here or here.
Below is also a little sample where you can see how to generate file names, read and write files, retrieve different solving info and results.
I hope this helps,

main {  var modelFile = "sched_jobshop.mod";  var source = new IloOplModelSource(modelFile);  var def = new IloOplModelDefinition(source);  var number_of_problems = 4;  for(var probno = 1; probno <= number_of_problems; probno++) {  var cp = new IloCP();  var theModel= new IloOplModel(def, cp);    //var ifile = "input" + probno + ".txt";    var ifile = "sched_jobshop_dat" + probno + ".dat";    //Read input file    //Create optimization model based on inputfile    var data = new IloOplDataSource(ifile);    theModel.addDataSource(data);    theModel.generate();    var result;    var time0 = new Date();    if (cp.solve()) {      result = cp.getObjValue();      var time1 = new Date();      //Output file      var ofileName = "output" + probno + ".txt";      var ofile = new IloOplOutputFile(ofileName);      if (!ofile.isOpen)         ofile.open(ofileName);            //Save output such as objective function value      ofile.writeln("Objective is ", result);            //time, number of nodes explored, etc      ofile.writeln("solve time : ", (time1-time0)/1000);      var nodes = cp.info.NumberOfChoicePoints;      ofile.writeln("Number Of Choice-Points : ", nodes);      ofile.writeln("====== All CP Info : ");      for (var i in cp.info) {        ofile.writeln(i + ": " + cp.info[i]);      }      ofile.close();      writeln("Input " + ifile + " gave output " + ofileName);    } else {      writeln("Input " + ifile + " : Fail to solve.");    }    theModel.end();    cp.end();    data.end();  }  def.end();  source.end();}

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 07:53 AM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

I have multiple input files named "input1.txt", "input2.txt",...,"input10.txt". I have to solve a separate optimization problem for each of these input files and save the results in "output1.txt", "output2.txt",...,"output10.txt" respectively.

In C/C++, I would roughly do the following.

int number_of_problems = 10;
#include <string>
#include <iostream>
int main(){
     for(int probno = 1; probno <= number_of_problems; probno++){
         std::string ifile = "input"+std::to_string(probno)+".txt";
         //Read input file
         //Using functions such as fscanf() etc.
         //Create optimization model based on inputfile
         std::string ofile = "output"+std::to_string(probno)+".txt";
         //Output file
         //Save output such as objective function value
         //time, number of nodes explored, etc
         //via fprintf(), etc.
         std::cout<<"Input "<<ifile<<" gave output "<<ofile<<std::endl;
     }
}

The output of the above code can be seen here.

What is the equivalent way in OPL IDE to obtain this functionality? That is, based on the loop counter value, how does one go about concatenating them into string/character array variables and then passing these variable to appropriate OPL functions?

Are there any examples that ship with CPLEX that demonstrate this functionality?

Thank you.

------------------------------
CPLEX User
------------------------------
#DecisionOptimization

7. RE: OPL - Reading in problem parameters from multiple files in a loop

Like

CPLEX User

Posted Sat July 17, 2021 10:46 PM

Christiane,

I came across this pdf presentation by CPLEX folks back in 2013.
On page 47, the authors state that:

the end() method is disabled by default in CPLEX Studio IDE, to enable it use:
thisOplModel.settings.mainEndEnabled = true;

I was curious about this since in the code examples you have provided in this thread, this line does not feature. I wonder if between 2013 and now, due to changes in OPL IDE, there is no longer any need for setting to true mainEndEnabled ? It suffices to call .end() on appropriate objects that have been `new`ed into existence?

Thank you.

------------------------------
CPLEX User
------------------------------

Original Message

Original Message:
Sent: Tue July 13, 2021 12:53 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You should use ".dat" files which must follow the right format.

For example, here is what can be declared in .mod file:

int k = ...;float d = ...;{int} is = ...;string s = ...;{string} ss = ...;int a[1..2] = ...;int a2[1..2][1..3] = ...;

and the corresponding values in the .dat file:

k = 2701;d = 0.5;is = { 37, 73 };s = "FOO";ss = { "foo", "bar" };a = [ 37, 73 ];a2 = [ [ 11, 12, 13], [21, 22, 23] ];

I modified your files to make them run and attached them to this message.

I hope this helps,

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 02:29 PM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Thanks Christiane, Alex:

I am running into errors on trying to implement the ideas presented in this thread.

My workspace directory/project has the following file (all of which are attached with this message as well):

opl.mod

int NORUNS = 2;//Number of LPs to solve
int NOVARS = 3;//Number of variables in each LP
int NOCONS = 2;//Number of constraints in each LP
string IFILEPREFIX = "input";//Prefix of input file to read
string OFILEPREFIX = "output";//Prefix of output files to generate

main{
var modelname="iterate.mod";
var source = new IloOplModelSource(modelname);
var def = new IloOplModelDefinition(source);
var number_of_problems = NORUNS;
for(var probno = 1; probno <= number_of_problems; probno++){
var cplex = new IloCplex();
var theModel = new IloOplModel(def, cplex);
var ifile = IFILEPREFIX + probno + ".txt";
//How exactly and where to read input file ?
var data = new IloOplDataSource(ifile);
theModel.addDataSource(data);
theModel.generate();

var result;
var time0 = new Date();
if(cplex.solve()){
result = cplex.getObjValue();
var time1 = new Date();
var ofilename = OFILEPREFIX + probno + ".txt";
var ofile = new IloOplOutputFile(ofilename);
ofile.open(ofilename);
ofile.writeln("Objective is ", result);
ofile.writeln("solve time : ", (time1-time0)/1000);
ofile.close();
}
else {
writeln("Input " + ifile + " : Fail to solve.");
}
theModel.end();
cplex.end();
data.end();
}
def.end();
source.end();
};

iterate.mod:

int NOVARS = ...; // does this not work as extern int NOVARS; would in C/C++?
int NOCONS = ...; //How can I pass information/data from opl.mod to iterate.mod?
int c[1..NOVARS] = ...;
int A[1..NOCONS][1..NOVARS] = ...;
int b[1..NOCONS] = ...;

dvar float+ X[j in 1..NOVARS];

maximize sum(j in 1..NOVARS) c[j]*X[j];

subject to{
A[1][1] * X[1] + A[1][2] * X[2] + A[1][3] * X[3] <= b[1];
A[2][1] * X[1] + A[2][2] * X[2] + A[2][3] * X[3] <= b[2];
};

input1.txt

2 4 7 //Maximize 2 x1 + 4 x2 + 7 x3
1 3 2 4 s.t. x1 + 3 x2 + 2 x3 <= 4
2 3 3 2 2 x1 + 3 x2 + 3 x3 <= 2

input2.txt

7 1 4 //Maximize 7 x1 + x2 + 4 x3
-3 1 4 9 s.t. -3 x1 + x2 + 4 x3 <= 9
2 4 2 6 2 x1 + 4 x2 + 2 x3 <= 6

When I try to debug / run this, it fails with error message that `NORUNS` is an unknown variable. I come from a C/C++ workflow and hence OPL is new to me. So, I am struggling with some things that work in C++ such as a global variable which is what I hoped everything defined prior to `main` would be, but apparently not. Also, after defining `ifile` in `opl.mod`, how exactly is this data passed onto the model residing in `iterate.mod`? In other words, how should I actually read the text input files and which data structures should I populate (and how should this be done) before calling `cplex.solve()`?

Thank you.

------------------------------
CPLEX User

Original Message:
Sent: Mon July 12, 2021 01:01 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You can find different sample of flow control in scripting part in the OPL distribution here or here.
Below is also a little sample where you can see how to generate file names, read and write files, retrieve different solving info and results.
I hope this helps,

main {  var modelFile = "sched_jobshop.mod";  var source = new IloOplModelSource(modelFile);  var def = new IloOplModelDefinition(source);  var number_of_problems = 4;  for(var probno = 1; probno <= number_of_problems; probno++) {  var cp = new IloCP();  var theModel= new IloOplModel(def, cp);    //var ifile = "input" + probno + ".txt";    var ifile = "sched_jobshop_dat" + probno + ".dat";    //Read input file    //Create optimization model based on inputfile    var data = new IloOplDataSource(ifile);    theModel.addDataSource(data);    theModel.generate();    var result;    var time0 = new Date();    if (cp.solve()) {      result = cp.getObjValue();      var time1 = new Date();      //Output file      var ofileName = "output" + probno + ".txt";      var ofile = new IloOplOutputFile(ofileName);      if (!ofile.isOpen)         ofile.open(ofileName);            //Save output such as objective function value      ofile.writeln("Objective is ", result);            //time, number of nodes explored, etc      ofile.writeln("solve time : ", (time1-time0)/1000);      var nodes = cp.info.NumberOfChoicePoints;      ofile.writeln("Number Of Choice-Points : ", nodes);      ofile.writeln("====== All CP Info : ");      for (var i in cp.info) {        ofile.writeln(i + ": " + cp.info[i]);      }      ofile.close();      writeln("Input " + ifile + " gave output " + ofileName);    } else {      writeln("Input " + ifile + " : Fail to solve.");    }    theModel.end();    cp.end();    data.end();  }  def.end();  source.end();}

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 07:53 AM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

I have multiple input files named "input1.txt", "input2.txt",...,"input10.txt". I have to solve a separate optimization problem for each of these input files and save the results in "output1.txt", "output2.txt",...,"output10.txt" respectively.

In C/C++, I would roughly do the following.

int number_of_problems = 10;
#include <string>
#include <iostream>
int main(){
     for(int probno = 1; probno <= number_of_problems; probno++){
         std::string ifile = "input"+std::to_string(probno)+".txt";
         //Read input file
         //Using functions such as fscanf() etc.
         //Create optimization model based on inputfile
         std::string ofile = "output"+std::to_string(probno)+".txt";
         //Output file
         //Save output such as objective function value
         //time, number of nodes explored, etc
         //via fprintf(), etc.
         std::cout<<"Input "<<ifile<<" gave output "<<ofile<<std::endl;
     }
}

The output of the above code can be seen here.

What is the equivalent way in OPL IDE to obtain this functionality? That is, based on the loop counter value, how does one go about concatenating them into string/character array variables and then passing these variable to appropriate OPL functions?

Are there any examples that ship with CPLEX that demonstrate this functionality?

Thank you.

------------------------------
CPLEX User
------------------------------
#DecisionOptimization

8. RE: OPL - Reading in problem parameters from multiple files in a loop

Like

ALEX FLEISCHER

Posted Sun July 18, 2021 05:43 AM

Hi

you can use

var m1=thisOplModel._memoryUsage;

to measure memory use

and in CPLEX 20.1 documentation you can read

The end method is set to true by default in the CPLEX Studio IDE. This is handled by the script method mainEndEnabled.

------------------------------
[Alex] [Fleischer]
[EMEA CPLEX Optimization Technical Sales]
[IBM]
------------------------------

Original Message

Original Message:
Sent: Sat July 17, 2021 10:45 PM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Christiane,

I came across this pdf presentation by CPLEX folks back in 2013.
On page 47, the authors state that:

the end() method is disabled by default in CPLEX Studio IDE, to enable it use:
thisOplModel.settings.mainEndEnabled = true;

I was curious about this since in the code examples you have provided in this thread, this line does not feature. I wonder if between 2013 and now, due to changes in OPL IDE, there is no longer any need for setting to true mainEndEnabled ? It suffices to call .end() on appropriate objects that have been `new`ed into existence?

Thank you.

------------------------------
CPLEX User

Original Message:
Sent: Tue July 13, 2021 12:53 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You should use ".dat" files which must follow the right format.

For example, here is what can be declared in .mod file:

int k = ...;float d = ...;{int} is = ...;string s = ...;{string} ss = ...;int a[1..2] = ...;int a2[1..2][1..3] = ...;

and the corresponding values in the .dat file:

k = 2701;d = 0.5;is = { 37, 73 };s = "FOO";ss = { "foo", "bar" };a = [ 37, 73 ];a2 = [ [ 11, 12, 13], [21, 22, 23] ];

I modified your files to make them run and attached them to this message.

I hope this helps,

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 02:29 PM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Thanks Christiane, Alex:

I am running into errors on trying to implement the ideas presented in this thread.

My workspace directory/project has the following file (all of which are attached with this message as well):

opl.mod

int NORUNS = 2;//Number of LPs to solve
int NOVARS = 3;//Number of variables in each LP
int NOCONS = 2;//Number of constraints in each LP
string IFILEPREFIX = "input";//Prefix of input file to read
string OFILEPREFIX = "output";//Prefix of output files to generate

main{
var modelname="iterate.mod";
var source = new IloOplModelSource(modelname);
var def = new IloOplModelDefinition(source);
var number_of_problems = NORUNS;
for(var probno = 1; probno <= number_of_problems; probno++){
var cplex = new IloCplex();
var theModel = new IloOplModel(def, cplex);
var ifile = IFILEPREFIX + probno + ".txt";
//How exactly and where to read input file ?
var data = new IloOplDataSource(ifile);
theModel.addDataSource(data);
theModel.generate();

var result;
var time0 = new Date();
if(cplex.solve()){
result = cplex.getObjValue();
var time1 = new Date();
var ofilename = OFILEPREFIX + probno + ".txt";
var ofile = new IloOplOutputFile(ofilename);
ofile.open(ofilename);
ofile.writeln("Objective is ", result);
ofile.writeln("solve time : ", (time1-time0)/1000);
ofile.close();
}
else {
writeln("Input " + ifile + " : Fail to solve.");
}
theModel.end();
cplex.end();
data.end();
}
def.end();
source.end();
};

iterate.mod:

int NOVARS = ...; // does this not work as extern int NOVARS; would in C/C++?
int NOCONS = ...; //How can I pass information/data from opl.mod to iterate.mod?
int c[1..NOVARS] = ...;
int A[1..NOCONS][1..NOVARS] = ...;
int b[1..NOCONS] = ...;

dvar float+ X[j in 1..NOVARS];

maximize sum(j in 1..NOVARS) c[j]*X[j];

subject to{
A[1][1] * X[1] + A[1][2] * X[2] + A[1][3] * X[3] <= b[1];
A[2][1] * X[1] + A[2][2] * X[2] + A[2][3] * X[3] <= b[2];
};

input1.txt

2 4 7 //Maximize 2 x1 + 4 x2 + 7 x3
1 3 2 4 s.t. x1 + 3 x2 + 2 x3 <= 4
2 3 3 2 2 x1 + 3 x2 + 3 x3 <= 2

input2.txt

7 1 4 //Maximize 7 x1 + x2 + 4 x3
-3 1 4 9 s.t. -3 x1 + x2 + 4 x3 <= 9
2 4 2 6 2 x1 + 4 x2 + 2 x3 <= 6

When I try to debug / run this, it fails with error message that `NORUNS` is an unknown variable. I come from a C/C++ workflow and hence OPL is new to me. So, I am struggling with some things that work in C++ such as a global variable which is what I hoped everything defined prior to `main` would be, but apparently not. Also, after defining `ifile` in `opl.mod`, how exactly is this data passed onto the model residing in `iterate.mod`? In other words, how should I actually read the text input files and which data structures should I populate (and how should this be done) before calling `cplex.solve()`?

Thank you.

------------------------------
CPLEX User

Original Message:
Sent: Mon July 12, 2021 01:01 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You can find different sample of flow control in scripting part in the OPL distribution here or here.
Below is also a little sample where you can see how to generate file names, read and write files, retrieve different solving info and results.
I hope this helps,

main {  var modelFile = "sched_jobshop.mod";  var source = new IloOplModelSource(modelFile);  var def = new IloOplModelDefinition(source);  var number_of_problems = 4;  for(var probno = 1; probno <= number_of_problems; probno++) {  var cp = new IloCP();  var theModel= new IloOplModel(def, cp);    //var ifile = "input" + probno + ".txt";    var ifile = "sched_jobshop_dat" + probno + ".dat";    //Read input file    //Create optimization model based on inputfile    var data = new IloOplDataSource(ifile);    theModel.addDataSource(data);    theModel.generate();    var result;    var time0 = new Date();    if (cp.solve()) {      result = cp.getObjValue();      var time1 = new Date();      //Output file      var ofileName = "output" + probno + ".txt";      var ofile = new IloOplOutputFile(ofileName);      if (!ofile.isOpen)         ofile.open(ofileName);            //Save output such as objective function value      ofile.writeln("Objective is ", result);            //time, number of nodes explored, etc      ofile.writeln("solve time : ", (time1-time0)/1000);      var nodes = cp.info.NumberOfChoicePoints;      ofile.writeln("Number Of Choice-Points : ", nodes);      ofile.writeln("====== All CP Info : ");      for (var i in cp.info) {        ofile.writeln(i + ": " + cp.info[i]);      }      ofile.close();      writeln("Input " + ifile + " gave output " + ofileName);    } else {      writeln("Input " + ifile + " : Fail to solve.");    }    theModel.end();    cp.end();    data.end();  }  def.end();  source.end();}

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 07:53 AM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

I have multiple input files named "input1.txt", "input2.txt",...,"input10.txt". I have to solve a separate optimization problem for each of these input files and save the results in "output1.txt", "output2.txt",...,"output10.txt" respectively.

In C/C++, I would roughly do the following.

int number_of_problems = 10;
#include <string>
#include <iostream>
int main(){
     for(int probno = 1; probno <= number_of_problems; probno++){
         std::string ifile = "input"+std::to_string(probno)+".txt";
         //Read input file
         //Using functions such as fscanf() etc.
         //Create optimization model based on inputfile
         std::string ofile = "output"+std::to_string(probno)+".txt";
         //Output file
         //Save output such as objective function value
         //time, number of nodes explored, etc
         //via fprintf(), etc.
         std::cout<<"Input "<<ifile<<" gave output "<<ofile<<std::endl;
     }
}

The output of the above code can be seen here.

What is the equivalent way in OPL IDE to obtain this functionality? That is, based on the loop counter value, how does one go about concatenating them into string/character array variables and then passing these variable to appropriate OPL functions?

Are there any examples that ship with CPLEX that demonstrate this functionality?

Thank you.

------------------------------
CPLEX User
------------------------------
#DecisionOptimization

9. RE: OPL - Reading in problem parameters from multiple files in a loop

Like

CPLEX User

Posted Sun July 18, 2021 07:19 AM

Alex,

Thanks. I am still on CPLEX 12.8. I will include the line at the top of my script:

thisOplModel.settings.mainEndEnabled = true;

just to be sure in any case.

------------------------------
CPLEX User
------------------------------

Original Message

Original Message:
Sent: Sun July 18, 2021 05:43 AM
From: ALEX FLEISCHER
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hi

you can use

var m1=thisOplModel._memoryUsage;

to measure memory use

and in CPLEX 20.1 documentation you can read

The end method is set to true by default in the CPLEX Studio IDE. This is handled by the script method mainEndEnabled.

------------------------------
[Alex] [Fleischer]
[EMEA CPLEX Optimization Technical Sales]
[IBM]

Original Message:
Sent: Sat July 17, 2021 10:45 PM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Christiane,

I came across this pdf presentation by CPLEX folks back in 2013.
On page 47, the authors state that:

the end() method is disabled by default in CPLEX Studio IDE, to enable it use:
thisOplModel.settings.mainEndEnabled = true;

I was curious about this since in the code examples you have provided in this thread, this line does not feature. I wonder if between 2013 and now, due to changes in OPL IDE, there is no longer any need for setting to true mainEndEnabled ? It suffices to call .end() on appropriate objects that have been `new`ed into existence?

Thank you.

------------------------------
CPLEX User

Original Message:
Sent: Tue July 13, 2021 12:53 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You should use ".dat" files which must follow the right format.

For example, here is what can be declared in .mod file:

int k = ...;float d = ...;{int} is = ...;string s = ...;{string} ss = ...;int a[1..2] = ...;int a2[1..2][1..3] = ...;

and the corresponding values in the .dat file:

k = 2701;d = 0.5;is = { 37, 73 };s = "FOO";ss = { "foo", "bar" };a = [ 37, 73 ];a2 = [ [ 11, 12, 13], [21, 22, 23] ];

I modified your files to make them run and attached them to this message.

I hope this helps,

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 02:29 PM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Thanks Christiane, Alex:

I am running into errors on trying to implement the ideas presented in this thread.

My workspace directory/project has the following file (all of which are attached with this message as well):

opl.mod

int NORUNS = 2;//Number of LPs to solve
int NOVARS = 3;//Number of variables in each LP
int NOCONS = 2;//Number of constraints in each LP
string IFILEPREFIX = "input";//Prefix of input file to read
string OFILEPREFIX = "output";//Prefix of output files to generate

main{
var modelname="iterate.mod";
var source = new IloOplModelSource(modelname);
var def = new IloOplModelDefinition(source);
var number_of_problems = NORUNS;
for(var probno = 1; probno <= number_of_problems; probno++){
var cplex = new IloCplex();
var theModel = new IloOplModel(def, cplex);
var ifile = IFILEPREFIX + probno + ".txt";
//How exactly and where to read input file ?
var data = new IloOplDataSource(ifile);
theModel.addDataSource(data);
theModel.generate();

var result;
var time0 = new Date();
if(cplex.solve()){
result = cplex.getObjValue();
var time1 = new Date();
var ofilename = OFILEPREFIX + probno + ".txt";
var ofile = new IloOplOutputFile(ofilename);
ofile.open(ofilename);
ofile.writeln("Objective is ", result);
ofile.writeln("solve time : ", (time1-time0)/1000);
ofile.close();
}
else {
writeln("Input " + ifile + " : Fail to solve.");
}
theModel.end();
cplex.end();
data.end();
}
def.end();
source.end();
};

iterate.mod:

int NOVARS = ...; // does this not work as extern int NOVARS; would in C/C++?
int NOCONS = ...; //How can I pass information/data from opl.mod to iterate.mod?
int c[1..NOVARS] = ...;
int A[1..NOCONS][1..NOVARS] = ...;
int b[1..NOCONS] = ...;

dvar float+ X[j in 1..NOVARS];

maximize sum(j in 1..NOVARS) c[j]*X[j];

subject to{
A[1][1] * X[1] + A[1][2] * X[2] + A[1][3] * X[3] <= b[1];
A[2][1] * X[1] + A[2][2] * X[2] + A[2][3] * X[3] <= b[2];
};

input1.txt

2 4 7 //Maximize 2 x1 + 4 x2 + 7 x3
1 3 2 4 s.t. x1 + 3 x2 + 2 x3 <= 4
2 3 3 2 2 x1 + 3 x2 + 3 x3 <= 2

input2.txt

7 1 4 //Maximize 7 x1 + x2 + 4 x3
-3 1 4 9 s.t. -3 x1 + x2 + 4 x3 <= 9
2 4 2 6 2 x1 + 4 x2 + 2 x3 <= 6

When I try to debug / run this, it fails with error message that `NORUNS` is an unknown variable. I come from a C/C++ workflow and hence OPL is new to me. So, I am struggling with some things that work in C++ such as a global variable which is what I hoped everything defined prior to `main` would be, but apparently not. Also, after defining `ifile` in `opl.mod`, how exactly is this data passed onto the model residing in `iterate.mod`? In other words, how should I actually read the text input files and which data structures should I populate (and how should this be done) before calling `cplex.solve()`?

Thank you.

------------------------------
CPLEX User

Original Message:
Sent: Mon July 12, 2021 01:01 PM
From: Christiane Bracchi
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

You can find different sample of flow control in scripting part in the OPL distribution here or here.
Below is also a little sample where you can see how to generate file names, read and write files, retrieve different solving info and results.
I hope this helps,

main {  var modelFile = "sched_jobshop.mod";  var source = new IloOplModelSource(modelFile);  var def = new IloOplModelDefinition(source);  var number_of_problems = 4;  for(var probno = 1; probno <= number_of_problems; probno++) {  var cp = new IloCP();  var theModel= new IloOplModel(def, cp);    //var ifile = "input" + probno + ".txt";    var ifile = "sched_jobshop_dat" + probno + ".dat";    //Read input file    //Create optimization model based on inputfile    var data = new IloOplDataSource(ifile);    theModel.addDataSource(data);    theModel.generate();    var result;    var time0 = new Date();    if (cp.solve()) {      result = cp.getObjValue();      var time1 = new Date();      //Output file      var ofileName = "output" + probno + ".txt";      var ofile = new IloOplOutputFile(ofileName);      if (!ofile.isOpen)         ofile.open(ofileName);            //Save output such as objective function value      ofile.writeln("Objective is ", result);            //time, number of nodes explored, etc      ofile.writeln("solve time : ", (time1-time0)/1000);      var nodes = cp.info.NumberOfChoicePoints;      ofile.writeln("Number Of Choice-Points : ", nodes);      ofile.writeln("====== All CP Info : ");      for (var i in cp.info) {        ofile.writeln(i + ": " + cp.info[i]);      }      ofile.close();      writeln("Input " + ifile + " gave output " + ofileName);    } else {      writeln("Input " + ifile + " : Fail to solve.");    }    theModel.end();    cp.end();    data.end();  }  def.end();  source.end();}

------------------------------
Christiane Bracchi

Original Message:
Sent: Mon July 12, 2021 07:53 AM
From: CPLEX User
Subject: OPL - Reading in problem parameters from multiple files in a loop

Hello,

I have multiple input files named "input1.txt", "input2.txt",...,"input10.txt". I have to solve a separate optimization problem for each of these input files and save the results in "output1.txt", "output2.txt",...,"output10.txt" respectively.

In C/C++, I would roughly do the following.

int number_of_problems = 10;
#include <string>
#include <iostream>
int main(){
     for(int probno = 1; probno <= number_of_problems; probno++){
         std::string ifile = "input"+std::to_string(probno)+".txt";
         //Read input file
         //Using functions such as fscanf() etc.
         //Create optimization model based on inputfile
         std::string ofile = "output"+std::to_string(probno)+".txt";
         //Output file
         //Save output such as objective function value
         //time, number of nodes explored, etc
         //via fprintf(), etc.
         std::cout<<"Input "<<ifile<<" gave output "<<ofile<<std::endl;
     }
}

The output of the above code can be seen here.

What is the equivalent way in OPL IDE to obtain this functionality? That is, based on the loop counter value, how does one go about concatenating them into string/character array variables and then passing these variable to appropriate OPL functions?

Are there any examples that ship with CPLEX that demonstrate this functionality?

Thank you.

------------------------------
CPLEX User
------------------------------
#DecisionOptimization

Decision Optimization

Decision Optimization