Decision Optimization

 View Only
Expand all | Collapse all

Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

  • 1.  Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Wed July 01, 2020 04:53 AM
    Hi,

    I've read this post: on exporting the model into a .lp fileI've also read on exporting file and running oplrun

    - I've always used the IDE to run my MIP model, but now i need to use to command prompt to run the files on cloud (high performance computing).
    steps i've tried:
    -When i open the command prompt and i type 'cplex', the interactive optimizer comes up the 'read' command  as stated in the first link does not read .mod/.dat/.ops files which I've used in the IDE.
    -'oplrun' does not exist in the directory which i type 'cplex' and the optimizer comes up, i read post from link 2, where you change directory 
    'cd C:\Program Files\IBM\ILOG\Cplex_Studio<version>\opl\bin\x64_win64>oplrun' then i see the same screen on the command prompt as link 2.
           - here i can run 'oplrun' but my project files are saved in another directory, how do i access those files so that i can run the command stated by Alex in post 1 to convert my .mod files into .lp?
            -Do i need/ how do i convert my .dat file to input the data?
           - How do i import the settings in the .ops file?

    1) All I want is to run my model like i could in the IDE through the command prompt, how can I do that?
    1a) I intend to use 128 cores to run the MIP to speed up the branch-and-bound nodes exploration.
    2) After running, how do i access the log and view the results(solutions) like the IDE?

    Thank you in advance.







    ------------------------------
    marvin cheung
    ------------------------------

    #DecisionOptimization


  • 2.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Wed July 01, 2020 05:49 AM
    Hi,

    have you had a look at 

    Command line call OPL CPLEX model in Watson Machine Learning





    ?

    regards

    ------------------------------
    ALEX FLEISCHER
    ------------------------------



  • 3.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Wed September 09, 2020 03:12 PM
    Thanks Alex, it didn't work when the files are saved on my desktop. When i copied the files to C:\ it threw up this exception

    *** FATAL[PARSE_103] at 1:1-2 C:\HKGori_1800_0000.xls: Unexpected character in "D", try using quoted string data, or check the encoding.
    ### OPL exception: Processing failed.

    C:\Program Files\IBM\ILOG\CPLEX_Studio1210\opl\bin\x64_win64>oplrun "C:\tests1s2perf.mod" "C:\perfdata1.dat" "C:\HKGori_1800_0000.xls" "C:\tests1s2settings.ops" <<< setup *** FATAL[PARSE_103] at 1:1-2 C:\HKGori_1800_0000.xls: Unexpected character in "D", try using quoted string data, or check the encoding. ### OPL exception: Processing failed.​
    do you have anyidea why?

    ------------------------------
    marvin cheung
    ------------------------------



  • 4.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Thu September 10, 2020 01:18 AM
    You cannot specify an xls file on the command line. xls files must be referenced from .dat files instead. Check the oil example in the distribution or check the documentation for detailed explanations of `SheetConnection`.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 5.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Thu September 10, 2020 01:59 AM
    Edited by System Fri January 20, 2023 04:50 PM
    Hi Daniel,

    • Thank you, i managed to run with only the .mod and .dat file.
    • However, it throws up error when i run the .ops together. In that case, how do i specify the settings that I have indicated in the .ops file? I have some time limit settings and solution polishing settings in the .ops file.
      • How do i set the number of threads if i want to run it on cloud?
    • And when it completes, how do i save the engine log +solution because i only see that it displays the objective and some summary statistics.
    Thank you for your patience, because when i run in the cplex studio it does all these through the .ops files...
    Thank you so much..

    ------------------------------
    marvin cheung
    ------------------------------



  • 6.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Thu September 10, 2020 02:11 AM
    If you want to use an OPS file you must create an OPL project (i.e. an XML project configuration file) and then use the option of oplrun that executes a project rather than a set of files (OPS files cannot be specified on the command line).

    Alternatively, you can set CPLEX parameters inside the .mod file using syntax
    execute { cplex.tilim = 20; }
    You could even try to do something like
    float TILIM = 0;
    execute { if ( TILIM != 0 )  cplex.tilim = TILIM; }
    and then invoke oplrun with argument -DTILIM=20 to specify a custom time limit for this particular run.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 7.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Thu September 10, 2020 02:20 AM
    Oh, ok, I will set all the settings in the .mod file and try.

    Thanks Daniel.

    ------------------------------
    marvin cheung
    ------------------------------



  • 8.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Thu September 17, 2020 08:40 AM
    Edited by System Fri January 20, 2023 04:31 PM
    Hi Daniel,

    I've managed to:
    • Run the .mod and .dat file through the command prompt using oplrun on my PC
    • Set all the settings of the .ops files through the .mod file using execute{cplex parameters}
      • MIP cuts settings
      • time limit settings
      • solution polishing
      • no. of threads
    • Save the engine log
    However, when I run it on cloud, it is in LINUX and cannot read the .xlsx in the .dat file. What are my options? 
    • My data come from different sheets in the .xlsx, i tried in .csv but .csv only can save a single sheet...
    Thank you in advance.

    ------------------------------
    marvin cheung
    ------------------------------



  • 9.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Thu September 17, 2020 09:37 AM
    One option would be to use multiple CSV files or concatenate the data from multiple CSVs into one.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 10.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Sun September 20, 2020 07:01 AM
    Hi Daniel,

    I followed the IloOplInputFile guide on stackoverflow https://stackoverflow.com/questions/56100539/read-csv-file-in-cplex?rq=1

    {string} arr = {}; execute{ var f = new IloOplInputFile("HKG1.csv"); while (!f.eof){ var data = f.readline(); var ar = data.split(","); arr.add(ar[5]); } writeln(arr); }​

    I always get a "null" after the last row, why is that so? How can I avoid this?

    {"HX706" "KA903" "UO754" "KA789" "KA809" "MU725" "PR312" "CX934" "UO731" "UO689" "BR827" "CI921" "MH78" "HX313" "SQ868" "MM67" "null"}

    For arrays with indexing, i did this:

    range winindex = 1..4; int cost[winindex]; execute{ var f = new IloOplInputFile("HKGcost.csv"); while (!f.eof){ for (var i in winindex){ var data = f.readline().split(","); cost[i] = Opl.intValue(data[0]); } } }

    but there is error saying that "Not an integer value "376". " The input .csv is just 4 rows of numbers

    in the .csv 376 376 376 376

     When i try just to see if i have indexed correctly with

    cost[i] = i+1 ;

    it works, so i think the for loop is ok, but not sure why it says 376 is not an integer value?

    Thank you..




    ------------------------------
    marvin cheung
    ------------------------------



  • 11.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Sun September 20, 2020 08:47 AM
    Hi,

    if you rewrite your csv into

    376
    376
    376
    376

    then your code works fine

    regards

    ------------------------------
    ALEX FLEISCHER
    ------------------------------



  • 12.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Sun September 20, 2020 09:04 AM
    Hi Alex,

    Yes that is how the csv is, i'm not sure why it all appears as such in the code block.

    Do you have any idea why there will be a null value at the end of the string indexing? The line after "MM67" is empty.


    ------------------------------
    marvin cheung
    ------------------------------



  • 13.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Sun September 20, 2020 09:58 AM
    I created a new .csv file from notepad and it works. I'm not sure why the .csv file saved from excel creates that error.
    I've attached both files, HKGcost.csv is from excel, while hkgslotcost.csv is saved from notedpad.

    Not sure why that happens, becasue the contents in the file are identical.

    ------------------------------
    marvin cheung
    ------------------------------



  • 14.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Thu September 24, 2020 06:41 AM
    Hi Alex and Daniel,

    I'm posting my approach that resolved the questions I've had to hopefully benefit other users.

    Initializing index strings (where strings in .csv is single column and multiple rows):
    {string} nameofstring = {}; execute{ var f = new IloOplInputFile("inputfile.csv"); while (!f.eof){ var data = f.readline(); var ar = data.split(","); nameofstring.add(ar[0]); } //print in scripting log to check writeln(nameofstring); }​

    To initialize variable with string index(1D/2D):

    {string} stringindex; int variable4[stringindex]; //1D case data in 8th column (ref data[7]) execute{ var f = new IloOplInputFile("inputfile3.csv"); while (!f.eof){ for (var i in stringindex){ var data = f.readline().split(","); variable4[i] = Opl.intValue(data[7]); } } } //2D case int variable2D[stringindex][stringindex]; execute{ var f = new IloOplInputFile("inputfile5.csv"); while (!f.eof){ for (var i in stringindex){ var indi = 0; var data = f.readline().split(","); for(var j in stringindex){ variable2D[i][j] = Opl.intValue(data[indi]); indi=indi+1; } } } }


    To initialize variable with range index (where data for variable1/2/3 in columns 1/2/3:

    range rangeindex = 1..5; int variable1[rangeindex]; int variable2[rangeindex]; int variable3[rangeindex]; execute{ var f = new IloOplInputFile("inputfile2.csv"); while (!f.eof){ for (var i in rangeindex){ var data = f.readline().split(","); variable1[i] = Opl.intValue(data[0]); variable2[i] = Opl.intValue(data[1]); variable3[i] = Opl.intValue(data[2]); } } }Output files post-run:
    execute{ var f = new IloOplOutputFile("outputfile.csv") for(var i in rangeindex){ f.writeln(variable1[i],",",variable2[i],",",variable3[i],","); } f.close(); //print some results in script writeln(anyDecisionVariablename); // print mip relative gap writeln(cplex.epgap) }​

    On the .csv file issue I've faced "Not an integer value "376"
    https://stackoverflow.com/questions/1552749/difference-between-cr-lf-lf-and-cr-line-break-types
    The .csv created has to be in CR LF format, otherwise Cplex does not recognize the values.

    On the last row {null}: Edit the .csv to remove the last line after the last row of data. When i created the .csv from excel, it has an additional empty line after the last row of data.



    ------------------------------
    marvin cheung
    ------------------------------



  • 15.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Wed July 01, 2020 09:01 AM
    Your .mod and .dat file can be anywhere on disk. Just specify the absolute path to them when passing them as arguments to oplrun.

    ------------------------------
    Daniel Junglas
    ------------------------------



  • 16.  RE: Running .mod+.dat +.ops in Interactive Optimizer(command prompt)

    Posted Wed September 09, 2020 03:13 PM
    Thanks Daniel,

    I tried when i saved the files on desktop it did not work, when i saved the files in C:\ i ran into some errors as described in my reply to Alex above.

    ------------------------------
    marvin cheung
    ------------------------------