Decision Optimization

Decision Optimization

Delivers prescriptive analytics capabilities and decision intelligence to improve decision-making.

 View Only
Expand all | Collapse all

Run CPLEX with mod and dat files in MATLAB

  • 1.  Run CPLEX with mod and dat files in MATLAB

    Posted Wed July 25, 2012 05:47 AM

    Originally posted by: cordobes


    Dear all,
    I have prepared *.mod and *.dat files for a MILP that run smooothly using CPLEX by command line. Now I would like to run the optimization from MATLAB. For using the MATLAB toolbox function "cplexmilp()" I only found examples in which the matrices / vectors have to be defined (e.g., x = cplexmilp(f,Aineq,bineq)). Is there a way to use cplexmilp() with .mod and .dat-files instead of explicitly putting in the matrices?

    Thanks a lot!

    cordobes
    #CPLEXOptimizers
    #DecisionOptimization


  • 2.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Thu July 26, 2012 10:54 AM

    Originally posted by: EdKlotz


    > cordobes wrote:
    > Dear all,
    >
    >
    > I have prepared *.mod and *.dat files for a MILP that run smooothly using CPLEX by command line. Now I would like to run the optimization from MATLAB. For using the MATLAB toolbox function "cplexmilp()" I only found examples in which the matrices / vectors have to be defined (e.g., x = cplexmilp(f,Aineq,bineq)). Is there a way to use cplexmilp() with .mod and .dat-files instead of explicitly putting in the matrices?
    >
    > Thanks a lot!
    >
    > cordobes
    I am not sure what you mean when you write 'CPLEX by command line.' Normally
    I would infer this to mean the interactive CPLEX optimizer, but it does not
    read in .mod and .dat files directly. .mod and .dat files are typically input
    files to specify the model to the OPL IDE that is part of CPLEX Optimization Studio. Are you referring to the OPL IDE when you say 'CPLEX by command line.'?

    Regardless, while I don't think you can directly input your current model format
    into the MATLAB toolbox functions, I think you can get the into MATLAB using
    the CPLEX Class API instead. Specifically, it sounds like you either already
    have an MPS, LP or SAV file of the model if you are using the interactive CPLEX
    optimizer, or you can generate one from the OPL IDE after providing your .mod
    and .dat files by telling OPL to export an MPS, LP or SAV file. Once you have
    that, you can use the readModel method of the CPLEX class in the MATLAB API to
    input the MPS, LP or SAV file.
    #CPLEXOptimizers
    #DecisionOptimization


  • 3.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Fri July 27, 2012 03:05 AM

    Originally posted by: cordobes


    Dear EdKlotz,

    thanks a lot for your help! You're right, by "command line" I meant the interactive optimizer, so it is very good to know that it doesn't read .mod or .dat files! Basically, the problem I'd like to solve is as follows:

    • I have a MILP formulated in OPL
    • The input data for the MILP is generated in a MATLAB model
    • I want to solve the MILP problem, then I want to fix all integers at the optimal solution of the MILP problem and solve the remaining LP (in order to get duals)
    • I want to have the optimal solutions and the duals read into MATLAB
    • Generally, I would like to be able to execute everything just by running a MATLAB script; thus, I could either use the CPLEX toolbox or call the interactive CPLEX optimizer or OPLRUN by command line
    • And.. I would like to be able to also set parameters, e.g., for probing.

    At the moment I see two alternatives:
    Alternative A)
    • Write my input data from MATLAB to a .dat-file
    • In the OPL model (*.mod), I could include scripting ("main"-command) and solve first the MILP, write all results to output files, then solve the fixed MILP, write optimal solution and duals to output files.
    • Read in the output files in MATLAB

    Alternative B) which I just figured by your help:
    • Export my OPL model to LP file
    • Use the CPLEX class in MATLAB: read in the model, use directly data existing in MATLAB, perform the two optimizations, and the optimal solutions and dual can be retrieved directly by in MATLAB
    So you would recommend alternative B? And can I do everything I mentioned above with the MATLAB toolbox? Then I will try this out, unfortunately, I can't access the IBM servers containing the references for CPLEX in MATLAB and CPLEX class API. It has been a couple of days and I hope I will be able to access them soon and to try out alternative B)!

    Thanks a lot and best regards,

    Sven
    Thanks a lot!
    #CPLEXOptimizers
    #DecisionOptimization


  • 4.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Sun July 29, 2012 08:19 PM

    Originally posted by: EdKlotz


    > cordobes wrote:
    > Dear EdKlotz,
    >
    > thanks a lot for your help! You're right, by "command line" I meant the interactive optimizer, so it is very good to know that it doesn't read .mod or .dat files! Basically, the problem I'd like to solve is as follows:
    >
    > - I have a MILP formulated in OPL
    > - The input data for the MILP is generated in a MATLAB model
    > - I want to solve the MILP problem, then I want to fix all integers at the optimal solution of the MILP problem and solve the remaining LP (in order to get duals)
    > - I want to have the optimal solutions and the duals read into MATLAB
    > - Generally, I would like to be able to execute everything just by running a MATLAB script; thus, I could either use the CPLEX toolbox or call the interactive CPLEX optimizer or OPLRUN by command line
    > - And.. I would like to be able to also set parameters, e.g., for probing.
    >
    > At the moment I see two alternatives:
    > Alternative A)
    > - Write my input data from MATLAB to a .dat-file
    > - In the OPL model (*.mod), I could include scripting ("main"-command) and solve first the MILP, write all results to output files, then solve the fixed MILP, write optimal solution and duals to output files.
    > - Read in the output files in MATLAB
    >
    > Alternative B) which I just figured by your help:
    > - Export my OPL model to LP file
    > - Use the CPLEX class in MATLAB: read in the model, use directly data existing in MATLAB, perform the two optimizations, and the optimal solutions and dual can be retrieved directly by in MATLAB
    >
    >
    > So you would recommend alternative B? And can I do everything I mentioned above with the MATLAB toolbox? Then I will try this out, unfortunately, I can't access the IBM servers containing the references for CPLEX in MATLAB and CPLEX class API. It has been a couple of days and I hope I will be able to access them soon and to try out alternative B)!
    >
    I agree with you that both A) and B) are viable options for you to consider.
    I think you could also consider making use of MATLAB's ability to call external
    programs. So, you could use MATLAB to generate the data for the mod file, then
    have it call a program written in C++ or Java that calls the OPL API to solve
    the model. This requires more programming, but it results in a more integrated
    application with much less passing of data through files.

    Overall, I think B) works a little better than A). However, A) has one advantage in that it involves writing the data to disk instead of the LP
    or MPS file. That LP or MPS file is much larger, so potentially the
    file I/O time is greater. Or, if your models can get really big, you have
    the potential to consume significant disk space. However, if the models
    are modest in size, this really should make much of a difference.

    In general, you want to try to keep the file I/O in an application to a minimum,
    which is why I mentioned a third option that potential has no file I/O at all.
    However, the importance of minimizing file I/O depends on what you are trying
    to do. If your work is primarily for research, proof of concept, or prototyping, it's not very important. But, if you plan to create an application that will be run in an industrial setting, then it becomes more
    significant.
    #CPLEXOptimizers
    #DecisionOptimization


  • 5.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Thu May 22, 2014 08:37 PM

    Originally posted by: KhalidAbdulla


    Hi EdKlotz,

    So it's been a few years since the conversation above. Is there now any better way of making use of *.mod files and running CPLEX from within MATLAB?

    Similar to the original poster, I have an existing model running using .mod, and .dat files with the CPLEX Optimisation Studio; but I would like to run multiple variations of the models from within MATLAB (because my data is more easily accessible via MATLAB, and it is the tool I am used to using for plotting and data manipulation).

    I can of course output an MPS file from the CPLEX optimisation studio; but this is then fixed to the particular instance (data) of the problem.

    I like the compact problem definition which is available in the *.mod file format, is there anyway to make use of that via MATLAB (I am happy to use the APL or the functions).

    One option would be to write a script which converts the *.mod problem definition (and corresponding data file) into a *.mps problem file but this seems like a lot of extra/repeated work. Can you think of a better way of doing this?

    I am happy to use APIs based in other languages called from MATLAB if that would give me more options.

    Thanks,

     

    - Khalid.


    #CPLEXOptimizers
    #DecisionOptimization


  • 6.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Thu May 22, 2014 08:57 PM

    Originally posted by: Laci Ladanyi


    It looks like you could use the OPTI toolbox. It seems to support reading a .mod / .dat pair directly from matlab and then call aml with cplex specified as solver. See: http://www.i2c2.aut.ac.nz/Wiki/OPTI/index.php/File/AMPL.

    --Laci

    Disclaimer: I have never used OPTI, just got curious after reading your question and found this link.


    #CPLEXOptimizers
    #DecisionOptimization


  • 7.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Thu May 22, 2014 10:51 PM

    Originally posted by: KhalidAbdulla


    Many thanks for the response, and the suggestion. However, I do not have an AMPL license. To date I have been using the CPLEX Optimisation Studio to create and run models (defined *.mod and *.dat files).

    To be able to use the OPTI toolbox file I would need to obtain an AMPL license that can to generate the *.nl file. The web-page linked above shows an 'example 5' where it generates MATLAB problem definition directly from *.mod and *.dat file, however I have downloaded and tried this and it is still looking for the AMPL executable to be able to generate the intermediate *.nl file. (Also I am not sure AMPL and OPL *.mod files are fully compatible).

    I still haven't found any existing routine which I can use which will 'unpack' the nice compact *.mod formulation (and corresponding *.dat file) into an *.mps problem definition.

    Does anyone else have any thoughts on best way to do this? I can do this from within the CPLEX optimization studio; but the reason for wanting to be able to do this from MATLAB is to be able to run multiple versions of models automatically.

    - Khalid.


    #CPLEXOptimizers
    #DecisionOptimization


  • 8.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Fri May 23, 2014 12:00 AM

    Originally posted by: Laci Ladanyi


    Oops, I'm sorry... I missed that you have OPL .mod files and not AMPL .mod files... In that case you might have better luck asking how to use OPL from matlab on the OPL forum: https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000002053. Though back in 2012 there was a question on the topic (https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014807443) and there was no reply to that... :-(. Good luck!


    #CPLEXOptimizers
    #DecisionOptimization


  • 9.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Fri May 23, 2014 04:00 AM

    There is still no direct connection between OPL and matlab and there is also no way to process .mod files in CPLEX directly (you need OPL for that).

    In order to create an MPS file from a .mod (and corresponding .dat) file you don't need a script. You can use the oplrun binary that is shipped with CPLEX Optimization studio. This is a command line tool that can load a .mod file and export the corresponding CPLEX model (see the -e option). I would recommend using a SAV file instead of an MPS file, though.

    If you are willing to call other code from matlab then it might be an option to implement something in Java. Take a look at the opl/examples/opl_interfaces/java/oplrunsample/ project in the installation. This is Java code that basically does the same as oplrun. So if you can somehow hook this up with matlab then you should be in business. I am not an expert in matlab so I don't know how to do this but it should be possible in one way or other. If Java does not work the same example is also available in C++.


    #CPLEXOptimizers
    #DecisionOptimization


  • 10.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Tue May 27, 2014 09:25 PM

    Originally posted by: KhalidAbdulla


    Daniel,

    Many thanks for the response. The oplrun binary was just what I needed (I was unaware of it because I had always been running my models from within the Optimisation Studio till now).

    In case others with similar requirements to me end up reading this post what I ended up doing was simply calling the oplrun binary as an operating system command from Matlab. Along the lines of:

    >> command = 'oplrun myModel.mod myData.dat';

    >> [status, cmdout] = system(command);

    It is fairly straight-forward to then use MATLAB to make modifications to (or new versions of) 'myData.dat' to run mutliple instances of a problem, carry out sensitivity studies etc. To get the results back from the model you can either get the model to directly print output files which can then be read into MATLAB or you could parse the 'cmdout' variable assigned above which will contain the solution values for the decision variables (assuming it solved).

    This may not be the most efficient/effective way of working but it is serving my purposes for now. If anyone has suggestions for a better way of doing this please post a reply.

    Many thanks for the help Daniel,

    - Khalid.


    #CPLEXOptimizers
    #DecisionOptimization


  • 11.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Thu May 12, 2016 03:37 AM


  • 12.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Thu May 12, 2016 02:59 PM

    Originally posted by: Samira Fazel


    Hi all,

    I am beginner in this stuff and I have similar problem now. I want to run my MILP written in OPL from MATLAB and use the data I generated in MATLAB to apply in my OPL model. Based on the explanations, I used two lines with my model name as below:

    command = 'oplrun wholeprocess.mod wholeprocess.dat';
    [status, cmdout] = system(command);  

    However, when I open the result it is saying file not find. Just wondering for more explanations.

    Thanks,

    Samira


    #CPLEXOptimizers
    #DecisionOptimization


  • 13.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Fri May 13, 2016 12:49 AM

    First thing I would try is to use absolute pathnames for all the items in the command.


    #CPLEXOptimizers
    #DecisionOptimization


  • 14.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Fri May 13, 2016 01:02 PM

    Originally posted by: Samira Fazel


    You are right. That works.

    Thanks. 


    #CPLEXOptimizers
    #DecisionOptimization


  • 15.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Thu November 10, 2016 10:01 PM

    Originally posted by: MdHabibUllah


    Hello Khalid

    Using this "oplrun" command I can execute the myModel.mod and myData.dat files but confuse about how to get the output decision variables in Matlab workspace. Could you please explain a bit if possible with an example?

     

    Thanks in advance

    Habib 

     


    #CPLEXOptimizers
    #DecisionOptimization


  • 16.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Tue November 15, 2016 09:18 PM

    Originally posted by: KhalidAbdulla


    Hi Habib,

    It's been a long-while since I was doing this. As I recall there are two options:

    1) Modify your model file so that you model saves its output, for example to a csv file, and then read this in from matlab

    2) Or more directly just parse the cmdout variable.

    If you have run a model as indicated above, just type >>cmdout

    at the matlab terminal, and you will see it contains all of the text output that would normally appear at the terminal when running oplrun (which includes the solution if one was found). If you are going with this approach you'll need to put some effort into parsing this output.

     

    Kind regards,

     

    - Khalid.


    #CPLEXOptimizers
    #DecisionOptimization


  • 17.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Sat April 16, 2016 11:14 PM

    Originally posted by: SashaMale


    Hi Dears

    How I can convert my .mod & .dot file to an .MPS file in the IDE and OPL environment?

    I didn't found any option in the IDE for doing this.

    Thank you so much.


    #CPLEXOptimizers
    #DecisionOptimization


  • 18.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Mon April 18, 2016 04:46 AM

    You have to use a settings file (.ops) and set the "export format" to MPS. Then running your model will create an .mps file in the project directory.


    #CPLEXOptimizers
    #DecisionOptimization


  • 19.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Sat April 16, 2016 11:20 PM

    Originally posted by: SashaMale


    Hi Dears

    How I can Use a .MPS File in Concert Technology (C++ API)? any references?

    Thanks in Advance


    #CPLEXOptimizers
    #DecisionOptimization


  • 20.  Re: Run CPLEX with mod and dat files in MATLAB

    Posted Mon April 18, 2016 04:47 AM