Decision Optimization

Decision Optimization

Delivers prescriptive analytics capabilities and decision intelligence to improve decision-making.

 View Only
  • 1.  How to import data from csv file

    Posted Wed May 31, 2017 09:22 AM

    You should use IloOplInputFile:

    Suppose you have this export.csv file

    Nicolas;2;
    Alexander;3;

    Then you could write

     

    tuple t
    {
    string firstname;
    int number;
    }

    {t} s={};

    execute
    {
    var f=new IloOplInputFile("export.csv");
    while (!f.eof)
    {
    var str=f.readline();
    //writeln(str);
    var ar=str.split(";");
    if (ar.length==3) s.add(ar[0],Opl.intValue(ar[1]));
    }
    f.close();
    }

    execute
    {
    writeln(s);
    }

    which will read the csv file and compute the tuple set s:

    {<"Nicolas" 2> <"Alexander" 3>}

    regards

     

    Alex Fleischer

    PS:

    Many how to with OPL at https://www.linkedin.com/pulse/how-opl-alex-fleischer/

    Many examples from a very good book : https://www.linkedin.com/pulse/model-building-oplcplex-alex-fleischer/

    Making optimization simple : https://www.linkedin.com/pulse/making-decision-optimization-simple-alex-fleischer/


    #DecisionOptimization
    #OPLusingCPLEXOptimizer


  • 2.  Re: How to import data from csv file

    Posted Wed January 31, 2018 01:54 PM

    Originally posted by: Esteban.Alvarez


    Hello, thank you for the information, is very usefull, but I have a question.

    How do you do if you want to import a csv file with titles in the columns?

    For example:

    Name, Age, Country

    Jhon, 35, Canada

    Peter, 29, USA

    Alex, 33, England

    If I use your way to imput this information will get the names of the columns, but I don't need them. It's possible to remove those elements?

    I would appreciate your answer

    Kind regards.

     


    #DecisionOptimization
    #OPLusingCPLEXOptimizer


  • 3.  Re: How to import data from csv file

    Posted Thu February 01, 2018 12:16 AM

    Hi

    skipping a line will do the job.

    Let me call friends.csv your data

    Then

    tuple t
        {
        string firstname;
        int age;
        string country;
        }

        {t} s={};

        execute
        {
        var f=new IloOplInputFile("friends.csv");
        var str=f.readline(); // skip first line
        while (!f.eof)
        {
        var str=f.readline();
        //writeln(str);
        var ar=str.split(",");
        if (ar.length==3) s.add(ar[0],Opl.intValue(ar[1]),ar[2]);
        }
        f.close();
        }

        execute
        {
        writeln(s);
        }

    gives

    {<"Jhon" 35 " Canada"> <"Peter" 29 " USA"> <"Alex" 33 " England ">}

    regards


    #DecisionOptimization
    #OPLusingCPLEXOptimizer


  • 4.  Re: How to import data from csv file

    Posted Thu February 01, 2018 01:03 AM

    PS:

    You may also have a look at https://www.ibm.com/developerworks/community/forums/html/topic?id=476469ad-eec4-46f5-93a5-7b04e87b2836&ps=25

    and write

     

       

        execute
        {

        function readCSV(csvfilename,datfilename,tuplesetname)
        {
            var quote="\"";

            var python=new IloOplOutputFile("c:\\readCSV.py");
            python.writeln("from pandas import *");
            python.writeln("df=read_csv(",quote,csvfilename,quote,",sep=\',\')");
            python.writeln("res = open(",quote,datfilename,quote,",",quote,"w",quote,")");
            python.writeln("res.write(",quote,tuplesetname,"={",quote,")");
            python.writeln("res.write(",quote,"\\","n",quote,")");
            python.writeln("for i, row in enumerate(df.values):");
           
            python.writeln("   res.write(",quote,"<",quote,")");
            python.writeln("   for j in row:");
            python.writeln("      if (j==j):");
            python.writeln("         res.write(str(j))");
            python.writeln("         res.write(\",\")");
            python.writeln("   res.write(\">,\")    ");
            python.writeln("   res.write(",quote,"\\","n",quote,")");
            python.writeln("res.write(\"};\")");
            python.writeln("res.close()");
            python.close();
            
            IloOplExec("C:\\Users\\IBM_ADMIN\\AppData\\Local\\Programs\\Python\\Python36\\python.exe c:\\readCSV.py");
        }


        readCSV("c:\\\\friends.csv","c:\\\\friends.dat","s");

        }
      

    which will generate friends.dat

    s={
    <Jhon,35, Canada,>,
    <Peter,29, USA,>,
    <Alex,33, England ,>,
    };

    regards


    #DecisionOptimization
    #OPLusingCPLEXOptimizer


  • 5.  Re: How to import data from csv file

    Posted Thu February 01, 2018 08:48 AM

    Originally posted by: Esteban.Alvarez


    Thank you very much for your answer, I really appreciate it.

    I saw that you used "readline()" twice for delete the first line, so I have two questions about it.

     

    1) I'm trying to find a book or document for learn more about this kind of code but nothing explain things like the one you did. do you have any document or webpage that you could share for learn more about this?

     

    2)is it possible to choose a specific line in between for keep or delete the data?

    for example, if I wanted to make a variable only with Peter information

    Peter = (29 "USA")

     

    Thank you for your time

    Kind regards


    #DecisionOptimization
    #OPLusingCPLEXOptimizer


  • 6.  Re: How to import data from csv file

    Posted Thu February 01, 2018 12:58 PM

    Hi

    1) Many useful links at https://www.ibm.com/developerworks/community/forums/html/topic?id=1bf1f4de-b6fa-4375-a397-741e9239513e&ps=25

    and especially https://www.linkedin.com/pulse/how-opl-alex-fleischer/

    2) This is slicing and you can do that in OPL

    tuple t
        {
        string firstname;
        int age;
        string country;
        }

        {t} s={};

        execute
        {
        var f=new IloOplInputFile("friends.csv");
        var str=f.readline(); // skip first line
        while (!f.eof)
        {
        var str=f.readline();
        //writeln(str);
        var ar=str.split(",");
        if (ar.length==3) s.add(ar[0],Opl.intValue(ar[1]),ar[2]);
        }
        f.close();
        }

        execute
        {
        writeln(s);
        }
        
        {t} s2={i | i in s: i.firstname=="Peter"};
        
        execute
        {
        writeln(s2);
        }

    gives

     

    {<"Jhon" 35 " Canada"> <"Peter" 29 " USA"> <"Alex" 33 " England ">}
     {<"Peter" 29 " USA">}

    regards


    #DecisionOptimization
    #OPLusingCPLEXOptimizer


  • 7.  Re: How to import data from csv file

    Posted Mon February 05, 2018 12:12 PM

    Originally posted by: Esteban.Alvarez


    Thank you, I imported all the data the way I wanted, now I want to use the data but I have some problems.

    I invented a problem about chicken sales, the idea is to maximize the sales of male and female chickens.

    The table shows the consumption of vitamins and corn from male and female chickens and the availability of the corn and vitamins (those are the constraint that I will use) and the sale price of the male and female chickens.

     

    Using this table I convert this to csv. and I got that:

     

    With this I use the OPL for create variables (tuples and string) with the following code:

     

    // Data

    {string} Food;

     tuple consumption
     {
    int cons_male;
    int cons_female;
    }

     {consumption} Consumption={};
     
     tuple availability
    {
    int avail;
    }

    {availability} Availability={};
     
    tuple sale_price
    {
     int sale_price_male;
     int sale_price_female;
    }

    {sale_price} Sale_price={};

    execute
    {
    var f=new IloOplInputFile("sales.csv");
    var str=f.readline(); // skip first line 
    while (!f.eof)
    {
    var str=f.readline();
    //writeln(str);
    var ar=str.split(",");
    if (ar.length==4) Food.add(ar[0]);
    if (ar.length==4) Consumption.add(Opl.intValue(ar[1]),Opl.intValue(ar[2]));
    if (ar.length==4) Availability.add(Opl.intValue(ar[3]));
    if (ar.length==3) Sale_price.add(Opl.intValue(ar[1]),Opl.intValue(ar[2]));
    }
    f.close();
    }

    tuple sex
    {
    string male;
    string female;
    }

    {sex} Sex={};

    execute
    {
    var f=new IloOplInputFile("sales.csv");
    while (!f.eof)
    {
    var str=f.readline();
    //writeln(str);
    var ar=str.split(",");
    if (ar.length==4) Sex.add(ar[1],ar[2]);
    }
    f.close();
    }

    {sex} Sex_name={i | i in Sex: i.male==" male"};

    execute
        {
     writeln(Consumption);
     writeln(Sale_price);
     writeln(Availability);
     writeln(Sex);
     writeln(Sex_name); 
        }

    And I got:

    Now I want to use the data in the model, so I created decision variables and objetive function, but I got a error:

     

    (Is not possible load the model / it is not a type of matrix)

     

    This error said that there is a problem with the type, so I compared the information with an example from the program and I realised that there are some icons which describe the data type.


    My problem is that I cant change the data type and I don't know what means those icons.


    Would you help me to finish this exercise so I could learn how to modify those parameters?

    I attached the model

     

    I appreciate very much your answer
    Kind regards

     

     


    #DecisionOptimization


  • 8.  Re: How to import data from csv file

    Posted Mon February 05, 2018 12:42 PM

    Hi,

    I am not too sure about what you d like to achieve but as far as syntax is concerned

    dvar float+ production[Sex_name];

    // Objective Function
    maximize
    sum (s in Sex_name)
    sum(sp in Sale_price)  
    sp.sale_price_male*
    production[s];

    works

     

    And  by the way do not hesitate to open new threads since this has not much to do with cvs read

    regards


    #DecisionOptimization
    #OPLusingCPLEXOptimizer


  • 9.  Re: How to import data from csv file

    Posted Mon February 05, 2018 01:31 PM

    Originally posted by: Esteban.Alvarez


    The aim is to know how many "male" and "female" chickens I need for maximize the income, so the objetive function should be something like this:

    Maximize

    Production(male) * sale_price_male + Production(female) * sale_price_female

     

    I should get 2 results, one production for male and another production for female, but with the actual code (the one you just posted) I get only one result (you can see that is only a single result which is "0"). That's the problem that I'm trying to solve.

     

    Plus, the table in the data visualization shows (size 1):

     

    That means that it will give only one value for female and male, but I need 2 results, one for male and another for female, that's the problem that I'm trying to solve.

     

    I'm sorry for keep posting here, but I don't know if the way of import data from CSV is important for fix this issue. Next time I will open a new topic.

     

    Thank you very much.


    #DecisionOptimization


  • 10.  Re: How to import data from csv file

    Posted Mon February 05, 2018 02:07 PM

    Hi,

    then you could get some ideas from

    {string} Sex_names={first(Sex_name).male} union {first(Sex_name).female};

     

    dvar float+ production[Sex_names];

    // Objective Function
    maximize

    sum(sp in Sale_price)  
    (sp.sale_price_male
    *
    production[" male"]
    +
    sp.sale_price_female
    *
    production[" female"]
    )
    ;

    regards


    #DecisionOptimization
    #OPLusingCPLEXOptimizer