Decision Optimization

Decision Optimization

Delivers prescriptive analytics capabilities and decision intelligence to improve decision-making.


#Analytics
#DecisionOptimization
#DecisionOptimization
 View Only
  • 1.  Using the data file without loading it to the memory

    Posted 01/01/09 03:48 PM

    Originally posted by: SystemAdmin


    [shadi said:]

    Happy New Year for every one,

    i have a huge data file (> 1GB), this file includes set of tuples (millions of tuples), i create this file in the slave model and i want to choose some of these tuples in the master model.
    so in the master model i have to load this file in order to be able to use it, but when i load it i use a lot of memory and i get the "out of memory error".

    can i use this file without to load it to the memory????

    Kshieboun Shadi

    #ConstraintProgramming-General
    #DecisionOptimization


  • 2.  Re: Using the data file without loading it to the memory

    Posted 01/02/09 03:49 PM

    Originally posted by: SystemAdmin


    [jfk said:]

    Hello Shadi,
    well, it seems that  you are pushing the approach to use a huge number of columns at once. I don't know how you create your columns but probably a dynamic approach would help you. Here I outline what I mean:
    1. you create a small number of columns which definitely solve your problem (the initial number of columns) - that is cover all the necessary duties
    2. compute reduced costs and when you create a new column put a constraint saying that the reduced cost of the new column can't be worse than the computed reduced costs. Then the newly created column can be added to your master problem
    3/A. add the new column(s) to your master prb and solve it. Keep generating new columns 
    3/B. Keep track of the columns if they are chosen in your solution and when. If a column is not chosen for a solution for a long time probably you can delete it from the master prb.

    if you take a look at CPLEX's sifting basically it does the same except that the columns are already generated and the aim is to improve performance reducing the size of the problem (reducing the number of columns).

    I hope it helps

    cheers
    #ConstraintProgramming-General
    #DecisionOptimization


  • 3.  Re: Using the data file without loading it to the memory

    Posted 01/06/09 10:59 AM

    Originally posted by: SystemAdmin


    [shadi said:]

    Dear katai,
    can you please explain to me the second step!! what do you mean when you say "reduced costs"????

    do you mean the final cost (the master model output)??

    Shadi


    #ConstraintProgramming-General
    #DecisionOptimization