Decision Optimization

Decision Optimization

Delivers prescriptive analytics capabilities and decision intelligence to improve decision-making.

 View Only
Expand all | Collapse all

CPXlpopt runs slower and slower when called mutiple times

  • 1.  CPXlpopt runs slower and slower when called mutiple times

    Posted Fri September 21, 2012 09:18 AM

    Originally posted by: zhixin


    I my C code, I call CPXlpopt multiple times to solve different Quadratic programs, and find that the function takes longer and longer time to run. Then I tested the following simple code, and find the problem still happens:

    long tt=clock();
    for(long ii=0;ii<10000000;ii++)
    {
    CPXlpopt (env, lp);
    if(ii%100==99)
    {
    printf("%d\n",clock()-tt);
    tt=clock();
    }
    }

    The CPU time used for every 100 runs of CPXlpopt increases fast as ii becomes larger. Can anybody help me solve this problem? Is there something specific I need to set in CPXsetintparam? In fact, I tried several different parameters in CPXsetintparam, but the problem still occurs.

    Thank you for the help.
    #CPLEXOptimizers
    #DecisionOptimization


  • 2.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Mon September 24, 2012 12:21 PM

    Originally posted by: SystemAdmin


    I cannot reproduce that here. What machine do you use? What version of CPLEX? Could you attach the model you are solving? Could you also attach the output you see, i.e., the running times of the loops?
    #CPLEXOptimizers
    #DecisionOptimization


  • 3.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Mon September 24, 2012 12:37 PM

    Originally posted by: zhixin


    I use Microsoft Visual C++ Express 2010 with Concert Technology, with 64-bit compiler. The operating system is Windows 7 Enterprise. The code I have is very long, so it is hard to post here. But I suspect the problem occurs for a general optimization problem.

    For the test code I posted, the outcome is
    168
    126
    120
    114
    135
    205
    145
    164
    251
    176
    209
    225
    238
    216
    241
    266
    263
    257
    270
    263
    369
    348
    304
    310
    322
    313
    348
    345
    388
    370
    416
    429
    433
    421
    436
    442
    433
    497
    475
    462
    525
    521
    505
    555
    631
    543
    628
    560
    559
    584
    596
    589
    599
    675
    668
    675
    669
    678
    659
    672
    768
    745
    738
    749
    736
    739
    780
    807
    769
    863
    787
    918
    845
    838
    863
    873
    866
    877
    ...

    So even though not increasing consistently, overall it takes longer time for the CPXlpopt to solve the same problem.

    To test the problem, we can arbitrarily specify env and lp in CPXlpopt (env, lp).

    Thank you for the help.
    #CPLEXOptimizers
    #DecisionOptimization


  • 4.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Mon September 24, 2012 03:52 PM

    Originally posted by: SystemAdmin


    OK, the time looks to increase but I am wondering whether this may just be error in time measurement. What is CLOCKS_PER_SEC on your system?
    Instead of
    printf("%d\n",clock()-tt);
    

    you should do
    printf("%f\n",((double)clock()-tt) / CLOCKS_PER_SEC);
    

    what does this print?
    As I said, I tried to reproduce the described behavior with an arbitrary LP and did not succeed. Maybe you can do a CPXwriteprob(env,lp,"model.sav",0) before the loop and attach the model hre? Then I could try with the same model like you.
    #CPLEXOptimizers
    #DecisionOptimization


  • 5.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Tue September 25, 2012 09:58 AM

    Originally posted by: zhixin


    I tried a general 100*1000 linear program, and still have a similar problem. The code and output is as follows:

    void example()
    {
    CPXENVptr env = NULL;
    CPXLPptr lp = NULL;
    int status = 0;
    int numcols=100;//number of variables
    int numrows=100;//number of constraints
    int objsen = CPX_MIN; /* The problem is minimization */
    double* obj = new doublenumcols;//coefficients of the objective function
    double* rhs = new doublenumrows;//right handside values
    char* sense = new charnumrows;// sign for constraints
    int* matbeg=new intnumcols;//The nonzero elements of every column must be stored in sequential locations with matbeg[j] containing the index of the beginning of column j starting from 0
    int* matcnt=new intnumcols;//matcnt[j] containing the number of entries in column j
    int* matind;//For each k, matind[k] specifies the row number of the corresponding coefficient, matval[k]
    double* matval;//coefficient
    double* lb=new doublenumcols;//lower bound
    double* ub=new doublenumcols;//upper bound
    long i,j;

    //the linear objecitive coefficient
    for(i=0;i<numcols;i++)
    obj[i]=(double)rand()/(RAND_MAX);
    //right hand side values of the constraints
    for(i=0;i<numrows;i++)
    rhs[i]=(double)rand()/(RAND_MAX);
    //the sign of constraints
    for(i=0;i<numrows;i++)
    sense[i]='G';//all >=
    //lower and upper bounds, and type of variables
    for(i=0;i<numcols;i++)
    lb[i]=0;
    for(i=0;i<numcols;i++)
    ub[i]=RAND_MAX;

    int NUMNZ=numcols*numrows;//number of nonzero entries in the constraint matrix
    //beg and cnt
    for(i=0;i<numcols;i++)
    {
    matbeg[i]=i*numrows;
    matcnt[i]=numrows;
    }

    //the index of nonzero constraint coefficients
    matind=new intNUMNZ;
    for(i=0;i<NUMNZ;i++)
    matind[i]=i%numrows;
    //constraint coefficients
    matval=new doubleNUMNZ;
    for(i=0;i<NUMNZ;i++)
    matval[i]=(double)rand()/(RAND_MAX);
    /* Initialize the CPLEX environment */
    env = CPXopenCPLEX (&status);
    /* Create the problem. */
    lp = CPXcreateprob (env, &status, "linear");
    /* Now copy the problem data into the lp */
    status = CPXcopylp (env, lp, numcols, numrows, objsen, obj, rhs,sense, matbeg, matcnt, matind, matval,lb, ub, NULL);

    /* Optimize the problem and obtain solution. */
    long tt=clock();
    for(long ii=0;ii<10000000;ii++)
    {
    CPXlpopt (env, lp);
    if(ii%100==99)
    {
    printf("%f\n",((double)clock()-tt) / CLOCKS_PER_SEC);
    tt=clock();
    }
    }

    status = CPXlpopt (env, lp);

    /* Free up the problem as allocated by CPXcreateprob, if necessary */
    status = CPXfreeprob (env, &lp);
    /* Free up the CPLEX environment, if necessary */
    status = CPXcloseCPLEX (&env);

    delete []obj;
    delete []rhs;
    delete []sense;
    delete []matbeg;
    delete []matcnt;
    delete []matind;
    delete []matval;
    delete []lb;
    delete []ub;
    }
    partial output:

    0.136000
    0.136000
    0.172000
    0.152000
    0.192000
    0.174000
    0.196000
    0.219000
    0.236000
    0.218000
    0.221000
    0.225000
    0.270000
    0.284000
    0.253000
    0.269000
    0.273000
    0.302000
    0.308000
    0.326000
    0.338000
    0.372000
    0.349000
    0.372000
    0.403000
    0.386000
    0.400000
    0.432000
    0.429000
    0.469000
    .
    .
    .

    I appreciate your help.
    #CPLEXOptimizers
    #DecisionOptimization


  • 6.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Wed September 26, 2012 09:51 AM

    Originally posted by: SystemAdmin


    I tried your code here and still could not reproduce the problem :-(
    I consistently get a solving time of 0.01 or 0.02 for 100 solves. Can you please take a look at the task manager while the loop runs. Do you see resources go up significantly while the program runs?
    Right now the times you print are in sub-second range. So different values may be caused simply by inaccuracies in the timer. Can you reproduce the problem (maybe with larger model instances) in a way that requires several seconds for a 100 iterations? Does the problem persist if you set CPX_PARAM_ADVIND to 0 before you enter the loop?
    #CPLEXOptimizers
    #DecisionOptimization


  • 7.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Wed September 26, 2012 11:48 AM

    Originally posted by: zhixin


    I tried the options you suggested but the problem is still there. The running time keeps increasing after many runs and becomes very long. After many many runs, my computer becomes very slow. From the task manager, the memory used is increasing slowly. It seems the memory used by CPXlpopt is not fully released.

    Do you use 32bit or 64bit compiler? I use 64 bit compiler.
    #CPLEXOptimizers
    #DecisionOptimization


  • 8.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Wed September 26, 2012 12:52 PM

    Originally posted by: zhixin


    In debug mode, each run of CPXlpopt generates some messages in the Output like:

    The thread 'Win64 Thread' (0x1087ec) has exited with code 0 (0x0).
    The thread 'Win64 Thread' (0x1087ac) has exited with code 0 (0x0).
    The thread 'Win64 Thread' (0x1087a8) has exited with code 0 (0x0).
    #CPLEXOptimizers
    #DecisionOptimization


  • 9.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Thu September 27, 2012 10:53 PM

    Originally posted by: zhixin


    I now use 32bit v12.4. The problem no more exists!

    I previously used 64 bit 12.3 version.

    Thank you for all the help!
    #CPLEXOptimizers
    #DecisionOptimization


  • 10.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Thu September 27, 2012 10:55 PM

    Originally posted by: zhixin


    The 12.4 32bit version does not have the problem. I previously used 64 bit 12.3 version.
    #CPLEXOptimizers
    #DecisionOptimization


  • 11.  Re: CPXlpopt runs slower and slower when called mutiple times

    Posted Mon October 01, 2012 11:26 AM

    Originally posted by: SystemAdmin


    That is really weird.
    But since I cannot reproduce the problem here there is actually not much I can do. I hope the switch to 32bit will not backfire to you.
    #CPLEXOptimizers
    #DecisionOptimization