Originally posted by: dfc02139

Hello all, I am trying to solve a large LP with CPLEX. The LP has 2mm constraints and 100mm variables. I create the model in C++ (the code is below.)

I have a machine with 128gb of memory at my disposal, although not quite all the memory is always available to me. When I try to run the instance, it uses about 70gb of memory. For 2-4 hours, it seems to run ok (if I look at the process, the CPU is being fully utilized.) After that, the process is still alive, but the CPU runs at 1%. 

I suspect this behavior is caused by the fact that CPLEX is using too much memory and starts using the disk. But isn't the memory usage way too high to begin with? I've tried halving the number of threads used to see if that decreases memory usage, but it didn't help. Any advice would be appreciated.

Also, is there some sort of verbose mode to CPLEX, as right now it is not outputting anything as it executes..

 

void Instance::ComputeCPLEXValue(vector<__gnu_cxx::hash_map<int, long double> >* matrix,

                                 vector<__gnu_cxx::hash_map<int, long double> >* transpose_matrix) {

        IloEnv env;

        IloInt i, j;

        IloModel model(env);

        

        cout << "building model \n";

        NumVarMatrix x(env, n_);

        

        cout << "building impression constraints \n";

        for(i = 0; i < n_; i++) {

            x[i] = IloNumVarArray(env, (*transpose_matrix)[i].size(), 0.0, 1.0, ILOFLOAT);

        }

        

        // Add impression constraints.

        for(i = 0; i < n_; i++) {

            model.add(IloSum(x[i]) <= 1.0);

        }

        

        vector<int> n_index;

        n_index.reserve(n_);

        for (int i = 0; i < n_; ++i) {n_index.push_back(0);}

        

        IloExpr obj_exp(env);

        for(j = 0; j < m_; j++) {

            IloExpr curr_adv_constraint(env);

            for (__gnu_cxx::hash_map<int, long double>::iterator iter = (*matrix)[j].begin();

                 iter != (*matrix)[j].end();

                 ++iter) {

                curr_adv_constraint += ((double) iter->second) * x[iter->first][n_index[iter->first]];

                n_index[iter->first]++;

            }

            model.add(curr_adv_constraint <= ((double) budgets_[j]));

            obj_exp += curr_adv_constraint;

        }

      

        model.add(IloMaximize(env, obj_exp));

        obj_exp.end();

        

        cout << "initializing cplex object \n";

        IloCplex cplex(env);

        cplex.setParam(IloCplex::Threads, 12);

        cplex.setOut(env.getNullStream());

        cout << "extracting model \n";

        cplex.extract(model);

        

        cout << "solving model \n";

        // Optimize the problem and obtain solution.

        if ( !cplex.solve() ) {

            env.error() << "Failed to optimize LP" << endl;

            throw(-1);

        }

       

        cout << "CPLEX opt is " << cplex.getObjValue() << "\n";

        

        env.end();

    }

 

 

#CPLEXOptimizers
#DecisionOptimization

You explicitly disabled CPLEX output by calling

cplex.setOut(env.getNullStream());

You should remove that statement to see what CPLEX is doing. Can you please post the output here? That would give us a better idea of what is going on here.

#CPLEXOptimizers
#DecisionOptimization

Originally posted by: dfc02139

I've rerun it, and I'm attaching the output. I'm seeing the same behavior, at some point, it just stops using the CPU. Is there a way to set CPLEX to give some sort of verbose output (like memory usage, etc..)?

#CPLEXOptimizers
#DecisionOptimization

There is no verbose mode for displaying memory or thread usage. Since the problem seems to appear while solving an LP you could try setting SimDisplay to 2 which gives some more information about Simplex iterations.

Does the problem persist if you use 1 thread instead of 12?

Could you export the model as SAV file (cplex.exportModel("model.sav")) before solving and attach the SAV file here?

#CPLEXOptimizers
#DecisionOptimization