Originally posted by: Jermerney

Hi,

I'm trying to solve a large MIP model using CPLEX in VS 2012 environment on a PC with 20 GB memory. The dimension of the variables is about 500*50*50*4, all of which are binary variables. I'm using "for" loops to represent the objective function. 

But, the model seems to be extremely difficult to load. It can only import the model with dimensions of about 200*30*20*4. The memory of my PC is exhausted quickly in this process. In addition,  once I add the first dimension to about 300, the computer can no longer load the model successfully, which sometimes crushes the computer .

I wonder if this problem that the model cannot be loaded successfully is due to the memory limitation of my PC, or is there some other way to deal with this tough problem?

Could you help me with this?

Thanks!

Jerry

The number of variables alone is not the problem. If you run the below code then you will see that memory for variables and objective only consumes half a gigabyte.

#include <stdio.h>
#include <ilcplex/ilocplex.h>

#define DIM1 500
#define DIM2  50
#define DIM3  50
#define DIM4   4

int
main(void)
{
   IloEnv env;
   IloArray<IloArray<IloArray<IloNumVarArray> > > x(env);

   for (IloInt i = 0; i < DIM1; ++i) {
      x.add(IloArray<IloArray<IloNumVarArray> >(env));
      for (IloInt j = 0; j < DIM2; ++j) {
         x[i].add(IloArray<IloNumVarArray>(env));
         for (IloInt k = 0; k < DIM3; ++k) {
            x[i][j].add(IloNumVarArray(env));
            for (IloInt l = 0; l < DIM4; ++l)
               x[i][j][k].add(IloIntVar(env, 0, 1));
         }
      }
   }

   std::cout << "Memory usage after variable creation:  "
             << env.getMemoryUsage() / (1024. * 1024.)
             << " MB" << std::endl;

   IloModel model(env);
   IloExpr obj(env);
   for (IloInt i = 0; i < DIM1; ++i) {
      for (IloInt j = 0; j < DIM2; ++j) {
         for (IloInt k = 0; k < DIM3; ++k) {
            for (IloInt l = 0; l < DIM4; ++l)
               obj += 1.0 * x[i][j][k][l];
         }
      }
   }
   model.add(IloMinimize(env, obj));
   std::cout << "Memory usage after objective creation: "
             << env.getMemoryUsage() / (1024. * 1024.)
             << " MB" << std::endl;

   env.end();
   return 0;
}

It may be the constraints that eat up your memory. Can you print out env.getMemoryUsage() frequently during model loading to figure out where the memory goes or which type of constraints consumes an unexpected amount of memory?

Originally posted by: Jermerney

Hello, Daniel!

Thanks so much for your suggestion. I just followed your suggestion and finally I found the problem. 

In my former programming, I use "obj = obj+x[i][j][k][q]" to represent the objective function, which consumes much more memory than "obj += x[i][j][k][q]". After I changed to this kind of representation, it consumes much less memory to read the model. I just wonder why does this happen? These two forms just representation the same meaning, but the memory consumption is totally different.

Thanks for your help!

Jerry

The explanation involves implementation details of expression building in Concert C++. An expression like

IloExpr = IloExpr + IloNumVar

uses an implementation of operator+ that basically builds an expression tree. So it will create a new IloExpr instance (the result) that has the original expression and the IloNumVar as children.

On the other hand

IloExpr += IloNumVar

uses an implementation of operator+= that just extends the existing expression. In many cases this can be done with a smaller memory footprint.
 

Originally posted by: Jermerney

Thanks a lot for your explanation! I did what you said, and it works now!

Thanks again for your help!

Jerry