Originally posted by: Qian Zhang

My problem may seem like a java memory management problem than a cplex memory management problem. 

I am building an integer program model in java. It contains a 6-dimensional parameter. Each entry in the parameter is double and this variable has 5 X 4 X 5 X 1000 X 4 X 300 entries. ( I don't know whether this considers huge or not)

I used Colt library's sparse matrix data structure, it still requires 6G heap space in JVM just to store this one with all other parameters. And the above size is just for a toy example. The real model may require the last dimension increase to around 1000 or 10000. 

I am using java 1.8, cplex 12.5 and the machine  (my laptop) only has 8G RAM. 

An obvious but last choice is to find a machine with larger RAM and run it on that one. 

Before doing that, I am wondering for several problems:

1) Are there spare matrix implementation in CPLEX for storing N-dimensional parameters?

2) This parameter is only in objective function and the computation of each entry in this parameter is quite easy.

    Is it a reasonable thing to compute its entry value every time when the model evaluates the objective function value?

3) Is it possible to install CPLEX in a machine from cloud service such as AWS and use it there?

Is this parameter sparse? What fraction of the entries are nonzero?

In answer to your second question, CPLEX has no provision for calling user code to evaluate the objective function. You need to supply the nonzero objective coefficients to CPLEX -- once -- before solving the model. You can certainly do that in a loop, computing each coefficient (once) and not storing it in your program's memory. CPLEX will have to store all the nonzero coefficients in its memory.

Originally posted by: Qian Zhang

Thanks for your answer.

This parameter is already stored in a sparse matrix data structure from colt library. The last dimension actually is around 1600 but non-zeros are around 300.

If it is possible to just pass the parameter through loop, is there any detailed references on this topic? Thanks.

Did you look at the examples that come with CPLEX and at the reference documentation? You will see that the objective function is represented as a linear expression. Like Paul said, you can easily build up that linear expression from a loop and it will only have the non-zeros you actually add.