Originally posted by: Michael_D

Using Concert C++ 12.2 and trying to limit the amount of memory used for NodeData objects, the following happens:

When using a CMyNodeData object, where CMyNodeData is a subclass of NodeData, that has some stack-allocated data members, an optimization run of one problem instance uses at most x MB of main memory.

When changing the CMyNodeData object and allocating the data members on the heap with operator new and not implementing the destructor to free the allocated memory, the same amount of x MB of main memory is used according to the Windows task manager.

When implementing the destructor ~CMyNodeData and calling operator delete there, still the same amount of main memory is used.

The destructor is actually called quite often during the optimization run, and not only at the end.

According to the Cplex node log output to the console, the number of open, that is, remaining nodes, is always low (less than 100). Thus, one would expect that stack allocation and new with delete require only a comparatively small amount of memory, that this amount of memory remains more or less constant in the course of the optimization, and that the case of new without delete occupies more and more memory in the course of the optimization. What could be the reason why this is not the case?
A further question:

If the incumbent callback rejects a solution, what branching decision does Cplex take then?

Using a branch callback, under which circumstances is it possible that getNbranches(), when called in the main() function of the branch callback, returns 0 (zero)? I thought this would be the case if and only if there is also an incumbent callback and this incumbent callback rejected the solution at the respective node. However, in my code, this also happens when the incumbent callback has not been called for this node before.

In particular, might there be a situation where getNbranches() returns zero even when there is still a fractional variable?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: EdKlotz

> Michael_D wrote:
> Using Concert C++ 12.2 and trying to limit the amount of memory used for NodeData objects, the following happens:
>
> When using a CMyNodeData object, where CMyNodeData is a subclass of NodeData, that has some stack-allocated data members, an optimization run of one problem instance uses at most x MB of main memory.
>
> When changing the CMyNodeData object and allocating the data members on the heap with operator new and not implementing the destructor to free the allocated memory, the same amount of x MB of main memory is used according to the Windows task manager.
>
> When implementing the destructor ~CMyNodeData and calling operator delete there, still the same amount of main memory is used.
>
> The destructor is actually called quite often during the optimization run, and not only at the end.
>
> According to the Cplex node log output to the console, the number of open, that is, remaining nodes, is always low (less than 100). Thus, one would expect that stack allocation and new with delete require only a comparatively small amount of memory, that this amount of memory remains more or less constant in the course of the optimization, and that the case of new without delete occupies more and more memory in the course of the optimization. What could be the reason why this is not the case?
>

But does the number of nodes left actually influence the number of times
you allocate your node data object? I would expect it depends more on
the number of nodes created.

Unfortunately I don't really have any answers to the above questions, but I
have a couple of suggestions for tests you can try that might shed more light
on the issue.

1. If the new NodeData allocations really do depend on the number of
nodes left rather than created, then try a model that generates a large number
of nodes left and see what happens. If you don't have a model of your own
that does that, you can find plenty on the MIPLIB 2010 web site that do
(e.g. liu.mps).

2. Run Purify or some other memory checking tool to check for any memory
issues in the program.
>
> A further question:
>
> If the incumbent callback rejects a solution, what branching decision does Cplex take then?
>
> Using a branch callback, under which circumstances is it possible that getNbranches(), when called in the main() function of the branch callback, returns 0 (zero)?

If CPLEX finds an integer feasible solution at a node due to branching
and you reject it in your incumbent callback, then CPLEX has no branching
to perform and getNbranches will return 0. You need to use the makeBranch
function to do the branching, unless you are willing to let CPLEX prune the
node at that point. On the other hand, if CPLEX finds a feasible solution
from a node heuristic, CPLEX will still have fractional variables (or the
equivalent for other discrete objects such as indicator constraints) and can
proceed in the case where your incumbent callback rejects the solution.
I thought this would be the case if and only if there is also an incumbent callback and this incumbent callback rejected the solution at the respective node. However, in my code, this also happens when the incumbent callback has not been called for this node before.
>
> In particular, might there be a situation where getNbranches() returns zero even when there is still a fractional variable?

I don't think that can. You can check by calling the getFeasibilities method at the point you call getNbranches and see if any fractional variables exist.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: SystemAdmin

I have general questions:
1. How do you allocate data members of an object on the stack if the object itself is not on the stack (because that is what I understand you are doing)?
2. I am not a Windows expert but are you sure that the memory reported by the task manager is accurate? For example, if you release memory will that immediately be reflected in the task manager? Or will the memory manager pool that memory and the task manager will not even notice that the memory was released internally?
3. Is the size of data members and the number of NodeData instances big enough to actually make a difference in the memory consumption. I mean, if the data members occupy only a few bytes any you only have a few hundreds or thousands of nodes then a few KB more or less will probably not make a significant difference the task manager. What happens if you add an (unused) data member like

int dummy[4096];

to the CMyNodeData class?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Michael_D

Thank you both for your hints. Apparently, my first question was not well formulated. What I meant was the following. Consider

CMyNodeData : public IloCplex::MIPCallbackI::NodeData
{
public:
some_type member_1;
};

For such a class, wenn calling

CMyNodeData* mynode = new CMyNodeData();

memory is allocated on the heap for some_type.

On the other hand, consider

CMyNodeData : public IloCplex::MIPCallbackI::NodeData
{
public:
some_type* p_member_1;
~CMyNodeData { delete p_member_1; }
};

Then,

CMyNodeData* mynode = new CMyNodeData();

allocates memory on the heap for a pointer to an std::container<type>,

and in addition,

mynode.p_member_1 = new some_type;

allocates memory for some_type.

Thus, wenn using either class definition, all memory should be freed when the destructor of the CMyNodeData object that mynode points to is called. However, without the destructor, the memory allocated with mynode.p_member_1 = new some_type; will not be freed, since the default destructor only frees the memory for the pointer. This should be reflected in the memory consumption indicated in the task manager, but it was not.

However, when I created a really large data member as you proposed, the effect of implementing the destructor or not implementing it became visible in the task manager after some time. Apparently, the original data members were indeed to small to make a difference in the task manager even if there are tens of thousands of nodes.

As for the second question, Ed, you write that 'if CPLEX finds a feasible solution from a node heuristic, CPLEX will still have fractional variables...'. I don't understand that. A feasible solution has the constitutive property that it does not have any fractional variables that are required as integer in the model formulation. What do I misunderstand here?
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: EdKlotz

>
> As for the second question, Ed, you write that 'if CPLEX finds a feasible
> solution from a node heuristic, CPLEX will still have fractional variables...'. > I don't understand that. A feasible solution has the constitutive property that > it does not have any fractional variables that are required as integer in the > model formulation. What do I misunderstand here?

Sorry, I wasn't clear; I was referring to the node LP solution, not the heuristic
solution derived from the node LP and its solution.

The feasible solution derived by the heuristic indeed has no fractional values.
But, the node from which it was derived still does have fractional values that
CPLEX can branch on. That differs from the case where CPLEX finds the solution
from branching. The node LP solution in that case has no fractional variables for CPLEX to choose to branch on. So, in the former case, if your incumbent callback rejects the heuristic solution, you need not supply a branch callback to instruct CPLEX how to branch; it can still consider fractional values from the
node LP solution. In the latter case, the node LP solution has no fractional
variables, so you need to tell CPLEX how to branch if you want to explore children of this node.

Note that even in the latter case of no fractional values, there are two separate subcases to consider. First, the node LP solution that is integer
feasible may have all of the integer variable bounds fixed. That differs from the case where some integer variable bounds are not fixed, i.e. some integer variables were pushed to integer values by other branches, so they were at integral values without their bounds being fixed. The latter subcase offers
more branching options than the former.
#CPLEXOptimizers
#DecisionOptimization

Originally posted by: Michael_D

OK, thank you again.
#CPLEXOptimizers
#DecisionOptimization