Originally posted by: srinit34

Hi

I am solving a difficult MIP which takes several days to solve and needs hundreds of GB of memory.

So I want to use "save node files to disk and compressed".

I am also using a branch callback in which I attach nodeData into every node that is created.

  makeBranch( vars[ZERO],  bounds[ZERO],dirs[ZERO],  lpEstimate  ,  new NodePayload());
  makeBranch( vars[ONE],  bounds[ONE],dirs[ONE],   lpEstimate,  new NodePayload());

In my branch callback I retrieve the nodedata and use it  

NodePayload nodeData = (NodePayload) getNodeData(); // do something with nodeData

My concern is that saving node files to disk and also using node data attachments will cause exceptions.

What are my options ? I can maintain the node data outside of CPLEX (i.e. not use node attachments) which may be slower .

Why do you think it will cause exceptions? This should work without problem.

Note however that the user data is not exported/compressed to disk. It is kept in memory. So this may eventually fill up your memory. If this happens then you have only one choice: Implement your own storage manager for NodePayload that can swap data to disk as need be and pass only handles as node data to CPLEX. getNodeData() will then return a handle and you have to resolve that handle with your NodePayload storage manager.

Originally posted by: srinit34

Hi Daniel

In the past, it have seen exceptions like "node data not found on disk" when I have used nodeData similarly. 

I concluded ( it seems incorrectly) that it was because of the reason I mentioned.

I cannot find the forum thread  about this. I had asked a similar question earlier and you had mentioned that an exception like "node data not found on disk"  was expected. I cannot find the exact question or thread.

The problem with using my own data structure for managing nodeData is that, I do not know which nodes were infeasible and feasible, so I unnecessarily keep holding node data for such nodes in memory even after they are gone from the cplex tree.

Thanks you for comment.

Originally posted by: srinit34

Hi Daniel

in this thread , you mention similar problems.  Is it revelant ?

https://www.ibm.com/developerworks/community/forums/html/topic?id=6c9250c7-4052-431a-a49d-99b518974ab2

It appears that there is a problem inspecting nodes that are currently sitting on disk in compressed format.

But you pointed out that there is no problem invoking getNodeData() for the current node. Am I correct ?

This is something different (and you may be confusing things).

1. There should be no specific problems with using user data at nodes and node files.

2. You cannot get any information about nodes that are currently swapped out to disk (this is what this other thread is about). This includes querying the node data. If you attempt to query information for nodes that are currently swapped out, you will see this exception. This is not a problem of user data attached to the node. The user data is just one of the pieces of information you cannot query.

I suggest you use user data at the node just like you had planned and just prepare yourself for this exception.

By the way, are you still using Java? If yes then if the data attached to the node implements the IloCplex.MIPCallback.NodeData interface then its delete() method will be invoked when the node is deleted (for whatever reason). That way you could notice whether a NodePayload has to be deleted.

In C++ things are even easier since you can just override the destructor of the NodeData class.