Decision Optimization

#DecisionOptimization

This question was answered two years ago on Stack Overflow: https://stackoverflow.com/questions/61981147/serialize-cplex-concert-objects. Curiously, the OP was "J. Bloom" -- you?

At any rate, if you have an instance "lp" of IloCplex, you can serialize lp.getModel(). When you deserialize it, you deserialize to an instance (call it "x") of IloCplexModeler, then use lp.setModel(x) to turn that back into an IloCplex instance.

------------------------------
Paul Rubin
Professor Emeritus
Michigan State University
------------------------------

Original Message:
Sent: Sat January 14, 2023 03:57 PM
From: Jeremy Bloom
Subject: CPLEX and Spark

I'm building an application that uses Spark to manage optimization models to be solved with CPLEX. I'm creating and storing a bunch of CPLEX artifacts (IloNumVar,  IloRange, etc) in Spark RDDs. I'm using IloCplexModeler to create the artifacts.

I've worked out most of the bugs and I've run the application successfully using Java Streams in place of Spark RDDs.

However, when I run with Spark, I get the error 

 java.io.NotSerializableException: ilog.cplex.IloCplex.

I note that IloCplex implements Serializable, so I'm puzzled about why I'm getting this error. It's a show stopper.

I've done a bunch of research on the web, including checking Stack Overflow, but only a few posts address the integration of Spark with CPLEX, and I can't find any useful code.

Can anyone explain how to work through this issue? Sample code would be greatly appreciated.

BTW, I'm using Java, CPLEX Optimization Studio 12.9, and Spark 3.2.1.

Thanks.

#DecisionOptimization

Paul reminded me that I asked a similar question a couple of years ago. I've just come back to this project, so I'll look at the answer again.

What I've concluded is that you can't serialize the individual CPLEX modeling artifacts, like IloNumVar, only a whole model. That's not what I need, since I want to use the artifacts in Spark to create a model. I've decided to use Spark to create my own artifacts (which would be Serializable classes) and then generate the model from these artifacts, avoiding serializing the CPLEX artifacts.

------------------------------
Jeremy Bloom
------------------------------

Original Message:
Sent: Sat January 14, 2023 05:54 PM
From: Paul Rubin
Subject: CPLEX and Spark

This question was answered two years ago on Stack Overflow: https://stackoverflow.com/questions/61981147/serialize-cplex-concert-objects. Curiously, the OP was "J. Bloom" -- you?

At any rate, if you have an instance "lp" of IloCplex, you can serialize lp.getModel(). When you deserialize it, you deserialize to an instance (call it "x") of IloCplexModeler, then use lp.setModel(x) to turn that back into an IloCplex instance.

------------------------------
Paul Rubin
Professor Emeritus
Michigan State University

Original Message:
Sent: Sat January 14, 2023 03:57 PM
From: Jeremy Bloom
Subject: CPLEX and Spark

I'm building an application that uses Spark to manage optimization models to be solved with CPLEX. I'm creating and storing a bunch of CPLEX artifacts (IloNumVar,  IloRange, etc) in Spark RDDs. I'm using IloCplexModeler to create the artifacts.

I've worked out most of the bugs and I've run the application successfully using Java Streams in place of Spark RDDs.

However, when I run with Spark, I get the error 

 java.io.NotSerializableException: ilog.cplex.IloCplex.

I note that IloCplex implements Serializable, so I'm puzzled about why I'm getting this error. It's a show stopper.

I've done a bunch of research on the web, including checking Stack Overflow, but only a few posts address the integration of Spark with CPLEX, and I can't find any useful code.

Can anyone explain how to work through this issue? Sample code would be greatly appreciated.

BTW, I'm using Java, CPLEX Optimization Studio 12.9, and Spark 3.2.1.

Thanks.

#DecisionOptimization

After a bunch of research and coding, I'm abandoning my effort to use Spark to create CPLEX models. I'm happy to discuss my findings in detail if you want to contact me directly.

------------------------------
Jeremy Bloom
------------------------------

Original Message:
Sent: Sun January 15, 2023 07:25 AM
From: Jeremy Bloom
Subject: CPLEX and Spark

Paul reminded me that I asked a similar question a couple of years ago. I've just come back to this project, so I'll look at the answer again.

What I've concluded is that you can't serialize the individual CPLEX modeling artifacts, like IloNumVar, only a whole model. That's not what I need, since I want to use the artifacts in Spark to create a model. I've decided to use Spark to create my own artifacts (which would be Serializable classes) and then generate the model from these artifacts, avoiding serializing the CPLEX artifacts.

------------------------------
Jeremy Bloom

Original Message:
Sent: Sat January 14, 2023 05:54 PM
From: Paul Rubin
Subject: CPLEX and Spark

This question was answered two years ago on Stack Overflow: https://stackoverflow.com/questions/61981147/serialize-cplex-concert-objects. Curiously, the OP was "J. Bloom" -- you?

At any rate, if you have an instance "lp" of IloCplex, you can serialize lp.getModel(). When you deserialize it, you deserialize to an instance (call it "x") of IloCplexModeler, then use lp.setModel(x) to turn that back into an IloCplex instance.

------------------------------
Paul Rubin
Professor Emeritus
Michigan State University

Original Message:
Sent: Sat January 14, 2023 03:57 PM
From: Jeremy Bloom
Subject: CPLEX and Spark

I'm building an application that uses Spark to manage optimization models to be solved with CPLEX. I'm creating and storing a bunch of CPLEX artifacts (IloNumVar,  IloRange, etc) in Spark RDDs. I'm using IloCplexModeler to create the artifacts.

I've worked out most of the bugs and I've run the application successfully using Java Streams in place of Spark RDDs.

However, when I run with Spark, I get the error 

 java.io.NotSerializableException: ilog.cplex.IloCplex.

I note that IloCplex implements Serializable, so I'm puzzled about why I'm getting this error. It's a show stopper.

I've done a bunch of research on the web, including checking Stack Overflow, but only a few posts address the integration of Spark with CPLEX, and I can't find any useful code.

Can anyone explain how to work through this issue? Sample code would be greatly appreciated.

BTW, I'm using Java, CPLEX Optimization Studio 12.9, and Spark 3.2.1.

Thanks.