Thanks Leo for isolating this sample code, I'll study it in detail to see if I can improve this.
When you read a SAV into DOcplex, it delegates the read to the native-code CPLEX DLL, so the basic read is "as fast" as CPLEX C code.
The next step is to rebuild a full DOcplex model from the in-memory matrix, and this is what takes time in Python:
in your case, creating 10 million Python "var" objects takes time.
There's not much we can do there; more precisely,
it is not possible to build a partial docplex view of a huge model.
That's an interesting idea, and we'll see what we can do in that direction.
For now, the only working alternative for very large models is using the Python matrix api (or C/C++)
------------------------------
Philippe Couronne
------------------------------
Original Message:
Sent: Fri November 06, 2020 01:27 PM
From: Leo Singer
Subject: Reading docplex.mp model from a SAV file is slower than recreating it from scratch
Hi,
I have a large docplex.mp model with about 10 million binary variables. I would like to generate and save the model in advance, and then later load it, make some small adjustments, and solve it. I found that although it takes less only 10 seconds to write the model using Model.export_as_sav(), it is taking 7 minutes to read it back in using ModelReader.read(). Reading the model back in is by far the dominant cost, because solving it only takes 45 seconds!
Here's a very minimal example with a toy model that just contains 10M binary variables, and shows that ModelReader.read() is slower than building the model from scratch.
Are there any tricks to speed up reading in a saved model?
Thanks!
In [1]: from docplex.mp.model import Model
In [2]: from docplex.mp.model_reader import ModelReader
In [3]: m = Model()
In [4]: %time x = m.binary_var_list(10_000_000)
CPU times: user 17.1 s, sys: 468 ms, total: 17.5 sWall time: 17.5 s
In [5]: %time m.export_as_sav('model.sav')
CPU times: user 691 ms, sys: 278 ms, total: 969 msWall time: 970 ms
Out[5]: 'model.sav'In [6]: %time m2 = ModelReader.read('model.sav')
CPU times: user 43.3 s, sys: 2.27 s, total: 45.6 sWall time: 46.2 s
------------------------------
Leo Singer
------------------------------
#DecisionOptimization