IBM Watson Machine Learning (WML) is a service on IBM Cloud to execute any type of Data Science and Analytics job. As such, it includes not only all the Machine Learning libraries that you may want to use, but also the CPLEX and CP Optimizer engines to solve Decision Optimization problems.
The typical and recommended workflow is to create and test your model on Watson Studio (WS), and then deploy it to WML. Watson Studio offers a lot of features to develop DO models either using docplex, our Python modeling API, or OPL, our Optimization Programming Language. You can collaborate with others on these models, connect them to various datasets, compare the results of multiple scenarii, etc.
But what if you already have your DO model as a Python or OPL file? What if you already have some LP or MPS instances that CPLEX can read directly? In order to solve these on IBM Cloud, you will need to create the right type of deployment and submit your instances. That’s what dowml does for you…
With dowml, you have a Python library and a command line program to submit DO jobs, check their status, and retrieve their results. You can install it with pip install dowml, use it directly, or adapt the code (released under the Apache 2.0 license) in your own programs.
Note that dowml is not an official IBM product. It is a contribution that you are welcome to study and/or reuse. As such, it is not officially supported.
Here’s an example of a session where I solve a CPLEX instance stored in a compressed MPS file, with time limit of 30 seconds, on a Medium size deployment.
$ dowml
Decision Optimization in WML Interactive, version 1.0.0.
Submit and manage Decision Optimization models interactively.
(c) Copyright IBM Corp. 2021
Type ? for a list of commands.
Most commands need an argument that can be either a job id, or the number
of the job, as displayed by the 'jobs' command. If a command requires a
job id, but none is specified, the last one is used.
dowml>
When launching the interactive dowml program, you are greeted with a prompt so that you can enter the commands. Let’s check the type of model to be solved :
dowml> type
Current model type: cplex. Known types: cplex, cpo, opl, docplex
dowml>
Ok, we’re good here… Let’s check the time limit:
dowml> time
Current time limit: None
dowml> time 30
dowml> time
Current time limit: 30
dowml>
The default is to not have a time limit, and we just changed this to 30 seconds. Let’s check the deployment size:
dowml> size
Current size: S. Known sizes: S, M, XL
dowml> size M
dowml> size
Current size: M. Known sizes: S, M, XL
dowml>
The default is to use the S size of DO deployments, and we changed this to M. The size of the various DO deployment pods is documented at https://dataplatform.cloud.ibm.com/docs/content/DO/WML_Deployment/Paralleljobs.html#topic_paralleljobs__34c6?context=cpdaas&audience=wdp.
We are now ready to solve our model, which we will assume to be located in an ‘example’ subdirectory:
dowml> solve examples/acc-tight4.mps.gz
#######################################################################################
Synchronous deployment creation for uid: 'ffc3e89c-ac3f-428f-bc59-f59ba54f1c08' started
#######################################################################################
ready.
------------------------------------------------------------------------------------------------
Successfully finished deployment creation, deployment_uid='757e4970-a473-40b6-b358-7e5fdc38daee'
------------------------------------------------------------------------------------------------
Job id: 898b837e-83ee-410b-a4df-d6ec6957ad9a
dowml>
What we can see here is that a new deployment was created (I had not used a CPLEX deployment of size M before), and the instance was submitted in a job. As can be seen in the output below, this job is immediately queued for execution in the newly created deployment.
The jobs are ordered by creation date. We can see that my new job is still queued: when a deployment was just created, or has not been used in some time, it takes a few seconds for it to actually start processing jobs. The `*` before the name of the input file means that the file was stored as a data-asset in WML, and will not have to be uploaded again next time.
A few seconds later, the job is running:
And then it’s completed:
We can check the output of the job using the ‘log’ command. The '=>' at the beginning of the line in the output of the ‘jobs’ command marks the current job: a command that acts on a single job (e.g. log, details, output, delete) acts on that job by default. But you can specify a job number or a job id to act on other jobs (and make them the current job).
Given that the job executed correctly, we can retrieve the results:
The program created four files in the current directory : the solution found by the engine, a CSV file with some statistics from the engine, the log that was shown before, and the entire JSON structure that defines the job (this can be useful for debugging).
If you would rather give all the commands that you want to run as command line arguments, you can use the -c option to specify them:
$ dowml -c 'time 30' 'size M' 'solve examples/acc-tight4.mps.gz'
To install dowml, run pip install dowml in your Python environment. To learn more about the code, check the README at https://github.com/IBMDecisionOptimization/dowml. Or check the source code itself! The code is released under the open-source Apache 2.0 license. You are welcome to study it, reuse it, suggest improvements, or open some pull requests.
#DecisionOptimization