Cloud Pak for Data

There are multiple types of deployment, and we will consider some common patterns. While the reference here is in the context of IBM Cloud pak for data services, the concept applies to other tools as well.

Here is some typical pattern for Machine Learning  Model Deployment :

• Online Web service - This deployment provides an API endpoint, mostly used in case of online scoring. The API can be embedded into front end applications through REST API calls. Use this deployment if the scoring data is generated from user feedback such as forms, events in CRM, etc.   Anti-pattern –bulk scoring, a very high volume such as a recommendation system for millions of users.

• Batch – Use this deployment type If we need to do bulk analysis of the data. Usually this pattern is applied when you have big data data residing in databases, Hadoop or object store. Mostly scheduled over weekend or sometime as often as daily basic. Prominent example would be credit card individual risk profile generated based on historical datasets by most of the retail banks.

• Edge Deployment – This involves deploying model at the edge devices for an application that requires privacy, low latency, disconnected operation, and distributed computing. Example: Deploying models directly on mobile devices.  

• Remote Deployment - With docker based deployment, remote devices can be any disconnected nodes in a hub and spoke model for a multi-geo secure implementation. This allows security and low latency similar to edge deployment, even though it may not be really embedded into an application.

• In-database deployment – In this case, the model is deployed within a database to support batch prediction for data residing in a database. Most modern analytics database support built-in machine learning (ML) functions. These functions allow you to train and run models on data residing in databases. Operations are performed by the database engine itself, with no data movement required. In-database deployment help accelerates performance, but usually, the ML functions are limited and may need re-coding, validation if you built your model using the open-source framework.   However, this is a robust and time tested mechanism if you have a fewer category of models that are not updated frequently.

• Deployment with a Streaming application - Application like click-through analysis, IoT, or cybersecurity have data streaming endlessly, and ML model needs to detect a pattern, abnormal behaviour. In such a scenario, it is best to deploy the model as part of the streaming workflow. Most streaming engine such as Storm or IBM Streams provides an operator where the model can be deployed directly as part of the data flow.

Deployment with Watson Machine Learning (WML)

Watson machine learning is deployment service available as part of IBM Cloud Pak for Data. Watson Machine Learning supports all phases of the model development lifecycle, giving you the tools to design, train, and save your model. When it comes to deploying, Watson Machine Learning provides these benefits:

• One-click deployment for models saved to Watson Studio.
• No-code, form-based scoring for AutoAI.
• Code samples to help you structure your input data.
• Options for generating real-time predictions or for setting up and scheduling batch predictions.

 WML provides multiple method to deploy the models, the most common approach being client Python Library, Rest API  and deployment console.

Watson Machine Learning Client Python Library –

Python Client Library is my preferred mechanism and would recommend for any developers. This allows you to deploy and execute the model right from your notebook or python code even from a third-party development tool. This would be preferred approach for testing your experiments. The package is available on pypi. You can use below command to install it.

$pip install watson-machine-learning-client

For the usage of the library refer to the documentation here.

https://wml-api-pyclient.mybluemix.net/

Deployment through Rest API -

Just use a curl or https call to deploy, list, run your model. Each instance have a swagger files that provides the list of API available specific to the deployed instance.

Here is a reference to the deployment API.

https://watson-ml-api.mybluemix.net/#/Deployments

WML console for deployment –

WML provides a web console through which a model administrator can deploy a model. The actual workflow varies slightly from version to version, the interface offers complete administrator control over the deployment and usually a preferred approach for a managed production cluster.

While the tools support most type of deployment, the choice of the build framework can also limit deployment choices. Here is the reference to the various supported framework.

https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/pm_service_supported_frameworks.html?linkInPage=true