In part one of this article, I looked at the impact that data is having on decisions and where to run and manage your analytics assets and why. In this second part, I will discuss the impact and benefits of running advanced analytics on some of the world’s most valued data and why the IBM Z platform offers some unique differentiation.
A Strong Foundational Data Source
Db2 for z/OS is a strong foundation for the IBM Z analytics portfolio with the latest iteration, version 12, providing enhanced performance over the previous version. Db2 leverages the reliability, availability and servicability capabilities of the IBM Z platform, which delivers five nines (99.999) percent
—near continuous data availability.
The Z analytics portfolio enables clients to gain near real-time insights from the data where it resides within the security rich environment of IBM Z.
The ability to ingest hundreds of thousands of rows each second is a critical application requirement, particularly for mobile computing and the Internet of Things (IoT) where tracking website clicks, capturing call data records for a mobile network carrier, tracking events generated by “smart meters” and embedded devices can all generate large volumes of transactions.
Many consider a NoSQL database essential for high data ingestion rates. Db2 12
, however, allows for very high insert rates without having to partition or shard the database—all while being able to query the data using standard SQL with Atomicity, Consistency, Isolation, Durability compliance.
Advanced in-memory techniques result in fast transaction execution with less CPU, making Db2 an in-memory database. Rich in security, resiliency, simplified management and analytics functionality, Db2 provides a strong foundation to help deliver insight to the right users, at the right time.
Db2 Analytics Accelerator for z/OS is an appliance that connects to the IBM Z platform. Using the optimizer code that has been in Db2 since version 7, Db2 decides whether it is best to run a query locally or on the appliance without any user intervention. Some queries executed on the Accelerator run up to 2,000 times faster.
Who wouldn’t want that capability?
Figure 1: IBM Db2 Analytics Accelerator and the IBM Z Platform
The Db2 Analytics Accelerator (Figure 1) can help deliver high value to Z clients as follows:
• Dramatically improves query response, up to 2,000x faster to support time-sensitive decisions
• Helps lower the cost of storing, managing and processing historical data
• Reduces data management, security and systems management costs
• Non-disruptive installation and easy migration in just days
• Helps eliminate off-platform data marts with a single, centralized infrastructure
• Extends Db2 for z/OS qualities of service and security to analytic workloads
• Protects and keeps data on a security rich platform
IBM Db2 Analytics Accelerator for z/OS V7.1 delivers support for the recently announced IBM Integrated Analytics System—a pre-configured high performance offering that includes hardware (i.e., compute, elastic data storage and networking) and software for easy deployment and management.
In addition to these capabilities, the Db2 Analytics Accelerator Loader for z/OS enables clients to load many of their other data sources to take advantage of the accelerator, including but not limited to IMS, IDMS, VSAM, sequential files, Oracle, Adabas and more.
Query Management Facility
Essential to the Z analytics story is the Query Management Facility (QMF)—a robust mobile-enabled analytics solution offering visual reporting, ad-hoc analysis and dashboards. It supports both Z and non-Z data sources.
Other business analytics tools may require data to move off the mainframe to another platform, resulting in potentially higher data management costs and the risk of insights being generated from outdated data. QMF virtualizes and federates within the z/OS environment, helping clients to analyze live IBM Z data.
Machine Learning for Data Scientists
For some organizations, some of their sensitive personal data originates and resides on IBM Z transactional systems supported by industry-leading qualities of service. So how does a data scientist access this rich, valuable source of data for machine learning without having to be a mainframe expert?
IBM Machine Learning (ML) for z/OS helps enable fast model development, deployment and monitoring for data scientists. ML for z/OS supports a hybrid approach to machine learning, supporting lifecycle management and collaboration (see Figure 2).
Figure 2: Machine Learning Lifecycle
Data scientists have the flexibility to train and evaluate behavioral models on IBM Z or an alternate platform of their choice. Application developers can deploy these models where the majority of their transactions occur: on the ML for z/OS platform. Co-locating machine learning execution with the data removes unnecessary network and infrastructure latencies.
Machine Learning production engineers can monitor resource usage and model accuracy. This complete lifecycle approach helps ensure that models remain accurate and continue to deliver optimized results.
ML for z/OS is built on open source technology, leveraging the latest innovations in this rapidly advancing software niche. It supports near real-time integration with transactional applications, allowing organizations to optimize interactions with customers. Data and insights can be integrated from other platforms. Data to be ingested within ML for z/OS doesn’t need to originate on IBM Z, but the IBM Z platform can provide encryption and a security-rich environment.
Machine Learning for z/OS Completeness
Some machine learning offerings only address specific components of the ML process. IBM ML for z/OS offers a broad spectrum of ML capabilities:
Cognitive Assistant for Data Scientists (CADS)
CADS Helps make it easier for data scientists to identify the algorithms and help create the right model. Without this capability, the data scientist has to find the best algorithm through trial and error.
Hyper Parameter Optimization (HPO)
This helps the data scientist select parameters to help optimize the predictive capabilities of the model.
DSX Pipeline User Interface
It provides an automated graphical user interface wizard making it easier for the data scientist to create, train and deploy the model. Data scientists can deploy their models in seconds with a single click.
Continuous Monitoring and Feedback Loop
Behavioral models have a tendency to degrade in performance over time. A continuous monitoring and feedback loop continually evaluates the performance of models as they’re exposed to new data. The data scientist can set a threshold for when he or she wants to be notified, if and when performance deteriorates, and schedule regular model evaluations. Feedback data is stored for retraining to help continuously improve the model performance.
Automated Model Maintenance
This helps organizations manage the hundreds and thousands of models via a visual dashboard.
The one thing you need to take away from all this is that these capabilities help make it easier for the data scientist to build, deploy and monitor behavioral models regardless of skill level.
Your Next Move
Having advanced analytics executing with co-located Z data and transactions, positions the IBM Z platform as a Hybrid Transaction/Analytics Processing (HTAP) platform. It’s a fast-moving environment and a lot is happening in the machine learning and artificial intelligence space.
To keep up to date on the analytics solution for IBM Z, check the latest news and announcements
. And to get a feel for machine learning, try it for no cost
Steven Astorino is vice president of Development, Private Cloud Platform and IBM Z Analytics, IBM. Follow Steven on Twitter.