Extending the value of mainframe data
The vast majority of Fortune 500 companies hold large amounts of critical business data on the mainframe. Much of this may be stored in relational structures on Db2 for z/OS. However, there is also a wealth of data stored on non-relational mainframe data sources such as Adabas, IMS, IDMS, VSAM, and flat files. Take as an example a bank ATM application, or airline reservation system which may be IMS based. IMS being a hierarchical vs. relational data store. This data can be extremely valuable for developing analytics applications based on customer data. It has proven to be a challenge to combine all these different data sources using traditional mainframe applications. Now, add to this the plethora of new data sources being structured, non-structured, on premises, cloud, etc and the challenge can become exponential. In the past companies have attempted to build data warehouses combining all this data into a single location. However, the tremendous volumes of data being generated today make the traditional ETL processes too costly and slow. In order for companies today to gain a competitive advantage they must be able to leverage all these disparate data sources for real-time analysis. Things like modern business analytics, 360 customer views, and mobile apps cannot afford to have potentially stale data.
Modern applications such as online-shopping, banking, etc are primarily API based. Leveraging mainframe data in these new applications has been a challenge due to the incompatible formats. It typically requires costly ETL processes in order to get the data into a recognizable format. Add to this, the typical programmer today has little or no experience with mainframe can make this an insurmountable task.
Virtualization can be the solution
IBM’s Data Virtualization Manager for z/OS is the only data virtualization technology that runs on the mainframe. Data Virtualization Manager for z/OS can make mainframe data much more consumable by providing an abstraction layer that provides real-time read-write SQL access to non-relational mainframe data sources without requiring any mainframe skills. Data Virtualization Manager for z/OS supports modern APIs such as web services, HTTP, and SOAP allowing developers to easily access mainframe data and join it with other data sources. And with z/OS Connect Enterprise Edition, Data Virtualization Manager for z/OS can readily support RESTful services. This can, for example, allow a developer to implement RESTful access to IMS or VSAM data.
IBM Data Virtualization Manager for z/OS architecture
View this short video for a more in-depth understanding of DVM architecture.
Data Virtualization Studio
Data Virtualization Manager for z/OS provides an Eclipse based UI called Data Virtualization Studio. This interface allows both main-framers and non-mainframers to easily build virtual tables from non-relational data sources.
It also allows a developer to generate code snippets in a broad range of APIs and interfaces:
Easily build virtual views that can join your z/OS data sources with other non-mainframe data sources such as Hadoop, Mongo, Oracle, Db2 LUW, Informix and many more.
Click this link for a walkthrough of the Data Virtualization Studio. You will learn about the following features:
- How to define virtual tables
- How to query virtual tables
- How to embed virtual tables in application code
- How to use virtual tables with web services
- How to create views using virtual tables
IBM Cloud Pak for Data can now readily access mainframe data
IBM Cloud Pak for Data provides data virtualization for many data sources both relational and non-relational. Cloud Pak for Data runs on a RedHat OpenShift cluster either on-premises or in the cloud of your choice (IBM Cloud, AWS, Microsoft Azure, or Google Cloud). Until recently Cloud Pak for Data’s mainframe data access was limited to Db2 for z/OS.
Cloud Pak for Data can now access IBM Z data sources via Data Virtualization Manager for z/OS, opening the door to all the other supported mainframe data sources. This was a missing link for deep analytics that can leverage all the valuable transactional data available on IBM Z. Developers and data scientists can now have transparent access to non-relational IBM Z data sources with the added benefit of Cloud Pak for Data’s collaboration and governance capabilities. View this video for a brief overview of Cloud Pak for Data’s data virtualization capabilities.