“Mainframe, Perception, Usefulness & Value”
As mentioned in my last Blog, Informatica has been working with many Enterprises & Organisations to help integrate Data from multiple structured and unstructured sources hosted on their IBM’s z & i5 Platforms (Mainframes) for multiple decades. Informatica provides access to the most common data sources held on the IBM platforms; namely Adabas, Db2, Datacom, IDMS, IMS, Sequential and VSAM. Now let’s take a more detailed look at the types of organisations that use Informatica solutions for integrating data hosted on the IBM platforms as well as the additional value that this provides to their businesses.
Over the past half-century of mainframe history, the machines have evolved rapidly. Modern mainframe computers are hardly the huge, crazily expensive, unwieldy machines of the original era. They readily integrate legacy with modern technologies, allowing you to do things like run COBOL apps on z/OS alongside Docker containers on Linux (using z/VM) on the same physical machine. So, the term “legacy mainframe”, for organisations that use the mainframe, are far from “legacy”; they form a critical part of their enterprise data processing infrastructure.
Mainframe?
So why should we care about mainframes? Most people probably don’t even realise that they connect with mainframes regularly in their day-to-day life, and even more under value of data that’s been accumulated on mainframes over the last 5 decades or more.
The reality is that just about everybody with a bank account, an energy provider, an insurance policy, or a driving license will probably connect with information residing on a mainframe. For example, if you’ve ever applied for a loan or used an automated teller machine (ATM) to get cash you will probably have connected to information on a mainframe to ensure that you are credit worthy for the request.
The mainframe owes much of its popularity and longevity amongst its customers to its inherent reliability, stability, scalability and security, a result of careful and steady technological advances that have been made since the introduction of the original mainframe systems. No other computer architecture can claim as much continuous and evolutionary improvement, from the original IBM platforms to the latest and soon to be released IBM z/Next architectures.
While other forms of computing are used extensively in business in various capacities, the mainframe (typically IBM Z platform) occupies a coveted place in today's e-business environment. In banking, finance, health care, insurance, utilities, manufacturing, government, and a multitude of other public and private enterprises, the mainframe computer continues to be one of the keystones to the foundation of modern business. Far from migrating away from it, organisations are expanding their capacities on it.
Informatica realised the importance of the mainframe many years ago and ensured that it also treated mainframe data as a cornerstone of its data integration capabilities, ensuring customers could get quick and easy access to one of their most valuable data assets without having to be a mainframe expert or write a single line of code.
Perception
The myth of mainframe technology is that it is old as well as being a very large collection of hardware stored in the back of the data centre that only certain people, of unconventional appearance, associated to a subculture in the organization who understand and know how to operate it all. Couple that with the understanding that the programming languages used for these machines, such as COBOL, PL/1 etc. are unpopular and out of date in the modern era. Such beliefs tend to make it difficult to get young developers interested in learning those languages and working on such hardware and software.
In a dramatic case in point, during the COVID Pandemic, a US New Jersey Governor, asked for volunteers who knew how to code COBOL to help handle the COVID-19 relief programs because many of the state's key applications still ran on mainframes. They didn’t have an easy method to get external access to mainframe data. The need for human intervention caused slowdown and delay in responding to challenge around the pandemic.
During the Pandemic however, Informatica helped overcome many of the challenges of connecting to complex data platforms and worked with many government & healthcare agencies to help seamlessly discover, collect, cleanse, govern and catalogue their data. All done with easy to use and codeless Informatica Graphical User Interfaces irrespective of where the data resides; even when that data resides on complex platforms like the mainframe. Informatica provided critical, up to date, streams of relevant cleansed data needed for a longer-term vision for feeding data to the evolving modern cloud-based systems that can enable regulated data collection, storage, governance, and analytics necessary even in the throes of a worldwide pandemic.
Usefulness
The mainframe environment has evolved with consistency for more than half a century. It’s been the rock on which many businesses built their IT infrastructure. Mainframes have reliably sustained the most critical business processes. However, while the rest of IT has galloped towards shared industry standards and even open architectures in on-premises/cloud systems, mainframe has stood aloof and unmoved. It operates largely within a framework of proprietary hardware and software that did not readily share data – and perhaps didn’t need to. But with the revolutionary pace of change, especially in the cloud, old notions of scale and cost have been cast aside. As big and as powerful as on-premises mainframe systems are, there are things the cloud can do more efficiently especially around flexibility of scale. IBM also provide cloud-based mainframe infrastructure to help mainframe customers with this major advantage over on-premises operation.
The mainframe is often still used by IT organizations to host the most important, mission and business-critical applications. Applications that typically perform order processing, financial transaction processes, production and inventory control, payroll, as well as many other types of application work. Mainframes are still a crucial component of a diverse and varied framework of information systems for industries like the following:
Banking
World banking typically relies on mainframes; needing to process enormous volumes of transactional data revolving around credit card transactions, ATM withdrawals, and online account updates. Investment banks prioritize high-frequency trading and need to react instantaneously to changes in financial markets. In both contexts, mainframes allow banks to process data on a scale that common commodity servers can’t handle.
Insurance
Mainframes are used by many of the top insurers worldwide. Insurance companies live and die on data – a lot of it. Data helps them assess risk, set prices, as well as invest in the right markets. Insurers depend on mainframes to make sure they can handle the data that drives their business.
Healthcare
Healthcare, too, is an industry where data is now king – and so, by extension, are mainframes. Mainframes power the secure, compliant, high-volume, and highly available data storage and transactions that make modern healthcare tick, helping keep people safe and healthy whilst also looking to find cures to the previously incurable.
Government
From the IRS to the National Weather Service, government agencies of all types need to store and analyse huge amounts of data. Mainframes are still helping them to do it.
Aviation
You don’t need to be a pilot to know that flight networks are complicated and ever-changing. That’s why airlines, not to mention government regulators that oversee airlines and even aircraft manufacturers, rely on mainframes to help ensure that people and planes arrive where they are supposed to in the most efficient way possible. When we travel the airline ticket has also probably been issued via mainframe systems.
Manufacturing
How many people know that many manufacturing operations still rely heavily on mainframes? Marque automotive brands, for example, still rely on systems that have been built on IMS, Db2, Adabas & IDMS but to mention a few.
Driven by demand, Educational Institutions around the world are teaming up with business to provide mainframe-based curriculum to (re)educate students on mainframe technology. 80% of the world's corporate data resides or originates from IBM mainframes. IBM mainframes (“z Platform”) are used by 95% of US Fortune 500 companies, 71% of global Fortune 500 companies, 25 of the top world banks, 23 of the top 25 US retailers and 9 out of 10 of the world's largest insurance companies.
Informatica, through its PowerExchange for Mainframe collection of connectivity products, as well as the newer Cloud Mass Ingestion suite of cloud services help customers in the above industries/institutions to easily connect to the common datastores on the IBM z-Platform mainframes, using simple easy to use and codeless Informatica Graphical User Interfaces. Evolving capabilities within Informatica mean that Informatica customers can now ingest data from mainframe, with both batch and CDC capabilities supported, to its new Informatica Data Management Cloud. Enabling customers to integrate the valuable data assets of the mainframe means that those assets can contribute to many of the clouds eco-systems (Azure, AWS, Google etc.) where reduced cost and ease of scale have enabled so many more data processing options.
Value
It has often been stated that data is “the new oil” that can power economic growth. If that’s true, then it is also true that mainframe data has been largely untapped, confined to use in traditional systems of record and given only the most limited exposure to modern analytics.
While the entire purpose of business intelligence (BI) is to find behavioural patterns in the data and infer future trends or actions that can benefit the business, many enterprises have been missing a key component, mainframe data. Without this precious core data, BI and modern analytics won’t live up to their potential. The mainframe must yield its secrets for overall success.
Analytics, providing insight to many different aspects of an organisations business whether it be for Fraud or Customer buying habits; Analytics is providing critical business insight which is helping shape direction and profitability.
Informatica is enabling technology and services for customers to not only ingest data quickly but also exploit unlimited scale when needed to at the flick of a switch. Informatica quickly integrates on-premises data from mainframes, relational databases, or data warehouse appliances to the fast-evolving plethora of Cloud Data Lake or Data Warehouse technologies. Data can be rationalised and cleansed for fit for processing downstream usage.
Such implementations, deployed against “new” data from newer on-premises and cloud systems, yielding extremely important insights. Those insights have been enhanced, often dramatically, by the availability and use of mainframe datastores of historic as well as current data.
Given an opportunity to use mainframe data more strategically and finally monetize its long-hidden value, most organizations would do so. Informatica has been providing technology to help expose that data for many years. Initially that was in conjunction with Informatica’s on-premises products.
However, Informatica is enabling those same capabilities for its new cloud services. Data can be extracted from the mainframe and loaded to a cloud target where it can be economically transformed into any standard format, combined with other data, and analysed as much and as often as needed. Informatica mainframe connectivity products and services can also provide low-latency, delta only, near real-time exchange of data for most mainframe data stores. In an expanding capability a lot of this activity can also be run near autonomously as processing can also accommodate Schema Drift. Lag time between data collection and analysis can be a tremendous barrier to intelligent data-driven decisions, especially in fast-moving scenarios such as a pandemic. To make the best choices, leaders need real-time access to information.
Regardless of the industry, an organization with the ability to liberate its mainframe data will have the capacity to monetize it, empowering new business applications, better services, and more refined operations. Since mainframe technology isn’t going away, businesses must do more to incorporate it and benefit from it.
.... and into the Cloud
The cloud was designed for handling vast amounts of data. The costs of storing, managing, analysing, and using data in the cloud are generally more favourable than any other option available on-premises. CEOs and key decision makers must find the right partner to manage mainframe data so they can move and transform it to a modern, standard format in the cloud where it is accessible to cloud applications. Informatica can be that partner as an independent and agnostic provider of all things data. Once in the cloud in a manageable, malleable format, the sky is the limit in terms of what enterprises can do and how they can benefit from this now accessible, usable data.
Grasping the potential of legacy data hidden away in mainframes can be a challenge. However, when business leaders get a taste of what is possible and see the ease of full, transparent integration of their mainframe data, they become quick converts to the process.
This evolution in mainframe data management will change how BI evolves over the next decade. User experience and product offerings will become more personalized. Companies will customize their offerings according to historical data liberated from the mainframe to enhance their business analytics. Enterprises that follow this path will have an opportunity to excel while those that don’t will likely be left behind.