Overcoming Inherent Challenges in Mainframe Application Modernization

Asit Dan

November 15, 2024

Contact: asitdan@optimum.net

LinkedIn: https://www.linkedin.com/in/asitdan/

Target Audience

This paper is developed keeping in mind three sets of readers who will benefit from the article:

• Decision makers/influencers in large organizations that are relying on core mainframe-based business applications supporting LOBs (Lines of Business): First and foremost, the objective of this article is to raise awareness with this key audience, that in spite of all the promises of magic solutions by various consulting practitioners and software providers, there is no easy option (i.e., simple, cheap and risk-free option) to address the inherent challenges for large scale modernization of such critical applications. Frequently, this is in response to Fear, Uncertainty and Doubt (FUD) in the marketplace. Second, maintaining the status quo will only exacerbate the challenges outlined in this paper, and may even lead to an eventual failure. However, on a positive note, there is a clear path with an incremental low cost, and low risk approach that not only aligns well with a longer-term application modernization strategy, but can also deliver near-term business value and agility with each incremental step.

• Consulting practitioners who have mostly been unsuccessful in selling large, big bang modernization proposals with high-risk solution approaches (e.g., total rebuild, replace with package solutions, rehost or magic transformation of core critical applications): These teams MUST understand the tremendous business risks and investment cost incurred by such approaches, without delivering any near-term benefits. This is the primary reason why such proposals have poor track records. Again, on a positive note, there are significant modernization opportunities with an incremental low cost, low risk approach that deliver near-term business value. Such approaches will definitely appeal to the LOBs relying on the core applications.

• Tools/Runtime software providers who would like to see their modernization-enabling products adopted: Product adoption depends very much on the alignment of LOB priorities established based on business risks, solution cost and near-term value delivery. Many times, organizations have purchased such tools and runtimes but have not put those into production, due to the inherent complexities and challenges described in this article. On a positive note, modernization is a team play; Tools/Runtime providers can see greater adoption by working jointly with consulting practitioners on LOB priorities by delivering incremental solutions. Without properly prioritizing business risks, cost and near-term value for specific LOBs, lack of adoption will remain the status quo.

Executive Overview

Modernization of mainframe applications is essential for all enterprises, dependent on mainframe applications. This is due to both the criticality of these core applications to businesses, and significant challenges faced in extending, enhancing, and maintaining these existing applications. On the other hand, wild west (unproven) approaches in transforming and/or migrating these applications to other platforms will introduce significant risks to such business-critical applications, and will also incur a higher cost of transformation. Furthermore, organizational gaps in failing to bridge the objectives, interests, and constraints faced by different roles, especially LOB/application owners, enterprise architects and other key stakeholders interfere with all modernization efforts. Regardless, maintaining the status quo by doing nothing will result in even more difficult challenges ahead (such as, further loss of knowledge and skills in maintaining these applications). Breaking this standstill, however, will require a well thought out modernization approach - with low risk and lower cost - which needs to be further aligned with the objectives of many different stakeholders.

In this article, I will first summarize the distinguishing characteristics of core mainframe applications, i.e., what makes mainframe applications so special: including scalable transaction processing in order to maintain integrity of business-critical systems of records (SOR) data, critical business dependence on these applications resulting in high risks in modifying these applications, and a long/complex evolution history, manifesting in many technical and organizational challenges. I will then summarize the dominant inherent complexities in modernizing such mainframe applications:

1. Difficulty in discovering undocumented business and detailed design requirements

2. Lack of business and IT functions alignment in the monolithic code, i.e., lack of componentization of code with identifiable business functions, essential for this alignment and for providing foundational support for any application modernization

3. Complexity of code due to organic evolution history with intermingling of presentation and business logic

4. And finally, lack of mitigation of high business risks with any large change (e.g., uncertain solution quality, high transformation cost and a lack of near-term value).

I will then emphasize why maintaining status quo poses an even higher business risk. Fortunately, there are incremental approaches for modernizing such applications which will mitigate such high risks. I will then elaborate on the primary focus of the paper, i.e., the key organizational challenges that MUST be addressed for any modernization approach to succeed.

Here is a summary of the key challenges and recommendations in addressing these:

Distinguishing characteristics and history of mainframe applications

Mainframe applications most often are core to large enterprises in maintaining shared and critical business information. For maintaining business integrity, this shared common information is updated transactionally while also applying a lot of due diligence checks as codified through business rules and application logic. This information is then shared across all business services spanning many lines of businesses (LOBs). Frequent transactional updates to a large set of SOR business critical data demands support for scalable transaction processing¹, which can be provided only by a scalable mainframe platform that is highly efficient, secure, and reliable². Mainframe architecture with large processing nodes, memory, a shared disk architecture and a shared coupling facility ³ copes extremely well with such dynamic situations⁴. In contrast, for transactional workloads, all distributed/partitioned and shared nothing architectures require a large amount of capacity to be set aside to cope with (i.e., to provide both a low response time and a high throughput under) various dynamic situations, such as, a sudden surge in load of one transaction class, varying transaction rates for all transaction classes, and a failure of a single processing node ².

Due to the inherent nature of these SOR applications, - i.e., core to the business and hence, the need to adapt to the changing business requirements - these applications have evolved organically over a long time period (e.g., many decades in some instances). On the other hand, given the criticality of these core applications, there is an extreme diligence by all businesses in making any changes to the code. This quite often results in extreme risk aversions by most businesses for making any new changes, especially any large changes. This manifests in the code with many unhealthy risk mitigation (including aversion) practices and many accumulated adhoc changes. As a result, there is a lack of agility in making any new changes to these applications, resulting in long delays in responding to new business requirements to meet marketplace demands, and incurring generally a higher cost of maintenance for these applications.

These aggregated adhoc changes over a long period in these applications have manifested not just in terms of monolithic structures of the applications but also tangled codes through intermingling of business rules, presentation logic and other business functions (i.e., with many ad hoc practices). The lack of structure in the code (i.e., componentization and well-defined business functions) due to aggregated adhoc changes and resulting complexity of the tangled code are especially noteworthy in dated screen-flow based business processes.

Inherent challenges in modernizing core mainframe applications

Modern software development practices address many of the agility challenges in developing new functions. These include, (to name a few)

• componentization of applications that helps in localizing changes to code, finding reuse of components across solutions, as well as automated testing and deployment of components,

• avoiding proliferation of data and data quality management.

Unfortunately, existing monolithic application code cannot always benefit from these modern practices until the existing code is transformed/modernized with componentization and alignment of business and IT functions, i.e., by turning components into reusable services, while also doing these transformations so carefully without introducing high business risks. This modernization, however, remains a challenging task. To make matters worse, the original and subsequent business requirements and long evolution of codes are not well documented.

Contemporary approaches, such as, microservices⁵, domain-driven design ⁶ and eventual consistency ⁷ for maintaining data integrity do provide some guidelines for modernization. Domain-driven design could provide a business-level framework to guide refactoring of existing application codes. It provides recognition of coarse-grained boundaries inherent to the problem-space. These boundaries form top-level associated components into which microservices can be formed as part of the refinement towards implementation. A common characteristic is that transactional consistency is mostly required within a domain whereas eventual consistency is tolerable for business functions spanning domains. However, employing them successfully with the existing code is challenging and one of the reasons this paper exists.

Summarized here are some key inherent complexities (due to the above characteristics of core critical applications) which make modernization of mainframe applications extremely challenging.

Systemic application modernization challenge 1:

How to decipher original and subsequent, business and detailed design requirements, that led to the development of the existing application code?

This includes not just the original high-level business requirements, but all subsequent refinements such as additional detailed constraints introduced in the application logic for extreme diligence (e.g., various forms of validity check or detailed steps in the business processes, say, in an insurance policy quote or tax calculation). Also, business requirements change with time, and subsequent continual incremental changes in business and detailed design requirements have led to further modification of the code. Unfortunately, in most cases the requirements and/or design changes were never formally documented in the first place or lost due to a long evolution history. Deciphering these aggregated business requirements and detailed design of the application solely from these lines of code alone is an impossible task.

  Lost or not documented application blueprint: business requirements & detailed design

The diagram above illustrates this challenge. The lines of businesses (LOBs) who conceived the original business needs and drove development of applications, provided further detailed requirements in designing various aspects of the application. For example, they provided details of business logic, such as sequence of business tasks to be implemented, business rules to be validated, information models to be maintained and even more demanding but essential aspects such as business data integrity to be implemented through transactional update of data. Other important aspects of the application design included interfaces or APIs to be supported and/or dependency on external functions via APIs, as well as important non-functional requirements spanning security of maintaining critical business data, support for high availability, lower response time, high throughput, dynamic workload, etc.

Soon after the initial design, development and deployment into production of the initial application, continuous stream of new business requirements either in the form of new functions or enhancing the original scope of the functions arose. Unfortunately, for historical reasons, in most cases the original business requirements and detailed design specifications (which I will refer to as application blueprint) were either never documented or subsequently lost. Additionally, to avoid high risks due to any large changes in the core critical applications, always minimal changes were made in the production code. So instead of iterating through a proper design cycles with new/modified requirements, ad hoc changes were made again and again over a very long period of time. This makes many applications very complex monolithic code segments.

In the physical world, all critical infrastructures (such as roads, buildings, power stations, etc.), large and small appliances (from planes, cars, large machines to small household appliances) and even houses have a design blueprint, essential for maintaining, repairing and remodeling these. Can you imagine digging a road without knowing the underlying gas, sewer, water pipes and electrical cables that can be torn, causing disruption and even fire? The same caution needs to be applied in updating critical core applications. Computer science has been a relatively young science (even with 50 or more years of history). Perhaps, the time has come for government regulations to emerge in providing guidance and setting standards for maintaining application blueprints for critical supporting services (from Financial, Transportation, Governmental to many other critical industries).

All modernization approaches – whether building anew completely from the ground up, replacing existing code with another new solution package, or transforming existing code - faces the same challenges of deciphering and documenting “original and aggregated business and detailed design requirements”, since any new or modified solution MUST VALIDATE that it will meet the business requirements as met by the existing code. Finally, all modernization approaches also MUST ENSURE the same high level of quality of services with any new or transformed solution as delivered via the original solution. (Also unfortunately, these non-functional requirements are not explicitly documented or specified).

To make the point on the need for explicit business requirements clearer, consider the necessary requirements, including necessary detailed specification, in purchasing ANY product, delivering ANY service, or building ANYTHING new. First, consider the simplest example of replacing any household item, say a TV, and you will probably have a check list of specifications, say, on the display size, display resolution, etc. for selecting this replacement TV. You may be indifferent on some other attributes, and may even find improvement with a replacement, but you MUST have an explicit list of attributes that you really care about. Next for a more complex replacement item or a service order, say for an interior design, you WILL NEED a more detailed specification (as captured typically via a service proposal). Going a step further, in replacing a mainframe-based information repository, say an existing set of VSAM files or an IMS DB, you need to redesign the new datastore with both the definition of information elements (e.g., customer, order, etc.) and the relationship across items. How can you then replace an existing solution with new codes without an explicit detailed specification of business and detailed design requirements for this solution?

This lack of explicit documentation of application blueprint (i.e., business and detailed design requirements), and even, lack of understanding of original business and detailed design requirements from older codes plagues all organizations in maintaining such codes. The real issue in maintaining older codes is this missing knowledge, and NOT solely the specific programming language or specific mainframe environment (as often made out to be).

Recommendation 1: Incrementally decipher each application component and gather/document business and design requirements supported by this code component

One obvious implication of the above challenge is to apply a great deal of diligence in incrementally drilling down existing code, as well as gathering contextual knowledge by talking to SMEs, in deciphering business requirements for that component. Subsequently make smaller, risk mitigated changes as per the current requirements, while aligning each such incremental step with the longer-term modernization objectives.

Priority: This is hard work and there is no easy path to avoid this step. This knowledge is also absolutely essential, even to replace an existing solution component with a new software package or recreating this code ground up. On the positive side, this step is both essential and complementary to all other recommended practices.

Systemic application modernization challenge 2:

How to align Business and IT functions, i.e., where each well-defined business function can be mapped to an identifiable application component?

This will mean not just all code is componentized, identifiable by a business function it supports, but a well-documented catalog is maintained for easily finding code components supporting a desired business function (e.g., interest calculation, changing customer address, etc.). This is essential in designing reusable shared services⁸. These code components are seldom static but evolve with changing business and detailed design requirements. The business and design requirements come from multiple stakeholders (i.e., solution developers/LOBs), and may differ across stakeholders at times. Hence, the development process needs to follow proper diligence protocols in reconciling requirements from multiple stakeholders. Upon proper deliberation, on some occasions, the application components are duplicated to diverge from existing codebase (i.e., to support similar but different business functions), such that new requirements are applied only on the replicated copy. The overall process necessitates implementing SOA (Service Oriented Architecture) governance⁹ not just for reconciling business requirements from multiple consumers/LOBs in evolving business functions and avoiding duplication of functions, but also for maintaining a trusted catalog for easily finding reusable services for any new business solution need, and finally, coordinating changes to reusable code in notifying stakeholders of the new version of this function.

As an illustrative example; consider an insurance application that was originally developed for implementing a policy rate calculation function only for individual customers, however, may now need to implement a similar function for a specific business group defined by many attributes (e.g., size, geography, etc.). The decision on whether to modify the original code component to cover new business requirements or duplicate this component to support a similar but different function – i.e., one for individual and one for the specific business group - are handled by the Governance process. Monolithic code in existing applications (i.e., lack of componentization) means generally a missing or weak service governance framework. Also, existing web services are complex interfaces, or at a very coarse level of function definition rather than meaningful easy to consume reusable business services. Hence, the future goal of refactoring old code MUST first address this lack of service governance framework to avoid opening the pandora’s box with code duplication and code quality issues. Second, many incremental patterns¹⁰, such as exposing existing code as services, extending existing code with new external/cloud-based functions, enhancing an existing identifiable component with small code changes or even replacing with a new implementation, etc. are excellent starting points for a low risk and low-cost approach to application modernization that will deliver near-term business outcomes with every step. Each incremental step leads to additional business and IT functions alignment.

Recommendation 2a: Incrementally define business services and identify application components implementing each service, resulting in incremental Business and IT alignment

Practicing incremental modernization patterns as listed on the IBM Z and Cloud Modernization Center, such as, EXPOSE, EXTEND, ENHANCE, REFACTOR an application component into a service as well other DATA and EVENT centric patterns ¹⁰, leads to incremental defining of business services and identifying corresponding supporting application components. (Incremental modernization patterns also help address various organizational challenges, as noted later in the article.)

Priority: This componentization practice is both incremental and essential.

Recommendation 2b: Incrementally implement a strong service/API governance framework^8,9 with a trusted reference repository, and governance processes in avoiding replication and proliferation of overlapping codes, as well as for orderly deployment of a new version of a code. (This is necessary to achieve Business and IT alignment)

Without a trusted reference repository and a governance process in enforcing proper due diligence while creating a new service (i.e., without aligning all stakeholders for this service) will lead to code proliferation and large overlaps across services. The reference repository MUST contain not just high-level business description of a service but also details of the requirements implemented by the service. New requirements are added and/or updated while evolving a service. As and when a service evolves, and a new version of code is to be deployed, a governance process needs to be followed to inform and obtain necessary participation of all stakeholders of this service.

Priority: The implementation of a complete governance framework may not be essential for getting started with incremental modernization patterns, and for enabling APIs for a handful of Services. However, this can be built ground up as more and more services are defined. This recommendation is complementary to all other recommended practices.

Systemic application modernization challenge 3: 

How to untangle complex monolithic code implementing screenflow processes that evolved over a long period with many incremental changes?

This often manifests not just in terms of monolithic structures of the applications but also through intermingling of business rules, presentation logic and other business functions (i.e., accumulated results of many ad hoc practices for mitigating business risks to core applications). Again, the starting point for untangling such complex existing code must be to incrementally isolate and externalize small identifiable functions (e.g., as reusable services across many solutions such as currency conversion, policy quote or frequently changing business rules).

Recommendation 3: Incrementally untangle each screen flow process, first through identifying and refactoring reusable services, and then through enhancing the overall process as a service

Untangling a 3270 screen-based application, where presentation, business process steps and business logic tasks or transactions are intermingled, is extremely challenging. The untangling of such code can be best approached in two steps:

• First, identify and factor out components as shared services, such as, currency conversion, rate calculation, policy validation, etc. as external services. Such services can be standardized and shared across the enterprise by multiple business solutions.

• Subsequently, the overall process can be exposed as service in one of two ways:

  • Use a screen scraping emulator and expose the remaining code (i.e., existing process) as a service

  • Alternatively, rewrite a new interactive process application using a preferred language as a service

Priority: Refactoring reusable services is the first step before modernizing the overall process. However, a screen scraping approach to expose existing solution can be prioritized (even before refactoring reusable services, if required to meet new requirements driven by marketplace needs.

Systemic application modernization challenge 4:

How to mitigate business risks given critical dependence on these core applications, i.e.., mitigating high risks in making large changes to these applications?

Risk mitigation is the highest priority, which makes it harder to transform all at once or replace with another solution to avoid high risk of creating an unreliable solution. Small changes with managed risks can be tolerable, and hence, the modernization approach SHOULD be incremental. As we will see later, small changes can also help in coping with budgetary constraint and investment risk associated with the high cost of larger transformation, where each incremental step will deliver near-term benefits.

Recommendation 4: Incrementally modernize by mitigating business risks from large changes and delayed time-to-value from modernized solutions by leveraging IBM Z application modernization patterns¹⁰

IBM Z and Cloud Modernization Center lists a set of well thought out IBM Z application modernization patterns that meet the above modernization criteria. Each pattern is driven by a specific modernization need and provides a low risk, low-cost solution that will deliver immediate value to the business.

The recommended stating points are exposing existing functions or data as APIs, extending existing application with new external functions, and generating business events for downstream consumption for new solution enhancements. Also, another good starting point is to incrementally modernize DevOps, i.e., new agile development and deployment processes with industry-standard tools (e.g., GitHub) and applying to a small set of applications.

These incremental modernization patterns will not only address immediate business needs, but will also help to build reusable service governance framework (see earlier discussion).

Priority: Applying these incremental patterns avoids creating large risks associated with alternative application modernization approaches.

Modernizing mainframe applications is essential and unavoidable

Maintaining the Status quo of leaving the code alone due to inherent risks and challenges is not a sustainable long-term option. As already noted, lack of agility in the development of either new functions or modifying and evolving an existing function incurs a huge opportunity cost and also operational cost in maintaining the existing applications. Hence, modernization of applications is necessary both to meet new requirements and to benefit from modern development practices. Modernization is also necessary to benefit from extending purely mainframe-based solutions to an agile and/or cost-effective hybrid-cloud deployment. On the other hand, any new approach in transforming mainframe applications that does not take into account business risks and constraints faced by the business is simply a talking head, academic exercise.

There are multiple risks and constraints. First and foremost, the risk of any transformation to core business critical applications is the risk that it poses to the quality of any new solution. Businesses have come to rely on mainframe-based applications in terms of performance, availability, security, and business integrity of information. Any modernization approach must minimize the risk to this quality of services. Equally, the risk of incurring a high cost of development in transforming existing code is a non-starter. Any potential investment for modernization must demonstrate a near-term return. As it stands, many of today’s wild west modernization approaches driven simply by cost savings measures (e.g., total migration to cloud and/or complete rewrite, etc.) do not withstand the scrutiny when examined from business risks perspective. As a result, most businesses are still sitting on the sidelines, and are yet to move forward with modernization of existing mainframe applications.

The choice of whether or not to modernize applications – in maintaining the status quo vs avoiding high immediate business risks - can be best understood with another situational metaphor: maintaining personal health. Without a proactive disciplined approach to well-being (through both avoiding a standard carb-laden everyday diet and embracing regular exercise), many people remain oblivious to the dangers to their health lurking underneath: obesity, heart-attack, diabetes, cancer, dementia, etc. With a proactive approach in maintaining health (say, with a low carb/high fiber diet and with regular exercise), one can avoid suffering from many forms of eventual diseases arising from such a laissez-faire lifestyle. Hence, maintaining a status quo means eventually suffering such diseases and then incurring a very high cost of expensive medical procedures, if not an early death. A well thought out proactive low risk, lower-cost modernization approach similarly avoids a comparable outcome for core business critical applications.

Recommendation 5: “ACT NOW” to avoid further ossification of monolithic applications and for incremental alleviation of pain-points that drive mainframe application modernization

Delaying of modernization of existing applications achieves nothing; it not only incurs further business pains (i.e., lack of agility, high cost of maintenance and erosion of application knowledge/skills) but also accumulates further technical debts. On the other hand, applying these incremental patterns alleviates not only immediate pain, but also enables a path to the longer-term goal of creating agile solutions with componentized services/business and IT alignment.

Priority: Get started TODAY with incremental modernization by developing modernized solutions to address immediate business needs. It can be a long journey with many incremental steps - in identifying business scenarios, identifying services, identifying supporting code components, deciphering and documenting business requirements, and implementing appropriate incremental patterns in exposing, extending, enhancing or refactoring applications. However, following this approach, each step is risk and cost mitigated and will deliver immediate business value, and is aligned with the longer-term business goals.

What is and is not modernization of mainframe applications?

Currently, there is no explicit definition of application modernization or any external measures by which to assess a modernization initiative. The trigger points and business drivers are well known, such as loss of skills, optimizing operational cost or lack of agility, i.e., difficulty in confidently making timely changes to code in responding to new business requirements. However, in the absence of an explicit definition or assessment, almost every technology provider or consulting service proposal – irrespective of business risks, actual feasibility or solution quality, high transformation cost and long delivery timeline - claims to modernize existing applications. Each promises a specific improvement in such proposals (say, improved operational cost after migration) and ignores many other factors which very much matter to the LOBs/organizations. Such proposals rarely see adoption in practice and many initiatives (and sometimes after huge expense) quietly fails as they never had clear modernization objectives addressing each of the important issues, or were too optimistic in ignoring the actual challenges. However, anecdotally (and from personal experience), there are several success stories of successful incremental modernization approaches, whether with expose, extend, enhance of existing code, virtualization of data for easier external access or with incremental DevOps modernization.

In light of the above discussion, I would define application modernization as incremental steps to transform existing applications which improves agility in meeting new solution needs and makes it easier to maintain existing solutions through service orientation and componentization, without incurring high business risks (i.e., in cost and solution quality). Equally, modernization of applications would mean to actually transform the applications by leveraging the architectural capabilities already put in place. Without this transformation of applications, no amount of architectural/modernization strategy and even purchasing and deploying tools and software capabilities will result in actual modernization of existing applications.

Modernization is a team effort

Any modernization of application delivering new solution will require bringing multiple teams together, by aligning their interests, and working jointly in developing new solutions:

• LOB executive - owns the applications, apprehensive of the risks and costs and hence, MUST approve any modernization effort, and better yet, champion any modernization initiative

• Business solution architect - defines the use cases driving application transformation and implement an end-to-end solution

• Mainframe application developers - possess the knowledge of the applications and will implement transformation to mainframe applications

• Enterprise architect - defines an architectural strategy and puts in place integration capabilities

• Cloud application developers - implement new cloud-based functions with a hybrid-cloud strategy

• There are other modernization participants as well; When developing any modernized solution by adopting new capabilities (say, exposing APIs), it must leverage existing DevOps or in conjunction incrementally modernize DevOps itself, such as automated testing and deployment.

The diagram below illustrates how multiple stakeholders collaborate in identifying, designing, developing and implementing a modernized solution. Platform owners are responsible for providing hybrid cloud platform with mainframe and cloud infrastructures.

The enterprise architecture team defines architectural strategy covering integration, security, DevOps and service/API governance. An application team supporting a LOB, are not only knowledgeable about the application but also are driven by LOB needs and constraints: new business scenarios to be supported, mitigation of business risks, budgetary constraints and time-to-value for such solutions. However, any changes to application MUST follow architectural strategy and seeks appropriate support from the hybrid-cloud platforms for deployment into production.

   Mainframe Application Modernization Stakeholders

Key organization challenges in application modernization

Having established the followings factors:

• Clear application modernization objectives for aligning business and IT functions with componentization and reusable services to improve business agility

• Inherent complexities which manifest as difficult application modernization challenges and can only be overcome with diligent incremental steps

• All such incremental modernization steps MUST pay attention to the business risks and outcomes, i.e., mitigate high risks, incur a lower cost and deliver near-term business value

I will now highlight four key organizational challenges which must be addressed to help businesses move forward with the modernization of mainframe applications by implementing a well thought out pro-active approach. In future articles, I will also cover how the modernization journey in organizations using an incremental approach and hybrid architectural strategy have resulted in successful incremental modernizations.

Organizational modernization challenge 1:

How to break the status quo by aligning the interests of all stakeholders with a pragmatic modernization strategy?

Any modernization initiative MUST bring all participants together with a coherent strategy and an architectural approach.

Let’s examine a few of the key stakeholders/roles participating in any application modernization as listed earlier.

• First, a LOB that is critically dependent on an existing application MUST approve any modernization initiative for this application from both solution risk and cost perspectives. Also, any modernization of a mainframe application would demand deeper participation of the mainframe application developers with both their knowledge of the application and making actual changes to the mainframe-based application. Typically, such teams are risk averse and also constrained by their budgets. So, without a proactive approach from this team modernization takes a back seat.

• Next, let’s examine the role of a business solution architect who will define business use cases and an end-to-end solution architecture implementing solutions (following the architectural strategy set by the Enterprise architecture team). Please note that modernization of applications are not some black-box operations, but rather an effort to improve existing solutions with transformation of applications by leveraging new approaches, capabilities, and platforms. Modernization of an existing application, including extending, exposing, enhancing, or componentizing code and building new or enhanced solution, need to be validated against such business use cases. Any modernization initiative should not introduce high risks, but equally importantly must demonstrate immediate value within a reasonable timeline.

• Next, the Enterprise architecture team defines not just the platform and architectural strategy, but also deploys integration capabilities across solution components. Hence, the architectural strategy MUST pay attention to business risks and cost incurred by different modernization approaches. Without buy-in from application teams, establishing architectural strategy and/or deploying integration capabilities has little use in practice.

• Next, all modernization efforts MUST embrace modern cloud-based approaches to building new functions/application components. Different approaches may differ in the overall cloud-based solution components and how much transformation is applied all at once.

• Finally, there are other participants to modernization as well: integration with existing DevOps Teams or in conjunction, incrementally modernizing DevOps with automated testing and deployment.

Any modernization initiative MUST bring these participants together with a coherent strategy and architectural approach.

Recommendation 6: Designate a mainframe application modernization leader

This leader MUST:

• Demonstrate a deep understanding of the business drivers for LOBs, i.e., risk and cost tolerance and demand for time-to-value, (and not just focused on technical architectural strategy)

• Persuade all stakeholders to take a proactive approach to mainframe application modernization, given the risk aversion and challenges

• Be able to bridge the organizational gaps by aligning interests of all stakeholders.

Good business solution scenarios are MUST in persuading all stakeholders to demonstrate the near-term value, risk mitigation while incurring a lower cost and keeping to longer term modernization objectives.

Priority: It is highly recommended to have such a champion who can take a proactive approach, prioritize immediate business needs, brings all the stakeholders together, aligns their interests and drive implementation of modernized solutions.

Organizational modernization challenge 2:

How to mitigate risks and the high cost of modernization across stakeholders?

As already noted, any modernization initiative that does not acknowledge and mitigate business risks associated with the selected approach and minimize costs while at the same time, demonstrates immediate value cannot find approval and/or will not likely succeed. In some instances, after a significant due diligence, an organization could take a proactive/deliberate long-term approach to radically rebuilt or replace with another solution, while introducing very little risk to an existing solution. There must be abundant caution not just on the cost of such a modernization approach, challenges in deciphering (original plus aggregated incremental) business requirements, but equally importantly on the qualities of the new solution (performance, availability, security, and business integrity).

Recommendation 7: Adopt an incremental modernization approach with low risk to business while incurring a lower cost for modernization and delivering near-term business value

Earlier recommendation (#4) already covers the need and priority for incremental modernization from a technical challenge and a low risk to solution perspectives. Incremental approach is also a realistic approach from a low cost and addressing immediate business solution need perspective.

• In selecting business scenarios, asses various business risks (e.g., solution quality, cost, time-to-value, available implementation skills, etc.) with any modernization approach by selecting an approach with low risk and lower cost which delivers near-term value

• Apply an appropriate modernization pattern for solutioning this scenario. Each pattern is driven by a specific modernization need and provides low-cost solution(s) that demonstrate immediate value to the business

• Please see recommendation #4 for good starting points for applying patterns

Priority: Applying these incremental patterns avoids creating large risks associated with alternative application modernization approaches.

Organizational modernization challenge 3:

Lack of knowledge in implementing application modernization, i.e., both knowledge of mainframe applications as well as hybrid-cloud implementation technical capabilities.

Mainframe application modernization actually means developing and deploying new modernized business solutions, irrespective of the modernization approach and the associated degrees of effort, risk, and cost. This is in contrast to modernization of a platform by putting in place enabling capabilities for a new architectural strategy and getting ready for actual application modernization. Hence, building this new solution is definitely a team effort with the mainframe application being at the core.

I will illustrate this team effort with a simple example with possibly the lowest risk, a lower cost, and a quick time-to-value modernization solution: Exposing an API from a core mainframe application to build a new digital transformation solution with an enhanced customer reach (say, with a mobile application in a bank or insurance). Building such a solution not only needs to leverage knowledge of the existing core application and DevOps capabilities but also needs to integrate new enabling capabilities for API and Cloud based functions.

• End-to-end solution design:

  • A business solution architect gathers business requirements, assesses business risks, development costs and time-to-value in mapping business drivers to potential solution alternatives. The incremental patterns provide guidance in this solution mapping process, and in this example will adopt exposing APIs from core mainframe applications to be invoked by cloud based new mobile applications.

  • While information needs and the set of APIs invoked by this new digital application may be provided by the end-to-end solution design team, realization and implementation of such APIs cannot be done without the deep knowledge of the mainframe-based application. An existing CICS or IMS program may take many input parameters which may or may not be available to the front-end application. This must be defaulted based on the context of the invoking application and or derived through other functions. On the side of retrieved information, a determination must be made about what critical business information returned by the core application is safe to reveal. Hence, it will require deep knowledge and approval of the information, i.e., application owner

• API Implementation: For brevity, I will not elaborate on this task. However, exposing an API will require use of tools/runtime like z/OS Connect and API management platforms

• API Deployment: Deploying a new solution into production will demand high qualities of service for the end-to-end solution, including deploying the API in a scalable and highly available configuration as well as a comprehensive security implementation

• DevOps integration: Developing and maintaining API artifacts and deploying into Test and production environments will require integration with existing DevOps processes

• API Management: Deployed APIs need to be published, and access to APIs need to managed and monitored

• Cloud based front-end app: Again, for brevity, integrating cloud-based functions that invokes exposed APIs from mainframe applications will involve collaboration across Cloud application development, integration, and mainframe application teams.

Recommendation 8: Assemble an end-to-end solution team consisting of SMEs in application, business solution architect and developers implementing all components of the hybrid-cloud solution

• Engage internal teams, e.g., business solution design, applications SMEs, API integration, Cloud, DevOps, etc. where feasible, and

• Engage external teams complementing internal skills as needed

Priority: Most organizations may seek external/additional help in delivering incremental solutions. Both being project driven and bounded nature of such external help makes incremental approach realistic.

Organizational modernization challenge 4:

Lack of a strong service governance framework across stakeholders

As noted earlier, a key goal of application modernization is to improve business agility through alignment business and IT functions which can only be achieved by componentizing existing code identifiable by the specific business function each component implements. Also noted earlier, multiple good starting points for incremental modernization, each with low risk, lower cost and near-term delivery of value. Each incremental step identifies a new service, e.g., expose, extend, enhance, etc., invokable by an API. New business solutions can be composed using these APIs. These APIs and the associated details of the business functions implemented by the APIs can be easily referenced in a trusted repository. There needs to be extreme diligence in not just publishing these APIs to the right consumers, but also coordination in creating and releasing new versions as business requirements change with time. Also, there needs to be governance process in duplicating existing code for bifurcating and creating new APIs.

However, existing monolithic code, with its lack of componentization can generally mean a missing or weak service governance framework. Also, existing web services are complex interfaces, or at a very coarse level of function definition rather than well-aligned, easy to consume reusable business services.

Recommendation 9: Incrementally implement a service/API governance framework across stakeholders for addressing the service orientation of existing code components

As businesses start with low-risk modernization steps and incremental documentation of exposed APIs/services, they also must put in place not just a trusted repository of APIs, but also processes to govern i) new APIs by gathering and reconciling requirements from multiple consumers, and ii) managing changes to the associated code, and orderly process of releasing new versions.

So, this recommendation of implementing governance processes across the stakeholders is complementary to recommendation #3 of implementing technical framework for trusted service repository, maintaining information on supporting code components, business requirements implemented by the code, etc.

Implementing a strong service/API governance framework with a trusted reference repository, and governance processes avoids replication and proliferation of overlapping codes, as well as for orderly deployment of a new version of a code. This is all necessary to achieve Business and IT alignment.

Priority: This goes hand-in-hand with recommendation #2b. The implementation of a complete governance framework may not be essential in getting started with incremental modernization patterns, and for enabling APIs for a handful of services. However, this should be built from the ground up as each new service is defined.

Conclusions:

Maintaining the status quo, in order to avoid the challenges of driving the modernization of mainframe applications will simply exacerbate the future challenges of maintaining these core critical applications, as well as responding to the enhancement needs. To avoid this stagnation organizations must seek:

• A proactive approach to mainframe application modernization by designating a mainframe application modernization leader who can align the interests and bring together primary and other stakeholders, especially, the LOB executives owning applications and the EA team driving the architectural strategy

• A pragmatic incremental modernization approach with low risk to business and lower transformation cost while delivering near-term returns

• A comprehensive solutioning team consisting of essential SMEs: business solution owners defining use cases driving mainframe applications, mainframe application developers transforming applications and implementing a hybrid cloud architecture spanning mainframe and cloud. This team can be assembled with skills both from internal teams and complementary external skills as needed.

In future articles, I will cover the next steps for:

• Establishing the right architectural strategy for successful mainframe application modernization that enables effective sharing of enterprise data, code, and platform

• Pragmatic incremental modernization journey given the criticality of these applications and an always constrained budget

• Effective ways the strategy and journey have resulted in successful incremental modernization

Acknowledgements:

I would like to thank Charles Rost, Ian Mitchell and Yves Tolod from IBM and Skip Richards from Kyndryl for reviewing multiple earlier drafts of the paper and providing in-depth feedback through multiple discussions in both sharpening the messages for different types of readers, and improving the clarity of the overall content of this article. Charles, Ian, Yves, and Skip also have been long time collaborators in driving mainframe application modernization with various organizations.

Selected References:

1. A. Dan (1992). Performance Analysis of Data-Sharing Environments. https://direct.mit.edu/books/book/3878/Performance-Analysis-of-Data-Sharing-Environments

2. P.S. Yu, A. Dan. Performance evaluation of transaction processing coupling architectures for handling system dynamics. IEEE Transactions on Parallel and Distributed Systems ( Volume: 5, Issue: 2, February 1994) DOI: 10.1109/71.265942

3. Jeffrey M. Nick, Gary M. King, Jen-Yao Chung, Nicholas S. Bowen, Ching-Shan Peng. Parallel Sysplex: A Scalable, Highly Available, High Performance Commercial System. Journal of Parallel and Distributed Computing, Volume 43, Issue 2, 15 June 1997, Pages 179-189, https://doi.org/10.1006/jpdc.1997.1342

4. A. Dan, P.S. Yu, D.M. Dias. Performance modelling and comparisons of global shared buffer management policies in a cluster environment. IEEE Transactions on Computers ( Volume: 43, Issue: 11, November 1994) DOI: 10.1109/12.324561

5. Martin Fowler, Microservices Guide. 21 Aug 2019, https://www.martinfowler.com/microservices/

6. Roman Glushach, Domain-Driven Design (DDD): A Guide to Building Scalable, High-Performance Systems. Medium, Oct 5, 2023, https://romanglushach.medium.com/domain-driven-design-ddd-a-guide-to-building-scalable-high-performance-systems-5314a7fe053c

7. Sukumar Sundar Singh Molleti, Understanding Eventual Consistency: A Key Concept in Distributed Systems. Medium, Jun 10, 2024, https://medium.com/@Sukumar_Sundar/understanding-eventual-consistency-a-key-concept-in-distributed-systems-12f4004186ab

8. Asit Dan, Robert D. Johnson, and Tony Carrato. SOA service reuse by design. In Proceedings of the 2nd international workshop on Systems development in SOA environments (SDSOA '08). Association for Computing Machinery, New York, NY, USA, 25–28. https://doi.org/10.1145/1370916.1370923

9. Martin Keen, et. al. Implementing Technology to Support SOA Governance and Management. https://www.redbooks.ibm.com/redbooks/pdfs/sg247538.pdf

10. A. Dan, et al. IBM Z application modernization patterns. https://www.ibm.com/community/z-and-cloud/application-modernization-patterns/