Authors: Angel Tcholtchev, Jonathan Heywood, Rob Tiberio, Subhash Ramachandran
Introduction
To open the topic, let’s look at a few interesting numbers. Gartner has followed IT markets for many years and predicts a doubling of the number of organizations using cloud services between 2018 and 2022. The total amount of money spent on cloud services is also set to double in the same period to almost €300 billion per year. Yet moving to the cloud brings its challenges. When questioned a couple of years ago by KPMG, 54% of CIOs said integration with the existing landscape was the biggest barrier to cloud adoption. McKinsey recently asked CIOs about their IT modernization projects, where cloud plays a major part. They found that 80% of such projects have not brought the agility or business benefits that were expected. So, if shifting to the cloud is so difficult, why are companies doing it? And how can they ensure a greater chance of success?
Cloud migration drivers
Let’s look first at a typical CEO’s priorities. They are not with technology, and they’re not even with cost reduction. According to McKinsey, CEOs’ top priority is revenue growth, closely followed by increased agility and reduced time to market. Most realize that digital technology is a key enabler for these goals and expect their CIO to be instrumental in delivering.
However, in most cases the CIO doesn’t have the luxury to be able to define a modern new architecture. They spend up to 80% of their annual budget just maintaining the systems they have, leaving precious few resources available for innovation. CIOs we speak to realize that IT modernization is needed, and that cloud technologies will be part of that modernization. But it is often difficult to know where to start. What options are available? And what approach is best for which type of application?
Different paths to the cloud
Let’s look at a couple of simple models that can help you think in a structured way about your journey to the cloud. There are basically three ways you can choose to move capabilities to the cloud.
Lift & shift
Here, you typically pick up an existing application and deploy it as-is to infrastructure-as-a-service (or IaaS) in the cloud. There are multiple advantages with this approach - you can move to the cloud quickly and the software will no longer run on a computer in your data center that you need to operate, maintain, backup and upgrade. The disadvantages are that it’s still the same software, and you still need to operate, patch and debug it on your own. This as-is, on-prem architecture will not be flexible under load nor will it have a built-in high availability.
Adopt SaaS
The second path is where you choose to adopt the capabilities provided by a software-as-a-service, or SaaS application. Good examples of this are Salesforce, ServiceNow or Marketo. As you move a particular business function to a SaaS application, you may be able to switch off or decommission an old application on-premises. The advantages here are that the applications are cloud native, with clear pricing and no operational effors from your side. As a disadvantage we can mention that those SaaS apps, are often adopted by your competitors, meaning you can’t differentiate your business with them.
Build cloud native solutions
The third path is where you choose to build brand new cloud-native applications using all the powerful capabilities provided by a platform-as-a-service (PaaS) such as the AWS stack or Azure stack. This may also allow you to wind down obsolete on-premises applications. A key advantage of this approach is the faster time to market, which makes it very suitable for your systems of innovation. Also, by building your apps from scratch, you can leverage the best the cloud providers offer – e.g. databases as a service and easy access to huge storages that can be used as data lakes for analytics. As disadvantages, we can list the danger of vendor locking to a particular cloud provider and the often unpredictable costs, that may easily explode if the solution is not engineered carefully.
You will most likely use a mixture of these three paths. But which approach is most appropriate for which type of application?
Well, Gartner gives us another great model that can help here: the so-called pace-layer model.
At the base of your landscape are the systems of record. These are the foundational systems that keep your company running – your financial ledger, your HR system, your warehouse management system. They hold all your core data, highly governed and without frequent change. You can’t afford to take any risks with these systems, but you will also not beat your competition with them. They are standard applications every enterprise runs but are also often heavily customized to fit your company specifics.
Systems of differentiation form the next layer. These are the systems that allow you to do what you do better, faster, cheaper than your competitors. They are what give you the edge and are usually not provided by packaged applications. By their very nature they are custom-built. And you will want them to be more agile and adaptive than your systems of record.
At the top are systems of innovation. These support your move into new markets, new channels, taking you to places where your competitors aren’t present. They require the utmost of speed and agility to succeed.
If we bring these two models together, it become easier to see when to use which path to the cloud.
Systems of record are the core of your business, and you would thread them very conservatively. Those systems are likely to remain on-premise, as they are business-critical for your company. Here we speak of the very modified ERP systems you run, or the mainframe that has not been altered in years, yet performs nicely and keeps their base function in good shape. You may try to lift and shift those systems or even replace them one by one by a SaaS software, but due to the criticality of the software, those migrations would be undertaken very carefully and will typically be the last to be performed.
Systems of differentiation and systems of innovation can also be lifted and shifted, but you will benefit most from leveraging all the power of modern PaaS platforms. They provide building blocks for massive data storage, analytics, AI, UI frameworks and much more.
In real life, the migration of your apps will not happen as a big bang and you would have a very diverse landscape of application and hosting types at every given moment. A lot of the systems of records will remain for a long time on-premise, while the business would adopt cool new SaaS apps rapidly for its new needs. And finally - any new systems of innovation would directly be written on the cloud platforms. So, your landscape will be partly on-premise, partly in VM in the cloud, partly in SaaS applications and partly in newly built cloud applications.
The role of the integrations
No matter where your environment is hosted, integration will remain the central piece that ties your enterprise together. The need to integrate has only increased with the proliferation of SaaS applications, IoT sensors and devices, and mobile app development in addition to the vast number of applications of record - mainframes, legacy and custom once.
Your integration software connects your data across all environments, overlapping all the domains and reaching all the endpoints. As your software starts being hosted outside of your data center, the integration starts to span across on-premises ERM, CRM, and mainframe, cloud and SaaS environments, edge devices and sensors and B2B partners. Very soon you’ll feel the need to modernize your integration software for the same reasons you migrated your other solutions – need of agility, robustness, and reduction of the operational costs. As the integration doesn’t bring value on its own, but connects applications, your organization will have to assess where the important data resides – in other words, where the center of gravity regarding your data is.
The importance of your center of gravity
As the key of the integration software is to free your data by connecting and transforming it, your integration workloads need to be closer to where your data volumes are. If most of them are still on-premise in ERPs and mainframes, moving your integration workloads to the cloud will get the typical operational benefits but will also increase the latency between your systems and will make your network topology more complex. On the other side, if your company is already a heavy user of SaaS software, such as Salesforce or Workday, and you leverage data lakes in hyperscalers (like AWS, Azure, or Google) to store all relevant analytic data, then your center of gravity is outside of your own data center. In this case, migration of your integration workloads to the cloud will greatly improve the overall setup and architecture of your software landscape.
Your integration options in a hybrid environment
As the data in the new world can be anywhere – in your data center, a hyperscaler of your choice or in a SaaS-- it becomes obvious that a “one size fits all” monolith integration from the past does not fit every integration workload. Instead you’ll start dispersing the architecture with separate integration flows depending on which applications are to be connected. Here are the available options in this new reality.
Remain on-prem
A lot of organizations originally solved the integration problem by adopting ESBs (enterprise service bus) and integration platforms, and these are still widely in use and in many cases powering mission-critical applications for organizations. If the connected applications are still in your data center, it doesn’t make sense to move your integration workloads elsewhere just for the sake of the operational cost benefits, as the message will go up the cloud just to come back to your data center.
Re-host
If the systems that are being integrated are already moved to the cloud, it makes sense your integration workflows are also moved. The simplest way to achieve this is by re-hosting your integrations as-is. This is a lift and shift of your integration architecture to VMs hosted outside of your data center. While you’ll gain some operational benefits with this approach, none of the cloud scalability and robustness will be available to you, and you’ll still have to operate the software.
Repackage
Another type of re-hosting that can allow you to leverage more of the cloud is the repackaging of the software into containers. The containers are used as a more standardized packaged distribution that contains all needed dependencies and is much more lightweight than a VM. By simply repackaging the software as a container, you won’t get scalability and high availability, but you may leverage the mobility of the container for easier migration – if a container runs in your data center, it will run in the same manner in the cloud.
Repackaging the software is not a trivial task that wraps an installation. It will require complete reshaping of the way you build and operate a solution – from a different CI/CD flow to a new way of monitoring and troubleshooting.
Re-factor
Repackaging in containers gives your software mobility and more confidence by promotion, as all dependencies and tools are promoted with the latest code. However, containers by themselves do not give you a flexible solution that can be upgraded partially with zero downtime, or a quick time to market for new ideas. To achieve those goals, you’ll need to change not only the way your software is packaged, but also your architecture – a monolithic software can’t meet the new requirements, so it needs to be altered to more manageable pieces that can function independently as microservices.
The ESBs are undergoing an evolution since they are considered a monolith, and organizations are seeing a need to deploy lightweight integrations as microservices. This has led to the emergence of a microservices runtime (MSR) that can be scaled very quickly to meet the needs of the business. MSRs are typically deployed using containers, scaled using a container orchestration environment like Kubernetes or OpenShift and subsequently monitored using open-source tools like Prometheus. This architecture steps on the foundations of the repackaging and comes with big requirements for your DevOps practices – you’ll need to fully automate the promotion and to dev/build/test/deploy. Choosing this option also requires specific know-how and good maturity in the CI/CD processes in your team. This approach can be considered the hardest of all options, because of the technology and operation challenges that your organization will face.
Re-platforming
Re-platforming is leveraging the integration platform as a service (iPaaS) for your workloads. This is a pure cloud offering, which easily enables cloud to cloud, or SaaS to SaaS integrations. By using the integration as a SaaS, you can fully forget about the operational aspect of this software layer – the software will be automatically updated and can scale on your demand. It is the natural choice when creating new integration between other cloud native systems or SaaS apps, and it does offer a lot of ready connectors and templates for such applications. If your applications are being moved to the cloud, it makes sense to create new integration flows in the iPaaS to replace your on-premise EBS.
Hybrid
We believe that your applications will never again be hosted in a single data center and you always go for a hybrid approach depending on the software’s criticality or innovation needs. Here, the integration workloads will follow your data architecture and will sideline the applications as close as possible in the different hosting scenarios. Eventually you’ll have a mix of all mentioned integration options working together in a holistic manner.
Where to start with the integration migration
We’ve already covered the drivers for a cloud migration software, where your applications might be moving toward, what roles does the integration play in this setup and what are your options regarding its coexistence within the rest of your landscape. As a reader of this article, you’re probably already into the cloud journey and you wonder where to start regarding the integration transformation. We believe there are several important aspects you need to figure out before you start moving your integration workloads out of your data center. For each of them, you need to answer some key questions described. This is the self-assessment part everybody needs to do before defining concrete action items.
Vision – You need to know what your drivers for the migration are. While this can vary per enterprise, there are several common drivers - Are you looking for cost reduction? More agility under load? Business continuity and robustness when performing maintenance? In each of those look for the long-term benefits and try to align them to your company business goals.
Assessment – What are the timelines you’re hoping to achieve your goals? Have you performed an analysis of your application landscape? Have you figured out your center of gravity?
Plan – A good number of customers jump directly to this phase by just chasing the latest technology trends. Once you have done the vision and the assessment parts, you will have a clear idea of which migration approaches fit certain integration flows best.
Execution – After you have the vision and the plan to achieve it you need to make sure you have the necessary skills and time to drive the transformation forward. There is no perfect migration path and no magic button to just run the software from a cloud with modern architecture. Figure out the best ways to achieve your business goals and define measurable criteria for your success.
Develop your new Integration Strategy
No matter where you move your integration workloads, it is always a good first step to carefully pick an API strategy that will help you easily modernize the landscape. Then, by understanding your “center of gravity” for each integration workload, you can figure out which integration options fit it best. Our recommendation is to start with quick wins – re-hosting as-is is a good option and will allow you to get a feeling of what it means to run in the cloud. You can track running and operational costs as well as monitor for any hidden problems that might get exposed – network topologies, legal aspects concerning your data, knowledge gaps in your organization and many others.
The next easy step can be re-platforming, especially if you start building integrations between brand-new SaaS applications. webMethods.io provides simplicity in building those workflows with good UX and a lot of ready recipes for many modern systems.
Meanwhile your organization can gradually acquire the skills to implement and deploy integration microservices in a container orchestration environment.
As you’ll end up with a lot of different integrations, you’ll need to prepare yourself to govern all those in a scalable manner.
Summary
Migrating an integration platform to a cloud infrastructure is much more challenging than migrating or replacing a single business application. An integration solution never lives a life of its own – it depends on your whole IT landscape. That is why an integration solution migration can rarely be a big bang – it is a rather phased approach when you carefully pick the workloads one by one and find out which approach is best for them. You might leave them where there are, move them to a VM in a cloud, create any brand-new SaaS integration with iPaaS platform like webMethods.io or spawn many new integration microservices that can be run and updated independently. The typical monolith ESB looks might become a rare sight in the next years. Instead the trend would go toward different deployment consuming and hosting APIs everywhere. We believe it would be beneficial to embed this into your vision and make sure you aim for a long-lasting solution.
This article is part of the TECHniques newsletter blog - technical tips and tricks for the Software AG community. Subscribe to receive our quarterly updates or read the latest issue.
#Integration-Server-and-ESB#newsletter#webMethods