The diagram below outlines a possible domain deployment architecture for IBM DataPower and UrbanCode Deploy. In this simple example, UrbanCode Deploy pulls artifacts directly from a source control manager (SCM) and creates a component version. UCD will then deploy to a DataPower appliance using an appropriate agent pool. Agents cannot be installed on a DataPower appliance, and therefore, must be installed on separate servers. We recommended having at least one agent for each DataPower appliance. However, the required amount will depend on your usage and deployment schedule. To help compute your agent needs, a full DataPower domain import takes approximately ten minutes.
Artifact Management
The DataPower plug-in for IBM UrbanCode Deploy requires two different artifacts for many of its steps:
- DataPower Export Configuration ZIP: Exported directly from IBM DataPower and contains all information about the domain or appliance. Different plug-in steps may take different configuration export types. We recommend for domain deployments, a full export containing all classes, objects, and files is saved as a component version.
- Definition File: This file is a DataPower Configuration Manager (DCM) specific artifact. It allows for targeted modifications of the domain import. Full documentation can be found on the DataPower wiki page.
Most domain deployments will only require a DataPower export configuration. However, more advanced configuration may utilize a definitions file. We suggest pairing every definitions file with an export configuration file in a single component version. This allows for a recreation of the environment as necessary.
Unlike other tools, IBM DataPower does not offer a simplified and automated way to export configurations. Some amount of personalized scripting will be required to pull export configurations and place them into source control. Typically, this is done with a build tool such as IBM UrbanCode Build or hosted in a SCM. In some customer scenarios, the export process is completed by the DataPower developer and placed in a SCM manually.
File System Approach
If you expect your users to initiate various deployments, one suggested approach is using a simple Git or File System (Versioned) approach. The File System (Versioned) source configuration plug-in will monitor a specified folder location for new versions and then pull them into IBM UrbanCode Deploy as component versions. This allows for an on-demand component creation approach with minimal setup. In addition, in an environment where many definition files are being written, this may be the easiest way to test various deployments.
UrbanCode Deploy Approach
If you would prefer a more automated approach with the help of UrbanCode Deploy, we have written documentation around creating version without a build or CI tool. In this documentation, we explain how to import the DataPower export zip as a component version and then immediately initiate a deployment. Because this method is a generic process within UCD, a recurring process initiation may be scheduled to import domain configurations at any interval. This same process can be applied to creating new versions in a SCM or other tools. However, we recommend you do not use UrbanCode Deploy’s CodeStation as your only SCM. All files should be replicated in a standalone SCM server, such as Artifactory or Nexus.
Mapping UrbanCode Deploy to DataPower objects
When moving DataPower server, authentication, and domain information into IBM UrbanCode Deploy, it is important that these values and objects are organizationally correct. DataPower configuration can be very challenging, but IBM UrbanCode Deploy offers many ways to ease this transition and make it seamless. If configured correctly, all deployment values should be handled inherently by the UCD relationships.
The first relationship to note is that one UCD component should match one DataPower domain. Therefore, all configuration specific information should be contained within the component properties. This statement assumes that a copy of that domain should exist in each of your DataPower appliances. For example, if you had three DataPower appliances, such as in the diagram above, the same domain would exist in each. Then, each of these DataPower appliances would be replicated as an environment in UrbanCode Deploy. All configuration specific to that environment, such as the hostname, port, user, and password, would be located as resource properties. As these environments would be shared across multiple applications, they need to set in the resource tree, rather than the application environments themselves. As for the UrbanCode Deploy applications, they can be any combination of components, or domains, that makes sense. Typically, multiple domains build up into a single larger project. That project object would manifest as an application. The below screenshot depicts how the component and environment relationship may look in a resource tree.
Sample SIT, UAT, and PROD DataPower resource tree
Component Management
Manually creating a new domain component with its configuration properties can be very time consuming. Fortunately, this process can be streamlined and automated with the use of component templates. For example, you may want to have a generic process to create a new component. Using additional supported steps from the Component plug-in, one can create a process that creates a new component, creates the domain in each DataPower appliance, and then attaches them to an application. Furthermore, if using a Component Template it greatly simplifies the resource tree management. For example, if all DataPower components made from the template were labeled as DataPower, the new resource tree would look like the below screenshot. All new components would be instantly added to the resource tree with access to their associated agent pool and server properties.
Simple resource tree using component tags
Domain Deployment Suggestions
The DataPower Configuration Manager (DCM) tool, which the DataPower plug-in utilizes, is highly configurable. The DCM allows any type of modification from a domain level to a single object property. Furthermore, any personalized SOMA or Shell scripts can easily be brought over to UCD either through a DCM command or the Shell script. However, many adopters using UrbanCode Deploy in their IBM DataPower domain deployments needed to modify their original procedures.
Ideal Golden Artifact Strategy
UrbanCode Deploy’s general approach to deployments is to have “golden” artifact that are deployed sequentially to every environment once the appropriate tests and validations are verified. In the instance of DataPower, a golden artifact would take the form of a domain configuration export that was developed in a sandbox environment. This artifact would then be either imported manually or automatically into UrbanCode Deploy. Once available as a component version, it may be deployed to the SIT environment automatically. In the SIT environment, automated tests may be performed. Once cleared, the same golden artifact that was imported into the SIT environment would be promoted to the UAT environment. This same progression would then occur for the final PROD environment. Coupled with gated environments, we can ensure the same process and artifacts are properly verified to eliminate any accidental deployment errors.
Updating Domains
The above approach ensures there are no intermediary domains or non-reproducible environments. Deploying artifacts in a uniform manner provides complete knowledge overall all domains and environments. In some manually configured DataPower environments, administrators may make occasional unique changes to a specific domain. This smaller or specific detailed modification is typically achieved through the definitions file. However, this methodology leaves each environment’s domain in a different state with no clear indicator of readiness for a promoted environment. In addition, there is no clear way to recreate these domains without first recreating the baseline and then rerunning every previous deployment. While it is possible to make these small domain changes, we discourage such deployment styles as it increases complexity and decreases visibility. IBM DataPower and UrbanCode regard complete “golden” artifact domain deployments as the optimal DevOps strategy.
Implementation
In typical use cases, DataPower developers will first configure the domain in their development environment. When ready, they will export the complete domain zip and place it in source control. Once in source control, UrbanCode Deploy will grab the zip file, create a component version, and deploy it to the SIT DataPower appliance. Then, once the tests have passed, their administrator can confirm its deployment to the UAT or PROD DataPower appliances. The deployment process will be identical to the process used in the TEST environment however, its properties (server, port, credentials, etc) will be specific to the UAT or PROD appliances. This sequential progression lends itself well to a self-service approach. If a DataPower developer has a new update, instruct them to place the complete export in the SCM’s expected location and IBM UrbanCode Deploy can handle the rest. Combined with proper role configuration in UCD, the administrator can give users a small set of deploy permissions.
Example
A DEV, SIT, UAT, and PROD deployment progression scenario may function as follows:
- DEV: This appliance is the DataPower developer’s environment which is not recorded in UrbanCode Deploy. It is used to create the domain exports which are then placed into the SCM or Build Tool.
- SIT: The SCM or Build Tool then automatically initiates the deployment to the SIT environment. Component versions can only be created in UrbanCode Deploy through this tool. Thus, a DataPower developer cannot manually place an export in UrbanCode Deploy.
- UAT: If the domain export passed SIT deployment and testing, UrbanCode Deploy may apply a version status on the component version. This will then permit a non-administrator to confirm the deployment into a UAT appliance.
- PROD: Finally, if it has cleared UAT deployment and testing, the administrator may permit its deployment to the PROD appliance. This allows the administrator to focus on a subset of deployments and versions nearing production.