This article was originaly published in 2017.08.04
DataPower Plug-in: Definition File Walkthrough
Summary
In our previous blog, DataPower Plug-in: Import (Basic) Walkthrough, we demonstrated the fundamental capabilities of the IBM DataPower plug-in for IBM UrbanCode Deploy, where to initialize various properties, and how to configure and run a basic import DataPower process. With this foundation, we will move onto an advanced tutorial using the Upload from Definition and Import (Definition) plug-in steps and configuration of the DataPower Configuration Manager (DCM) definition file. The definition file, combined with a domain export.zip, provides the configuration changes to every object and properties within an IBM DataPower domain and environment.
This blog post will largely focus around the DCM definitions file and how it can be modified to adjust your deployments. Additional documentation on the definitions file and all of its configuration possibilities can be found on the
GitHub DataPower Wiki. While incredibly powerful, the definitions file uses strict syntax. It is imperative you understand the uses of a definitions file to take full advantage of the DCM tool and plug-in. Beware, not all definition syntax is valid in every scenario. For example, the
dcm:upload element is only available to the
Upload from Definition (upload-from-def Ant target) step. As a general note, the names of the elements often match the plug-in step or Ant task that they are valid in. If a definition is accidentally specified in an incompatible step or target, it will be ignored.
Requirements
Throughout this blog post, we will reference general processes, properties, versions, applications, and components that were defined and outlined in the Import (Basic) article. If this is your first time using the IBM UrbanCode Deploy or the IBM DataPower plug-in, please review and follow our previous blog post. The Import (Basic) Walkthrough gives a thorough explanation of the IBM DataPower plug-in and how to initialize a simple process with the necessary properties and component versions. If you do not have an IBM UrbanCode Deploy or DataPower environment readily available, the other blog post can guide you through the server setup as well.
Some definition files may be associated with a domain export configuration. We will be using the same export.zip that we created and exported from DataPower in the Import (Basic) blog. If you do not have this export.zip, follow the directions there. If the export does not have an objects defined, create one manually before you create the export.
We will be using the two definition.xml files in the walkthrough's examples. Below, I will explain in depth the important parts of each definitions file. If you want to test them, copy and import these files into their own component versions. Example #2 will require the
export.zip to be packaged in the same component version, while Example #1 can be import alone. Ensure the names of the definition files in the component versions are
definition.xml or modify the step property to match. As you may have discovered in the Import (Basic) process, if the export.zips are named differently, your process will not be reusable and will require constant modification. All other step configurations are identical to the Basic Import component process.
Example #1: Upload from Definition
Setup
Using the same
DataPower Demo 1 Component from the Basic Import blog, create a new process called
Upload with Definition. This process will mirror the Basic Import process, but the Save Configuration step does not need to be specified. Unlike the Import (Basic) step, the Upload from Definitions step's changes are saved immediately on import. The screenshots of the overall component process with Upload from Definition step's configuration are below.
IBM UrbanCode Upload from Definition process
Once the component process is created, the matching application process will also need to be built. A single
Import Component step with the specified component process is sufficient for our examples. The application process creation steps were documented in the previous blog.
Definition File
<?xml version="1.0" encoding="UTF-8"?> <dcm:definition xmlns:dcm="urn:datapower:configuration:manager"> <!-- Upload file to DataPower appliance -->
<dcm:upload local="/root/datapower-demo/**.*" remotedir="local:///datapower-upload-demo" /> </dcm:definition> This is one of the simplest definition files you could come across. Combined with the
Upload from Definition step (upload-from-def target), the command will look in the
/root/datapower-demo directory on the agent's runtime file system and look for all files and folders under that location. They will then be uploaded to the DataPower appliance's file system under the
local:datapower-upload-demo folder. In the local /root/datapower-demo folder, I manually created two empty files: file1.txt and file2.txt. You can see the result of the upload in the screenshot below.
DataPower Upload from Definition result
You are not limited to uploading full folders. Specific files can also be uploaded in any DataPower folder location. For example, if you wanted to upload file1.txt into the root directory of the
temporary folder, use the below syntax. Example:
<dcm:upload local="/root/datapower-demo/file1.txt" remotedir="temporary:///" /> The upload element can be repeated any number of times in a single definitions file. You cannot rename the copied file. If you want to give the file a new name, rename it before the Upload from Definition step is run.
Complete documentation on the dcm:upload element is available on the GitHub Project Wiki.
Example #2: Deployment Policy
Setup
Using the same
DataPower Demo 1 Component from the Basic Import blog, create a new process called
Import with Definition (import-from-def target). Follow the same process flow as the
Import Basic however, move the
Save Configuration step to the side. The Save Configuration step is critical in confirming any modifications to DataPower. However, to see the changes in DataPower take place, we will want to view and confirm our changes manually in the DataPower user interface. In this new configuration process configuration, add
Import (Definition) after the Download Artifacts step.
IBM UrbanCode Deploy Import (Definition) process
Import (Definition) step configuration[/caption]
Once the component process is created, the matching application process will also need to be built. A single
Import Component step with the specified component process is sufficient for our examples. The application process creation steps were documented in the previous blog. The only difference between the
Import (Definition) and
Import (Basic) step is the Definition File property. As the file will be part of all new component versions, we reference it the same way as the Import File property.
Definition File
<?xml version="1.0" encoding="UTF-8"?><dcm:definition xmlns:dcm="urn:datapower:configuration:manager"> <dcm:deployment-policy> <!-- Modify Property: Set's LocalAddress of the sample-http-source-protocol object to "localhost" value --> <modify-property name="sample-http-source-protocol" prop-name="LocalAddress" new-prop-value="localhost"/> <!-- Delete Property: Cannot completely delete LocalPort property of the sample-http-source-protocol object. --> <!-- Sets default value to 443 --> <delete-property name="sample-http-source-protocol" prop-name="LocalPort"/> <!-- Black Listist: Do not import the Cryptocert_1 object --> <black-list name="Cryptocert_1"/> <!-- White List: Only import CryptoCertificates and HTTPSSourceProtocolHandler objects. --> <white-list type="crypto/cert"/> <white-list type="protocol/https"/> </dcm:deployment-policy></dcm:definition>The above definitions file shows a few simple example for the deployment-policy configuration. This file is used directly against the imported
export.zip artifact. This means object configurations already in place on the DataPower appliance will
not be changed, removed, or added. Each of these lines are relative to the defined
export.xml in the export.zip file. When determining what object to ignore or resource type to white-list, the export.xml can tell you exactly what string value should be specified. To find this file, simply extract and open the
export.zip folder. We highly recommend you become comfortable with inspecting the
export.xml file. The definitions file expects specific values for its object, property, and resource type names. Quite often, proper creation of the definitions file will confirmation with what the
export.xml file actually contains. Beware, the DCM tool and plug-in will not alert you if a specific object does not exist. We recommend a iterative approach when designing and building the definitions file.
Full documentation on identifying this properties can be found on the GitHub Project Wiki.
Deployment Policy Child Elements
The overall structure of the definitions file should look similar to the Example #1. The main difference is a new
deploy-policy element. All elements underneath this tag define what objects will be imported and how their properties will be reconfigured. Within each child element, its optional property definitions specify what object that particular command will affect. Note, these elements do not have the parent's namespace (e.g. dcm). These command elements include:
add-property: Create an undefined property for an object. Each object class has its own set of expected properties. If an unknown property is specified, then the add-property element will be ignored. When a domain is exported, objects should have all expected properties set with default value. This element is often used when the export.xml has been manually modified.modify-property: Update an object's defined property with a new value. This is one the most common elements and the most functional.delete-property: Remove an object's defined property. In some cases, this element name may be misleading. Many properties cannot be removed, and when this occurs, their values will be updated with the default value.white-list: Specify a range of export.xml object classes to import. This element is useful to trim the export.xml and only import specified objects types.black-list: Specify a range of export.xml objects to ignore during import. The black-list elements take precedence over the white-list elements.
Deployment Policy Child Element Properties
Each element may utilize optional properties to define which objects are affected. An element may contain any permutation of the optional properties and will use the value of * (everything) if left unspecified. While each property does not need to be specified, you will likely use some combination for your final deployment processes. Every element has access to these four optional properties:
name: The name of the object. This is the easiest property to test with as it will affect only one instance of an object. Furthermore, every object has a name, so they are easily identifiable in the export.xml.type: The object's Resource Type ID. (e.g. xml/aaapolicy or crypto/cert)domain: Specify objects that are a part of the chosen domain. (e.g. default)device: Specify the device-name serial to specify objects from a certain DataPower appliance. (e.g. aze3789122yf)
Complete documentation to help identify these optional properties can be found on the GitHub Project Wiki.
Modify and Delete Child Elements
From the definitions file above, the first
<modify-property> element will look for an object named
sample-http-source-protocol and update the
LocalAddress property to the value of
localhost. Comparing the
export.xml and DataPower pending changes screenshots below, you can see that the intended value would have been
0.0.0.0, but was updated on import. The second element,
<delete-property>, is an example instance where an object property cannot fully be deleted. Here I have tried to delete the
LocalPort property of the
sample-http-source-protocol object. However, because it is a required property, it cannot be deleted. Therefore, instead of the
export.xml value of
55555, it is now the default value of
443. Notice that the default value was not specified in the definitions or export.xml files.
Modify and Delete Property export.xml snippet
Modify and Delete Properties in IBM DataPower Gateway
Another important point to highlight is that the property name specified in the definitions file is not the same name in the DataPower configuration. This is another reason why it is good to be familiar with the
export.xml. For example, the
LocalAddress and
LocalPort property IDs are called
IP address and
Port in IBM DataPower's user interface.
IBM DataPower's user interface labels properties differently than the export.xml
White-list and Black-list Sub-Elements
The second set of
black-list and
white-list elements provide an object filtering mechanism. If you have used other IBM UrbanCode Deploy plugins, this capability should feel similar to includes and excludes fields. And like the other add, modify, and delete elements, these elements are also defined by the name, type, device, and domain properties. The
white-list element will specify objects that will be imported. While the
black-list element will prevent any defined objects, within the
white-listed subset, from being imported. If a
white-list element is not specified, all objects will be imported. If the
black-list element is specified without any parameters, no objects will be imported. In this specific example, the two
white-list elements will only allow the import of HTTPSSourceProtocolHandler and CryptoCertificate object classes. Of the identified objects to import, the
black-list element will look for an object named
Cryptocert_1 and prevent the import. In the results below, you can see that multiple
Cryptocert_XX certificates have been added, but not
Cryptocert_1.
CryptoCertificate white-list and black-list DataPower change results
The
type property, used in this example, can be one of the most important arguments, but the hardest to identify.
The documentation on the GitHub Project Wiki can help point you to your intended black or white listed objects.
Wrapping Up
The definitions file is incredibly powerful if configured correctly and a major component in all production integration strategies. I hope this documentation was able to provide further insight into how the definitions file is used and how one defines the deployment policy and upload elements. On the
GitHub Project Wiki, you will find many other elements available to use in a definitions file. The advice described here can be extrapolated for all other DCM elements. If you have a question about this documentation, please create a
GitHub issue.
#DataPower#UrbanCodeDeploy