XML Management Interface

• SOAP management URI
• SOAP configuration management
• SOAP configuration management (v2004)
• WS-Management endpoint
• WSDM endpoint
• AMP endpoint
• SLM endpoint
• UDDI subscription (deprecated)
• WSRR subscription

1. Login as an administrator on the DataPower default domain.
2. Choose Network -> Management -> XML Management interface. A configuration window will open as show below:

Figure 1   XML Management Interface

3. Set all the details such as Local Address, Port Number on which XML Management Interface will run. To use local host alias instead of static IP address, click Select Alias. A host alias resolves a locally configured alias to a static IP address. Aliasing can help to move configurations among DataPower Gateway instances
4. Setup Access Control list to prevent unauthorised access to XML Management Interface.

          For more details on the ACL setup please refer to:         
          https://www.ibm.com/support/knowledgecenter/en/SS9H2Y_7.7.0/com.ibm.dp.doc/acl.html

5. From Enabled services list, activate the check box for SOAP management endpoint.
6. Apply changes and save configuration.

        For more details please refer to:                                         
        https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.7.0/com.ibm.dp.doc/xmi_interfaceservices_enabling.html

After the XML Management Interface is up and running, you can send the request to configure and manage the DataPower® Gateway using SOMA endpoint service.

SOAP configuration management (SOMA)

The general format of the request is as shown below. If you wish to specify the domain where the action will run, you can set the domain attribute as shown in the example below.

<?xml version="1.0" encoding="UTF-8"?>

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">

    <env:Body>

        <dp:request domain="{domain}"

            xmlns:dp="http://www.datapower.com/schemas/management">

                   ……

        </dp:request>

    </env:Body>

</env:Envelope>

For example, to construct a request for the create configuration operation, first find the operation defined in the file store:/// xml-mgmt-ops.xsd which is as below:

<!-- create configuration -->

   
<xsd:element name="set-config" type="tns:AnyConfigElement"/>

<?xml version="1.0" encoding="UTF-8"?>

<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">

  <env:Body>

      <dp:response xmlns:dp="http://www.datapower.com/schemas/management">

      <dp:timestamp>timestamp</dp:timestamp>

      ....

    </dp:response>

  </env:Body>

</env:Envelope>

Refer to the store:///xml-mgmt-ops.xsd schema file for all the supported operations.

Generally, an organization uses the DataPower device with multiple domains as per the business requirement. Each domain needs to be configured for various application specific settings such as crypto certificates, crypto keys, password aliases, shared secret keys, network settings, domain settings etc.

There are various ways to manage configuration objects on the DataPower domain which are describe in below sections:

• DataPower® Gateway extension: 

<CryptoCertificate name="api-msg-sign"   xmlns:env="http://www.w3.org/2003/05/soap-envelope"

      xmlns:dp="http://www.datapower.com/schemas/management">

       <mAdminState>enabled</mAdminState>

      <Filename>sharedcert:///api-msg-sign-public.cer</Filename>

      <PasswordAlias>off</PasswordAlias>

</CryptoCertificate>

<CryptoKey name="api-msg-sign"    xmlns:env="http://www.w3.org/2003/05/soap-envelope"

      xmlns:dp="http://www.datapower.com/schemas/management">

        <mAdminState>enabled</mAdminState>

        <Filename>sharedcert:///api-msg-sign-private.pem</Filename>

        <PasswordAlias>off</PasswordAlias>

 </CryptoKey>

• SOMA based Scripts: 

• DP Configurator (Automated Utility using JSON configuration & DevOps pipeline): 

DP Configurator

1. Crypto Certificates
2. Crypto Keys and Crypto Keys with Password Alias
3. Crypto Shared Secret Key
4. Password Map Alias
5. LunaPartition
6. LunaHAGroup

The concepts discussed in this article can be easily extended to include other configuration objects too.

Use Cases

There are various scenarios where DP Configurator, which are as follows:

1. Setup of a new device
2. Configuration management of devices
3. Recreation of setup after firmware upgrade
4. After gateway resynchronization

Design

Figure 2 illustrates the overview of DP Configurator along with the automated deployment mechanism.

•  dp-configurator repository ¹   which has the implementation code for creating and managing configuration objects provided in dp-config.
• dp-config repository ² which has the configuration files.

Jenkins pipeline job ³ could be used to build the DP configurator artifact. It checks out dp-configurator and dp-config repository, builds the artifact, runs unit test and on success, pushes the artifact to the Artifact repository ⁴ (e.g. Nexus repository). The artifact will be pulled by the Jenkins job from the repository while creating the UCD deployment snapshot.
IBM Urban Code Deploy (UCD) job ⁵ can be used to run the DP configurator against the IBM DataPower Gateway ⁶. UCD application’s resource properties will be used to keep the DataPower gateway connection information such as DataPower host name, port, domain name and user credentials.

The artifact is run and tested in Dev environment first. On successful testing, it is further promoted to higher environment till Production following all the required processes and approval mechanism recommended by an organization.During the deployment process the UCD deployment job gets triggered with the deployment snapshot. The job extracts the DP Configurator artifact from the deployment snapshot. During the job execution process, DP Configurator constructs the SOMA request calls with payload for each configuration object specified in the DP Config and executes the call for the specified DataPower® Gateway domain.

DP Configurator

The DP Configurator has the implementation code which uses configurations provided in dp-config, constructs the SOMA request calls for each of the configuration object and executes on the specified DataPower® Gateway domain. The following Figure 3 shows the repository structure used for the DP configurator.

   Figure 3   DP Configurator repository structure

Jenkinsfile:  The Jenkinsfile contains the DevOps pipeline code.

createConfig.js/deleteConfig.js: This has the implementation to retrieve/read specified config json, formulate and execute the corresponding SOMA request calls to create, modify or delete the specified config objects.

somaRequest.js: This has the utility code to executes SOMA request calls for the specified DataPower device.

src/template_xml: This directory contains template xml’s for SOMA requests of configuration objects such as creating/updating/deleting Crypto certs, Crypto keys, password map alias, shared secret key, luna partition, luna ha group etc. The view of template_xml directory with sample template xml’s is shown in below Figure 3-1:

         Figure 3-1   template_xml directory

• dp-getConfig.xml

This template is used to create SOMA request for retrieving existing configuration at the specified DataPower domain.

<?xml version="1.0" encoding="UTF-8"?>

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"

  xmlns:dp="http://www.datapower.com/schemas/management">

  <soapenv:Body>

    <dp:request domain="{{domain}}">

      <dp:get-config/>

    </dp:request>

  </soapenv:Body>

</soapenv:Envelope>

Figure 3-2   dp-getConfig.xml

• dp-config-certs.xml

This template is used to create SOMA request for creating or updating specified crypto certificate object on the specified DataPower domain.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"

  xmlns:dp="http://www.datapower.com/schemas/management">

  <soapenv:Header/>

  <soapenv:Body>

    <dp:request domain="{{domain}}">

      <dp:set-config>

        <CryptoCertificate name="{{name}}">

          <mAdminState>enabled</mAdminState>

          <Filename>{{file}}</Filename>

          <PasswordAlias>off</PasswordAlias>

          <IgnoreExpiration>off</IgnoreExpiration>

        </CryptoCertificate>

      </dp:set-config>

    </dp:request>

  </soapenv:Body>

</soapenv:Envelope>

Figure 3-3   dp-config-certs.xml

• dp-config-keys.xml

<soapenv:Envelope

   xmlns:soapenv="http://www.w3.org/2001/12/soap-envelope"

   xmlns:dp="http://www.datapower.com/schemas/management">

   <soapenv:Body>

      <dp:request domain="{{domain}}">

         <dp:set-config>

            <CryptoKey name="{{name}}">

               <mAdminState>enabled</mAdminState>

               <Filename>{{file}}</Filename>

               <PasswordAlias>off</PasswordAlias>

            </CryptoKey>

         </dp:set-config>

      </dp:request>

   </soapenv:Body>

</soapenv:Envelope>

Figure 3-4   dp-config-keys.xml

• dp-config-luna-partition.xml

This template is used to create SOMA request for creating or updating specified luna HSM partition object on the specified DataPower domain. Refer to the page for more details on Luna HSM: https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.7.0/com.ibm.dp.doc/lunahsm.html

<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2001/12/soap-envelope"

    xmlns:dp="http://www.datapower.com/schemas/management">

    <soapenv:Body>

        <dp:request domain="{{domain}}">

            <dp:set-config>

                <LunaPartition name="{{name}}">

                    <mAdminState>enabled</mAdminState>

                    <UserSummary> {{summary}} </UserSummary>

                    <PartitionName> {{partitionName}} </PartitionName>

                    <PartitionSerial> {{serial}} </PartitionSerial>

                    <PasswordAlias class="PasswordAlias"> {{aliasName}} </PasswordAlias>

                    <LoginRole> {{role}} </LoginRole>

                </LunaPartition>

            </dp:set-config>

        </dp:request>

    </soapenv:Body>

</soapenv:Envelope>

Figure 3-5   dp-config-luna-partition.xml 

• dp-config-luna-ha-group.xml

This template is used to create SOMA request for creating or updating specified luna HSM HA Group object on the specified DataPower domain. Refer to the page for more details on Luna HSM: https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.7.0/com.ibm.dp.doc/lunahsm.html

<soapenv:Envelope

    xmlns:soapenv="http://www.w3.org/2001/12/soap-envelope"

    xmlns:dp="http://www.datapower.com/schemas/management">

    <soapenv:Body>

        <dp:request domain="{{domain}}">

            <dp:set-config>

                <LunaHAGroup name="{{name}}">

                    <mAdminState>enabled</mAdminState>

                    <UserSummary>"{{summary}}"</UserSummary>

                    <GroupName>"{{groupName}}" </GroupName>

                    <Member class="LunaPartition"> "{{partition1}}" </Member>

                    <Member class="LunaPartition"> "{{ partition2}}"</Member>

                </LunaHAGroup>

            </dp:set-config>

        </dp:request>

    </soapenv:Body>

</soapenv:Envelope>

Figure 3-6   dp-config-luna-ha-group.xml

• dp-config-certs-del.xml

This template is used to create SOMA request for deleting specified crypto certificate object on the specified DataPower domain.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">

    <soapenv:Body>

        <dp:request domain="{{domain}}" xmlns:dp="http://www.datapower.com/schemas/management">

            <dp:del-config>

                <CryptoCertificate name="{{name}}"/>

            </dp:del-config>

        </dp:request>

    </soapenv:Body>

</soapenv:Envelope>

Figure 3-7   dp-config-certs-del.xml

• dp-config-keys-del.xml

This template is used to create SOMA request for deleting specified crypto key object on the specified DataPower domain.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">

    <soapenv:Body>

        <dp:request domain="{{domain}}" xmlns:dp="http://www.datapower.com/schemas/management">

            <dp:del-config>

                <CryptoKey name="{{name}}"/>

            </dp:del-config>

        </dp:request>

    </soapenv:Body>

</soapenv:Envelope>

Figure 3-8   dp-config-keys-del.xml

The following node js files under src directory describes implementations for creating and executing the SOMA calls.

• somaRequest.js

async function callSomaRequest(payload, options) {

  let payload = payload.replace('{{domain}}', domain);

  return new Promise((resolve, reject) => {

    let req = https.request(options, function (res) {

      if (!res.statusCode || res.statusCode !== 200) {

        console.error(res.headers);

        reject(new Error('SOMA Request failed, response code: ' + res.statusCode));

      }

    let output = {

        code: res.statuusCode,

        headers: res.headers,

        body: ''

      };

      res.setEncoding('utf8');

      res.on('data', (chunk) => {

        output.body += chunk;

      });

     res.on('end', () => {

        if (!output.body.match(/<dp:result>[\s\n]*?OK[\s\n]*?<\/dp:result>/g)) {

          reject(new Error('SOMA Request failed, response: ' + output.body));

        }

        resolve(output);

      });

    });

    req.on('error', (e) => {

       reject(new Error(`Problem with request: ${e.message}`));

    });

    req.end(payload);

  })};

Figure 3-9   somaRequest.js

• createConfig.js

This is the main implementation file with the functions for creating configuration. It reads the resource properties from UCD jobs as shown below.

const somaEndpoint = '/service/mgmt/current';

 

const host = process.env.DP_HOSTNAME;

const port = process.env.DP_PORT;

const user = process.env.DP_USER;

const password = process.env.DP_PASSWORD;

const domain = process.env.DP_DOMAIN;

It has various utility functions such as to retrieve existing configuration from the specified DataPower domain, read the new configuration provided using dp-config. The new provided configuration could be verified and matched against the existing configuration while constructing SOMA calls.

async function getConfig() {

  console.log('Getting DP Configurations');

  let payload = fs.readFileSync('build/template_xml/getConfig.xml', 'utf8');

  payload.replace('{{domain}}', domain);

  return await callSomaRequest(payload, somaEndpoint);

}

To read the specified configuration file following function is used.

function processCreateConfigObject(configFile, somaRequestTemplate, existingObjects, jsonObjKey) {

  let configArray = [];

  try {

    let config = JSON.parse(fs.readFileSync(configFile, 'utf8'));

    configArray = config['' + jsonObjKey];

  } catch (err) {

    console.error(err)

  }

  return callCreateConfigObjects(somaRequestTemplate, configArray, existingObjects, jsonObjKey);

}

To create or update config objects the utility functions as shown below could be used, which helps to create or update the config object provided in json configurations.

async function callCreateConfigObjects(somaRequestTemplate, configJSONArray, existingObjects, objectType) {

  let existingConfigNames = Object.keys(existingObjects);

  return await Promise.all(configJSONArray.map(async (conf) => {

    console.log('processing configuration: ', JSON.stringify(conf));

 

    let overwrite = JSON.parse(conf.overwrite);

    let confName = conf.name.toLowerCase();

 

    if (((!overwrite && !existingConfigNames.includes(confName)) ||

      (overwrite && existingConfigNames.includes(confName)))) {

      console.log('Adding config object: ' + conf);

      return await createObject(somaRequestXML, conf, objectType);

    } else if (existingConfigNames.includes(confName) && !overwrite) {

      console.log('The config object Already exist:' + conf);

    } else {

      console.log('Wrong configuration for the object:' + conf.;

    }

  }));

}

 

Depending on the required object type various utility functions could be defined to create or update the configuration objects, for example, the below utility function is used for creating or updating crypto objects such as crypto certificates, crypto keys, crypto password alias map etc.

async function createCryptoObject(somaTemplate, config, isPasswordAlias) {

  let payload = getBasePayload(somaTemplate);

 

  payload = payload.replace('{{name}}', config.name.trim()).replace('{{file}}', config.filePath.trim());

  if (isPasswordAlias) {

    payload = payload.replace('{{aliasname}}', config.passwordAlias.trim());

  }

  return await callSomaRequest(payload, options);

} 

The following Utility method is used to save the domain after processing all the SOMA calls.

async function save() {

    let payload = fs.readFileSync('build/template_xml/savedomain.xml', 'utf8');

  payload.replace('{{domain}}', domain);

  return await callSomaRequest(payload, options);

}

  }));

}

•  deleteConfig.js

async function callRemoveObjectByName(somaRequestTemplate, configJSONArray, existingObjects) {

 

  return await Promise.all(configJSONArray.map(async (conf) => {

    console.log('processing conf for deletion:' + JSON.stringify(conf));

    if (existingObjects.includes(conf.name)) {

      console.log('Delete config object:' + conf.name);

      return await deleteObject(somaRequestTemplate, conf.name);

    } else {

      console.log('The configuration object does not exist:' + conf.name);

    }

  }));

}

 

async function deleteObject(somaRequestTemplate, name) {

  let payload = getBasePayload(somaTemplate);

  payload = payload.replace('{{name}}', name.trim());

  return await callSomaRequest(payload, options);

}

The main function of the DP configurator is to process all the specified config objects using all these utility methods and relevant template xml’s to create, update or delete the config objects on the DataPower gateway domain.

DP Config

This repository is used to maintain all the configuration object’s files. This could be segregated by environment as shown below.

            Figure 4   dp-config

Configurations are categorized into multiple files depending on the type of config object, for example, dp-config-certs.json has all the definitions for crypto certificate objects.

All files are in json format here.  

Sample configuration files are described below in detail.

• dp-config-certs.json

This file has all the definitions for crypto objects corresponding to crypto certificates.

Example configuration:

{

    "certs": [

         {

            "name": "access-token-sign-object",

            "filePath": "sharedcert:///access_token_sign_object.cer",

            "overwrite": false

        },

        {

            "name": "msg_sign_object",

            "filePath": "sharedcert:///msg_sign_object_public.cer",

            "overwrite": false

        }

    ]

}

Figure 4-1   dp-config-certs.json

Configuration Fields:

• dp-config-keys.json

This file has all the definitions for crypto objects corresponding to crypto keys.

Example configuration:

{

    "keys": [

        {

            "name": "access-token-sign-object",

            "filePath": "sharedcert:///access-token-sign-object-private.pem",

            "overwrite": false

        },

        {

            "name": "msg-sign-object",

            "filePath": "sharedcert:///msg-sign-object-private.pem",

            "overwrite": false

        }

    ]

}

Figure 4-2   dp-config-keys.json

Configuration Fields

Conclusion

References

https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.7.0/com.ibm.dp.doc/welcome.html 

http://www.redbooks.ibm.com/redpapers/pdfs/redp4327.pdf

https://www.redbooks.ibm.com/redbooks/pdfs/sg247901.pdf

https://www.redbooks.ibm.com/redpapers/pdfs/redp4446.pdf

https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.7.0/com.ibm.dp.doc/xmi_soapinterface.html

https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.7.0/com.ibm.dp.doc/networkaccess_xmi.html

https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.7.0/com.ibm.dp.doc/lunahsm.html

Authors & Acknowledgement

This article is the outcome of problems and challenges we recently encounter while working with the customer. 
The authors @RAVINDRA BAVISKAR and @Shishir Narain feels that this would be useful for others in the technical community.

We would like to express our gratitude to @Aiden Gallagher for reviewing this article.