App Connect

App Connect

Join this online user group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only

Explore the new features in App Connect Enterprise version 11.0.0.6 

Thu July 23, 2020 11:44 AM


We’re thrilled to announce the availability of IBM App Connect Enterprise v11.0.0.6. This is the sixth fix pack released for App Connect Enterprise software. We provide regular fix packs, approximately once per quarter, a cadence which we intend to continue through 2019. Fix packs provide both regular maintenance for the product, and also new functional content. This blog post summarizes all the latest and greatest capabilities:

  • LDAP Authorization
  • Encryption of credentials
  • User-defined Policy redeploy
  • New Policies for PeopleSoft, JDEdwards, and Siebel
  • 48 New Tutorials
  • Publication node enhancement to use MQ Client
  • Command and admin REST API for extracting a monitoring profile
  • Dynamically updating Message Flow User Defined Properties

Expand the sections below to find out more!

LDAP Authorization

Fix pack 6 introduces the ability for App Connect Enterprise to use an LDAP server to control the authorization of users executing administrative actions against an ACE integration node or independent integration server. Fix pack 4 has previously provided for authentication using LDAP, but this latest fix pack builds on top of this to apply authorization queries to the LDAP authenticated users.

Your LDAP users must belong to one or more LDAP groups, or have one or more LDAP attributes that map to roles in App Connect Enterprise, so that ACE can decide on the appropriate access privileges to provide. Roles in App Connect Enterprise have read, write, or execute permissions for objects in integration nodes or independent integration servers (which are not managed by an integration node). You can configure the authorization by setting properties in the node.conf.yaml file for the integration node, or the server.conf.yaml file for the integration server. There are flexible options for how to configure the LDAP authorization:

  • Authorize a single LDAP group to have a role in App Connect Enterprise.
  • Authorize multiple LDAP groups to have roles in App Connect Enterprise.
  • Authorize an LDAP authenticated user’s LDAP attributes to have a role in App Connect Enterprise.

Given the high degree of flexibility in the configuration of LDAP authorization, at first it can be quite confusing to understand all of the available options. There is a Knowledge Center page which goes into the details, but to help describe how things work, below you will find a simple worked example. This example involves an LDAP server which has been configured with four users:

  • Ben Thompson: cn=Ben Thompson,dc=example,dc=com
  • Trevor Dolby: cn=Trevor Dolby,dc=example,dc=com
  • Graham Haxby: cn=Graham Haxby,dc=example,dc=com
  • Martin Boag: cn=Martin Boag,dc=example,dc=com

Each of these user records, has an attribute ou. Ben and Trevor have a value of ou = businessgroup1, where as both Graham and Martin have a value of ou = businessgroup2. Ben’s record and Graham’s record are shown below as examples, with the ou attribute highlighted with a red box:

The screenshot below shows the server.conf.yaml file which is used to configure an independent integration server:

Note the ldap authorization search URL in line 66, which has been set to ldap://ibmexample1:10389/dc=example,dc=com?ou?sub?(cn={{username}}). Breaking this apart into sections, this URL is interpreted as follows:

  • ibmexample1 is the hostname of the LDAP server
  • 10389 is the port number of the LDAP server
  • dc=example,dc=com is the baseDN (Distinguished name)
  • ?ou is the attr-name
  • ?sub indicates a subset of the LDAP tree structure is to be searched
  • cn={{username}} is the filter expression

Lower down the server.conf.yaml file, lines 73 and 74 provide the mapping between the value located from the attribute ou of the logged-in user’s record, and the ACE role. Two roles are defined – viewRole which only has read privileges (line 79) and adminRole which has all privileges (line 80)

The culmination of these settings is such that when Ben or Trevor log in, being members of businessgroup1, they are assigned adminRole’s complete set of all privileges. In contrast, when Graham or Martin log in, being members of businessgroup2, they are assigned viewRole. As a result of these privileges, once logged in, should Ben or Trevor attempt to stop a deployed application, they would see a success message as shown below:

Should Graham or Martin log in, and attempt the same action, they would receive an exception as shown below:

Encryption of credentials

As part of standard operational practice, App Connect Enterprise message flows need to connect and exchange data with secured resources and systems. This fix pack introduces a new feature which makes it easy to encrypt the credentials which are used by independent integration servers for making these connections. The features discussed here are only currently exposed for independent integration servers, but we expect to incorporate integration node owned servers into the same commands and interfaces in future.

Until now, users have run the mqsisetdbparms command to specify credentials. For users who are new to App Connect Enterprise the name of this command might seem confusing. The command dates from a much earlier version of ACE when databases were the main external system that the product required credentials to access. Over time the syntax of this command was then extended for other systems and protocols. Execution of mqsisetdbparms stores the credentials on the runtime system. The files which are used are obfuscated using a private algorithm, but they are not encrypted. This approach doesn’t present any problems, for users implementing heavily secured systems as other steps are available to fully protect access, such as using file-level or volume-level encryption to protect the contents of the product configuration directory. However, over time we expect users will gradually wish to adopt the new capabilities described below which provide the option of full encryption using a built-in vault. Note that there is no mandatory need to move away from mqsisetdbparms, so there is no urgent need for action for users who are not ready to embrace this new feature.

There are two new commands used in this feature area:

  • mqsivault: Before you can store encrypted credentials for an independent integration server, you must configure an App Connect Enterprise vault. You create a separate vault for each integration server, and each vault has its own vault key. You can use the mqsivault command to create or destroy a vault, to change or verify a vault key, or to retrieve credentials from the vault. The vault stores the credentials (in encrypted form), and the independent integration server uses them to access secured resources.
  • mqsicredentials: You can use the new mqsicredentials command to configure an independent integration server to use encrypted credentials for connecting to a secured resource. You can use the mqsicredentials command to create, update, report, and delete credentials for independent integration servers. You can use the command to create and report credentials when the integration server is running or stopped, but you must stop the integration server before you can update or delete credentials.

To get a feel for the new feature, you can try it out using the following steps:

  • Create a working directory for an independent integration server:
    mqsicreateworkdir C:\ExampleWorkDir
  • Create a vault which will hold the encrypted credentials:
    mqsivault --work-dir C:\ExampleWorkDir --create --vault-key myvaultkey
  • Start the independent integration server (remembering to pass in the value of your vault key!):
    IntegrationServer --work-dir C:\ExampleWorkDir --vault-key myvaultkey --console-log --admin-rest-api 7600 --http-port-number 7900
  • Create a credential in the vault:
    mqsicredentials --work-dir C:\ExampleWorkDir --vault-key myvaultkey --create --credential-type smtp --credential-name mysmtpcred --username ben --password mysmtppassword

In the commands above, it is the vault key which grants the command, or the Integration Server, access to the vault. These examples specify the value via the vault-key command line parameter. Alternatively you can specify a vaultrc-location parameter which avoids the user showing their secret vault-key when typing terminal commands. The rc in vaultrc stands for “run command” and can be thought of conceptually in the same way as .bashrc. Another alternative is to use an environment variable named MQSI_VAULT_KEY. Details of these options are provided in the Knowledge Center.

You should then be able to use a web browser to connect to the integration server. Note there is now a new credentials tab:

You can select a credential to see the non-secret parts displayed:

You can then refer to credentials from other areas of the product in the usual way.

User-defined Policy redeploy

Fix pack 6 continues our efforts towards making all policy types redeployable. In this fix pack policies of type User-Defined can now be redeployed, allowing updated values to dynamically take effect without needing to restart the integration server process. With ACEv11.0.0.6, the status of all the policy types is now shown below.

These (19) policies can be redeployed:

  • Activity Log
  • Aggregation
  • Connect:Direct Server
  • CICSConnection
  • Collector
  • CORBA
  • EmailServer
  • FtpServer
  • IMSConnect
  • JDEdwards Connection
  • MQEndpoint
  • PeopleSoft Connection
  • Resequence
  • SAP Connection
  • Siebel Connection
  • SMTP
  • Timer
  • User Defined
  • Workload Management

These (10) policies cannot be redeployed (use the Delete all resources option and then deploy the updated policy):

  • DotNet Application Domain
  • Java Class Loader
  • JDBC Providers
  • JMS Providers
  • MQTTPublish
  • MQTTSubscribe
  • Security Profiles
  • TCPIP Client
  • TCPIP Server
  • WXS Server

New Policies for PeopleSoft, JDEdwards, and Siebel


IBM App Connect Enterprise v11 provides users with a WebSphere Adapter node drawer in the message flow palette of the Toolkit which contains nodes for connecting with the Enterprise Information Systems SAP, Siebel, PeopleSoft and JD Edwards. Although these functional capabilities have been available since v11 was first released in March 2018, the recent fix pack 6 release provides users with new policy types for Siebel, PeopleSoft and JD Edwards. These policies allow you to abstract various connection details away from the message flow itself, making it easier to move and re-use the same message flows across multiple environments (eg Test, QA, Production etc). The example for Siebel shown in the picture includes a Connection URL and Idle Timeout properties. Connection details can be provided to the runtime either from an adapter component or with this enhancement from a policy. For convenience, these new policy types for Siebel, PeopleSoft and JD Edwards can all be redeployed dynamically, which is an advantage over IBM Integration Bus v10 where to pick up the new values from the equivalent configurable service definitions, the integration server required a restart.

48 New Tutorials

Since the last ACE fix pack was released three months ago, we have been busy creating a large number of new product tutorials. Tutorials are useful for teaching some basics to those who are new to the product, but they can also be helpful for more experienced users by providing some reusable examples for demonstrating product concepts. All the product’s tutorials (71 currently, but growing all the time!) are available on public Github within the OT4I (Open Technologies for Integration) organisation. However, the best way to access the ACE tutorials is from a machine with an internet connection and the ACE Toolkit installed. Go to the Help menu and choose the Tutorials Gallery option and the tutorial navigator will be opened:

If you don’t have the internet available on your Toolkit machine, or if you’d like to checkout the tutorials before installing the product, then you will find links to browse the instructions and download all the tutorial artifacts from this handy web page. You will also find a set of instructions for manually downloading the tutorials on this page.

Publication node enhancement to use MQ Client

The Publication message flow node has been enhanced in this fix pack to to utilise an MQ Client connection and enable messages to be published to a queue on a remote MQ queue manager. The MQ connection properties which are configured on the node itself can also be overridden at run time using an MQEndpoint policy. The screenshot below shows the new MQ Connection tab of the node’s properties:

Alternatively, flow developers can assign the name of an MQEndpoint Policy (qualified by a Policy Project) on the publication node’s Policy tab:


When the message flow is deployed as shown in the picture above, the connection information defined in the policy can be used to provide or override the MQ connection settings.

Command and admin REST API for extracting a monitoring profile

Fix pack 6 introduces a new option on the mqsireportflowmonitoring command for extracting a monitoring profile which reflects the monitoring settings on a currently deployed message flow. Monitoring profiles are XML files which list the event sources in the message flow, and define the properties of each event. A monitoring profile can be dynamically applied after a message flow has been deployed to override its current monitoring settings. Consider a very simple message flow example, which has a monitoring event defined on its HTTP Input node as shown:

Once deployed, the new option on the mqsireportflowmonitoring command can be used as shown in the screenshot of the ACE command console below:

The extracted monitoring profile in this example is shown below:

ACE users might use this kind of approach as a starting point to then further edit the profile to include configuration for the precise style of monitoring required.

The new option which we’ve been exploring with the mqsireportflowmonitoring command, is underpinned by a new method in the product’s administrative REST API:

  • HTTP POST to /apiv2/servers/{server}/applications/{application}/messageflows/{messageflow}/extract-monitoring-profile

Dynamically updating Message Flow User Defined Properties

A user-defined property (or UDP for short) is a property that is defined on a message flow in the Toolkit. This property can be referred to programnatically within the logic of the flow to drive its behaviour. There are three options for accessing the value of a UDP at runtime: Compute node ESQL, java in a JavaCompute node, or a graphical data map. From Fix pack 6 of App Connect Enterprise, you can now change the behaviour of your flows at runtime by altering the value of a message flow UDP using a method provided by the administrative REST API. Consider a simple example message flow which has a UDP named MyUserDefinedProperty1. The two screenshots below demonstrate how the UDP has been defined with a default value of “Default”, and how the current assigned value before deploying the flow is the value Ben:

This example message flow uses a simple ESQL instruction to assign the current value of the UDP to an output JSON message which is sent as an HTTP reply from the message flow:

Once this message flow is deployed, you can query the current value of the UDP, and also apply a new value for the UDP, using a simple HTTP client program such as Curl:

For more information about the administrative REST API methods, you can explore the Open API specification document which is served on the URL /apidocs from all integration nodes and standalone integration servers. Two methods are provided:

  • HTTP POST to /apiv2/servers/{server}/applications/{application}/messageflows/{messageflow}/apply-global-udp-override?name=NAME&value=VALUE&type=TYPE
  • HTTP POST to /apiv2/servers/{server}/applications/{application}/messageflows/{messageflow}/remove-global-udp-override?name=NAME

9 comments on"Explore the new features in App Connect Enterprise version 11.0.0.6"

  1. Diganta Pati November 18, 2019

    Thanks Ben. I do understand message flow is not supported inside shared library. My initial question is ” how can we override an udp that is present in a subflow inside a shared library ? Do you have the REST API for that ? “

    Reply (Edit)
    • BenThompsonIBM November 19, 2019

      Hi Diganta,

      It’s worth reading up on the background in this pae in the Knowledge Center:
      https://www.ibm.com/support/knowledgecenter/en/SSTTDS_11.0.0/com.ibm.etools.mft.doc/ac00643_.htm#ac00643_

      In particular when considering subflows …

      To have a value that is available to both the main and subflow, define UDPs in one of the following ways:

      Define UDPs in the subflow and then promote them to the main flow. Set the value in the main flow.
      Define UDPs in the main flow only and allow the subflow to locate the UDP value in the main flow at run time.

      The REST APIs described in the article above should let you set the UDPs on the parent flow, which will then be available to the subflow.

      Cheers,
      Ben

      Reply (Edit)
  2. AlexPalamar November 17, 2019

    HI. Thanks for new features! I would like to ask, is that possible to work with ACE vault directly from the Java compute node (get credentials)?

    Reply (Edit)
    • BenThompsonIBM November 18, 2019

      Hi Alex,

      We have not yet enabled a public API (either REST or Java) for retrieving credentials from a vault, but this is absolutely something we have discussed with our users on our Early Experience Program and something we are keen to offer in a future fix pack.

      Cheers,
      Ben

      Reply (Edit)
  3. Teresa Lam October 16, 2019

    Will we put back Business Transaction Monitoring into any fixpak to come?
    I suppose messageflow nodes can still emit events, it’s just the database schema and UI are missing without BTM? Can I reuse v10’s BTM on v11 events?

    Thanks!

    Reply (Edit)
  4. Diganta Pati October 15, 2019

    I see there is API to apply global UDP override for a flow that is inside a library. This is true for a flow inside a static library.
    “http://localhost:4420/apiv2/servers/{server}/applications/{application}/libraries/{library}/messageflows/{messageflow}/apply-global-udp-override”

    But how can we override the UDPs for the flows inside a shared library ?

    Reply (Edit)
    • BenThompsonIBM October 25, 2019

      Hi Diganta, message flows are not one of the supported artifacts for deployment in a shared library, so the product does not have the concept of needing to override the UDP for a flow inside a shared library. Cheers, Ben

      Reply (Edit)
  5. Adam Tz October 10, 2019

    Thanks for the info. On the final new feature its clear you are changing the UDP dynamically but how would you check this value in a real environment? The web console shows the default value as does the REST API regardless of the override.

    Reply (Edit)
    • BenThompsonIBM October 11, 2019

      Hi,

      As part of ACEv11.0.0.6 we have enhanced the REST API response “active” section returned from a GET against a message flow.
      So you can execute this kind of request: curl -X GET http://localhost:4432/apiv2/servers/default/applications/Example_UDPs/messageflows/Example_UDPs
      Within the reply you will find the active section now has an object called userDefinedProperties which looks like this …

      "active": {
      "injectionMode": false,
      "isRunning": true,
      "monitoring": "inactive",
      "monitoringProfile": "",
      "resourceState": "stateStarted",
      "startupEpoch": 1570782465,
      "startupTime": "2019-10-11T08:27:45Z",
      "state": "started",
      "testRecordMode": false,
      "threadInstancesStartImmediately": false,
      "threads": 1,
      "threadsCapacity": 1,
      "threadsInUse": 1,
      "userDefinedProperties": {
      "local": {
      "MyUserDefinedProperty1": "Joe"
      },
      "subflows": {},
      "globalOverrides": {
      "MyUserDefinedProperty1": "Joe"
      },
      "all": {
      "MyUserDefinedProperty1": "Joe"
      }
      }
      },

      We are yet to fully document all of these subsections, but to satisfy your curiosity …

      local: Lists the UDPs available at the flow level and the values here factor in if an override has been applied
      subflows: The collective list of UDPs within all subflows being used by the message flow.
      globalOverrides: The list of global UDP overrides that have been created using apply-global-UDP-override
      all: A complete list of all UDPs that the flow knows about (ie a complete list of “local” + “subflows” + “globalOverrides”) This has been done for convenience to make it easier for consumers of REST responses to list active UDPs. Our mqsilist command now uses the content from this “all” section.

      Hope this helps!

      Cheers,
      Ben

      Reply (Edit)

#AppConnectEnterprise(ACE)
#ACEV11
#Fixpackv11.0.0.6

Statistics
0 Favorited
41 Views
0 Files
0 Shares
0 Downloads
Global message icon