Connecting Integration Server to Azure Service Bus

 View Only
Tue February 07, 2017 09:07 AM

Preface/Disclaimer

This document describes the required steps to create a JMS Connection between an IntegrationServer and an Azure Service Bus. The described installation was part of a technical proof and might not be configured “production ready”.

Overview

The Azure Service Bus (SB) talks (at least also) AMQP (Advanced Message Queuing Protocol), IS on the other side talks JMS. The goal is to still connect both without the need to write separate software to read from SB and write to UniversalMessaging (UM) to provide messages to IS (and the other way around).

This is possible with using Apache QPID. This is an Open Source Library that is basically an AMQP client, but can also be accessed with the JMS protocol, so it can bridge the gap between IS and SB. With using QPID the Service Bus can be configured in IS just like any other JMS Provider. This enables Developers to create JMS Triggers and to send JMS Messages just the same as with Universal Messaging for example.

Version Constraints

Here: https://docs.microsoft.com/de-de/azure/service-bus-messaging/service-bus-java-how-to-use-jms-api-amqp (or https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-java-how-to-use-jms-api-amqp) you can find instructions on how to use MS SB with AMQP and QPID. Unfortunately, this describes the usage of "Qpid 0.32" which is not the current version and the way to use the most current one - "Qpid JMS 0.20.0" - is a little different.

In this document I will focus one "Qpid JMS 0.20.0" - the current version.

Steps to install and configure the integration

  1. Network Requirements

    Make sure that the hostname of your Service Bus (e.g. "XXXXXXX.servicebus.cloudapi.de") can be resolved and reached and that tcp-port 5671 is accessible from the IS-Host.
     
  2. Download and install QPID

    QPID can be found here: https://qpid.apache.org/

    Here's the direct link to the version I am referring here: https://qpid.apache.org/releases/qpid-jms-0.20.0/
    copy all JAR Files contained in the Archive's "lib" folder to "/opt/webMethods9/IntegrationServer/instances/default/lib/jars/custom/" (or to the matching folder of your IS installation).

    Restart IS after that.
     
  3. Create or change a Trustrore

    As we want to establish a secured amqps connection to the Service Bus, we need to have all required SSL certificates stored in a truststore. As it seems that none of the existing ones contains the required certificates I - for transparency - created a new one like this:

    Acquire the needed certificates by visiting https://<URL-Of-Your-Servicebus> , e.g. https://XXXXX.servicebus.cloudapi.de in a browser. Then click on the "lock" in the URL (or whatever your browser shows for a secure connection) and open the certificate from there.

    Save the current certificate:



    When asked for the format to export, choose "DER binary" and save it as a ".cer" file, e.g. "1.cer"

    Most likely you can see that your certificate is based on a certificate chain, which means it depends on other certificates. For each of those click "show certificate":



    And save that one in the same way as the one before. Repeat until you reach the root-certificate. In this example you will end up with three *.cer files. For further reference I will call them 1.cer, 2.cer and 3.cer

    You should now create a new Truststore file for those certificates 
    ​/opt/webMethods9/jvm/jvm/bin/keytool -import -file
/opt/webMethods9/IntegrationServer/instances/default/packages/DEV_fse/resources/1.cer -keystore azureTruststore.jks -alias "D-TRUST Root Class 3 CA 2 2009"

    /opt/webMethods9/jvm/jvm/bin/keytool -import -file
    
/opt/webMethods9/IntegrationServer/instances/default/packages/DEV_fse/resources/2.cer -keystore azureTruststore.jks -alias “D-TRUST SSL Class 3 CA 1 2009”
    

    /opt/webMethods9/jvm/jvm/bin/keytool -import -file
    
/opt/webMethods9/IntegrationServer/instances/default/packages/DEV_fse/resources/3.cer -keystore azureTruststore.jks -alias “servicebus.cloudapi.de”

    You will be asked to set the password for this newly created truststore the first time. Now move the truststore to “/opt/webMethods9/IntegrationServer/config/certs/trusted” (for later reference). You might add it to IS as a truststore (by using the admin-UI "Security > Keystore " and “Create Truststore Alias”), but there is no technical need to do it, as in our case the IS is not using the truststore - it’s only used by QPID.

     

  4. Create a properties file for JNDI You need to create a "servicebus.properties" file to act as the data source for a pseudo JNDI Server. You can technically place that file wherever you want, but I would recommend putting it in the "resources" folder of the "XXXXXXConnection" package. This should be the content of that file: 
    # servicebus.properties - sample JNDI configuration
# Register a ConnectionFactory in JNDI using the form:
# connectionfactory.[jndi_name] = [ConnectionURL]
connectionfactory.SBCF = amqps://XXXXXX.servicebus.cloudapi.de?jms.username=xxxxx&jms.password=xxxxxxx&amqp.idleTimeout=120000&jms.receiveLocalOnly=true&transport.trustStoreLocation=/opt/webMethods9/IntegrationServer/config/certs/trusted/azureTruststore.jks
# Register some queues in JNDI using the form 
# queue.[jndi_name] = [physical_name] 
# topic.[jndi_name] = [physical_name]
queue.QUEUE = myqueue
​
    Some Explanations:​
    • "SBCF" will be the JNDI-Lookup name for the connection factory. This name is later needed in your JMS-Connection 
    • "xxxxxx.servicebus.cloudapi.de" is the URL to your Service Bus
    • "jms.username" will be provided by your friendly Azure administrator
    • "jms.password" will be provided by your friendly Azure administrator. But be aware that you will need to URL-Encode what you will get from the admin before you can use it in this URL. This can for example be done by calling the IS Service "pub.string:URLEncode" manually in Designer.​​ 
    • "amqp.idleTimeout" needs to be set to 120000 (or higher) because of otherwise you cannot connect to SB
    • "jms.receiveLocalOnly" needs to be set to "true" because of otherwise you cannot connect to SB 
    • "transport.trustStoreLocation" needs to point the truststore that contains all SSL Certificates that are required to create a secured (AMQPS) connection to SB
    • "queue.QUEUE": QUEUE is the JNDI-Lookup name you will use later in your JMS-Client to send messages or in your JMS-Trigger to receive them. You should set to a something more meaningful probably. The value of this ("myqueue" in the example) is the name of the queue on the SB and has to be provided by your Azure Admin.
       
  5. Create the JNDI Alias in IS

    Open the IS-Admin-UI and navigate to "Settings > Messaging > JNDI Settings", click on "Create JNDI Provider Alias". Enter the following settings:

    The only two important values are:

    • "Initial Context Factory": "org.apache.qpid.jms.jndi.JmsInitialContextFactory"
    • "Provider URL": Must point to your created servicebus.properties, e.g.  "file:/opt/webMethods9/IntegrationServer/instances/default/packages/XXXXXXConnection/resources/servicebus.properties"​

      After saving this, you can try to make a Test Lookup (by clicking the "play"-icon). It should produce an output like that:​​

      Basically at this point two things could have gone wrong:
       
    • The path to your properties file is wrong and the file therefore cannot be found 
    • The QPID-JARs are not correct or not in the classpath of IS (wrong directory or maybe IS not restarted) and therefore the class could not be instantiated
       
  6. Create JMS Connection

    Open the IS-Admin-UI and navigate to "Settings > Messaging > JMS Settings ", click on "Create JMS Connection Alias". Enter the following settings:


    ​
    • "Connection Alias Name": "QPID" (or whatever you like)​
    • "Transaction Mode": "NO_TRANSACTION"
    • "Connection Client ID": Whatever you like, but must be unique within the messaging provider
    • "Create Connection Using": "JNDI Lookup"
    • "JNDI Provider Alias Name": The name you set in your JNDI-Provider, e.g. "QPID"
    • "Connection Factory Lookup Name": The name you set in your servicebus.properties file, e.g. "SBCF"
    • "Create Temporary Queue": must be OFF

​The rest can be left on default values. After saving you can try to enable the connection.​

​Basically at this point two things could have gone wrong:

  • Connection is refused or Host not found: Check Firewall and/or Proxy and/or DNS. Hostname must be reachable and Port 5671 must be enabled
  • SSL/Certification errors: Check if you have put all required certificates in your truststore and that your servicebus.properties file points to the right truststore

7.Test the Connection

​You should now be able to create an ordinary JMS-Listener in Designer. Just pick your JMS-Connection Alias (e.g. “QPID”) and the JNDI-lookup name of your queue (e.g. “QUEUE”). For ease of use you might want to pick “pub.flow:tracePipeline” as the service to process incoming messages.



Then write a simple service that publishes a JMS-message to the queue. When invoking “pub.jms:send” you need to set:

  • connectionAliasName to yours (e.g. “QPID”)
  • destinationName again to your JNDI-lookup name for your queue (e.g. “QUEUE”)
  • destinationType should be set to “QUEUE”
  • set “JMSMEssage/body/string” to whatever test message you like.

By running the service you should be able to send a message to the Service Bus queue and your Trigger should receive it and invoke “tracePipeline”, so you can see the round-tripped message in your “server.log”.

​

Further Read:

  • Check out the latest version of Microsoft Azure Service Bus here: Microsoft Azure Service Bus
  • Learn step-by-step how to configure an Azure Serive Bus account in webMethods.io Integration workflow: Configuring Azure ServiceBus account in webMethods.io Integration Workflow


    • #qpid
    #apache-qpid-jms
    #webMethods
    #Integration-Server-and-ESB
    #wiki
    #azure
    #Integration
    #esb