Andy Emmett
Published on 16/11/2018 / Updated on 07/03/2019
Overview
Hello, and welcome to the second article in a series which looks at how to integrate on-premise Queue Manager Clusters with queue managers hosted the IBM MQ on Cloud service. If you have not yet seen Part One, please use the link provided below to take you there now as this provides the foundation which the remainder of this article assumes and is based. Otherwise, here is a reminder of the article…
- Creating a new cluster queue manager in IBM Cloud
- Creating a Secure Gateway service endpoint to allow connection to on-premises queue managers / Installing a Secure Gateway Client
- Adding the Cloud queue manager to queue manager cluster and updating the existing configuration
This is Part Two in the series where we’ll describe the Secure Gateway Service & Client, for use with a hybrid of queue managers running in IBM Cloud, and queue managers hosted in on-premise data centers.
The Secure Gateway Cloud Service
In Part One, we described the creation of a queue manager in IBM Cloud, and that this queue manager (in the context of the scenario) will integrate with existing on-premise queue managers. The cloud queue manager is hosted on IBM infrastructure and made available within the organizations account using a generated hostname and port accessible via the internet. Again, just as a reminder, the scenario diagram is shown below …
Fig 1. Scenario diagram
In brief
This depicts a number of queue managers running in a high availability configured pair of IBM MQ Appliances running in an on-premise data center (typically behind a firewall). Additionally, there is our internet facing CLOUD1 queue manager, exposed (though securely and only to the creating organization) via its hostname and port.
Key Points:
- The green line (labelled ‘Direct Queue Manager Connection’) represents a connection from a queue manager in the on-premise domain, connecting directly to the queue manager in the cloud. This is possible because the cloud-hosted queue manager (CLOUD1) is internet facing.
- The ‘messages’ flowing from the CLOUD1 queue manager to the partial repositories hosted on the pair on MQ Appliances, are routed through the item labelled Secure Gateway and Secure Gateway Client(s). This is because the outbound channel from CLOUD1, is routed to the on-premise queue managers via the Secure Gateway.
Perhaps a couple for questions spring to mind…
“I understand how my on-premise queue managers can reach any cloud queue managers I provision, but how can my cloud queue managers reach the clustered queue managers in my data center ? How can the queue managers on the IBM infrastructure, possibly know how to resolve the hostnames and ports behind my firewall ?”
The answer, and the reason for this article, is the Secure Gateway.
Secure Gateway ?
The Secure Gateway is an IBM Cloud service that enables the hybrid integration of cloud and on-premise worlds. It can be used with many different types of resource, but for the purpose of this article, we will only be describing it with respect to IBM MQ.
Together with a separate on-premise based client, the Secure Gateway enables the definition of gateways to individual on-premise resources or destinations such as queue managers. Assuming a Secure Gateway instance has been provisioned (and appropriate clients have been installed), it would enable in our example, the CLOUD1 queue manager to start an MQ channel and send messages to the queue managers running on-premise in the MQ Appliances. The reminder of this article describes what, is needed to achieve this end.
Fig 2. Secure Gateway parts
Provisioning a Secure Gateway Service instance
The Secure Gateway catalog is where you can provision a Secure Gateway service instance. Once an instance of the service has been associated with an organizations’ account, gateways can be created. The first requirement is to select a plan. The plan you choose is based on the number of on-premise resource to which you need to communicate. In our example, given future thoughts to migrate applications and services to the cloud, we want utilize the full set of queue managers hosted on the IBM MQ Appliances (and two full repositories on AIX servers). If we select the ‘Professional’ plan, we will need to add two gateways to fulfill our needs. In the ‘Professional’ plan (at the time of writing), we could create a total of five gateways each being able to manage five destinations. This would still leave us with destinations to spare.
Fig 3. Selecting a Secure Gateway plan
Adding a gateway
Before we proceed to adding a gateway, please keep in mind that this is a summary of using the Secure Gateway service and that the visual aspects of the service may in time change with respect to what is in this article. Full details of the Secure Gateway service can be found here online.
For each on-premise resource, a new destination will be defined. To get started, click on the Add Gateway button.
Fig 4. Adding a gateway to the provisioned Secure Gateway instance
Your first Gateway
After naming the new gateway and it has been created (in this example called ‘MYGATE’), “destinations” can be added that represent on-premise queue manager resources. To begin with though, it is worth downloading the details including the ID that has been assigned to the gateway. This will be used later with the Secure Gateway Client.
Fig 5. Dashboard with gateway showing no destinations
Gateway information
To see the Gateway ID, click on the cog icon at the bottom of the ‘MYGATE’ panel.
Fig 6. Important connection information for the Secure Gateway instance
Destination Wizard
The destination wizard is where destinations can be added to the gateway. A named destination definition provides details such as hostname and port for the on-premise queue managers. In the example, we have named the destinations, the same as the queue managers to which they will be used. There is a small number of pages in the wizard which are traversed to provide all of the required information. Launch the wizard, by switching to the Destinations tab and clicking + button.
Fig 7. Launching the wizard form the Destination tab
Destination Wizard : Resource Location
This page of the wizard is used to tell the gateway where your resource is located. We’ve talked so far about the resources being based on-premise, but equally, the destination could be in the cloud.
Resource Location : IBM Cloud
Firstly, if you are wanting to secure communications to an IBM Cloud resource from an on-premise application, then Fig 8 below (re created from the original Secure Gateway documentation) shows that kind of flow.
Fig 8. Secure Gateway resource location is IBM Cloud
Resource Location : On-premise
This is the type of destination that we are discussing in this article. The on-premise queue managers are the destinations that you will be selecting in the wizard. For our example, select the on-premise radio button and click ‘Next’ to move the wizard on.
Fig 9. Secure Gateway resource location is on-premise
Destination Wizard : Supplying hostnames and ports
Each time the wizard is being used, we are adding a single destination to a single queue manager. This page allows you to configure the destination with the hostname (or IP address) and port of the queue manager for which the destination is being configured. This is the address as used within the on-premise environment.
Destination Wizard : Protocol
The protocol page can be used to set the protocol for the destination being defined. Typically, the Transmission Control Protocol (TCP) would be used. Choose the protocol and move onto the next page when done.
Destination Wizard : Destination Name
Finally, to compete the configuration of this destination, give the destination a name. This could be as simple as the name of the queue manager to which this destination will apply. In the scenario for example, this could be for queue manager APPL03 and so the name we provide could be APPL03. Click the ‘Add Destination’ button to complete the definition of this destination.
Fig 10. Naming the destination being created
Your new destination
Once added, the destination will be assigned a port number. This port when combined with the hostname generated for provisioned Secure Gateway instance (MYGATE), can be used in the cloud to route connections to on-premise resources. In this case, it will be APPL03
Fig 11. Viewing the created destination
Details
Once you click the settings button, a popup appears containing information that you will need later. In particular, hostname and port information. So, a sender channel in the cloud queue manager that specifies this hostname and port as a connection name, will route messages to the on-premise resource (in this case a queue manager) as defined by the destination. In our example here, that will be to our APPL03 queue manager.
Fig 12. Showing information about the destination for on-premise queue manager, APPL03
The Secure Gateway Client
The Secure Gateway Documentation (see IBM Cloud catalog) contains the following information “The Secure Gateway Service provides a quick, easy, and secure solution for connecting resources in a protected environment to cloud resources. By deploying the light-weight and natively installed Secure Gateway Client, you can establish a secure, persistent connection between your environment and the cloud through an outbound call. Once the client is connected, you can safely connect your applications and resources by specifying their host and port to create corresponding destinations on the cloud. Rather than bridging your environments at the network level like a traditional VPN that begins with full access and must be limited from the top down, Secure Gateway provides granular access only to the resources that you have defined.”
The clients (available for a number of platforms and aptly named the Secure Gateway Client) can be freely downloaded onto servers in your environment. Destinations (the on-premise resource targets) are defined in the provisioned Secure Gateway and authorization rules are created in the client for the resources. In our case, these resources are the queue manager, both on the appliances and the two servers where full repository queue managers for the cluster are hosted.
Please note: Though the diagram depicts two servers for the Secure Gateway Client as being inside the cluster, they are in this case simple stand-alone servers. That said, there is no reason preventing the Secure Gateway Client from being installed on the same system already hosting queue managers (providing a client is available for that platform).
Fig 13. The Secure Gateway client in the ‘Big Picture’
Installing a Secure Gateway Client
The Secure Gateway Client is available for a number of different platforms. The downloads for the clients can be found when you have provisioned a Secure Gateway instance. Clients can be installed onto a number of servers in your environment. With the Professional and Enterprise Secure Gateway plans, you can connect multiple instances of the Secure Gateway Client to your gateway to automatically leverage built-in connection load balancing and connection fail-over should one Client instance go down.
Follow the instructions for the client(s) that you have chosen to install them on your nominated servers.
Important note, the Secure Gateway Client is not available for the IBM MQ Appliance.
Fig 14. Page showing supported operating systems for Secure Gateway clients
Launching The User Interface
Now that a client is installed, it’s possible to define rules as to which which resources can be accessed by the Secure Gateway. A web browser can be used to launch the client user interface. Navigate to http://localhost:9003 and the sign in page will be presented. Enter the requested details (a gateway ID will be required) and click CONNECT
Fig 15. Logging into the Secure Gateway client web interface
The Dashboard
Once the client has connected to the Gateway in IBM Cloud, the dashboard page is loaded. Here, only a small number of options are needed to configure the client. Click Access Control List to begin adding the resource rules for your environment.
Fig 16. Dashboard showing a connected Secure Gateway client
Access Control
The Access Control List page is where the on-premise resource hostnames and port can be added to grant access to them by the Secure Gateway.
Fig 17. Managing on-premise access
Summary
Having created the gateways (of which two are required for all eight on-premise queue managers), and defined the destinations for them, we now have the information to enable us to make necessary configuration changes to the on-premise and provisioned queue managers. In particular, we now have connection name information that we can add to cluster (CLUSRCVR) channels. This is covered in Part Three which is the final part of this article that provides examples based on our scenario. With this, you can hopefully apply the same principles in your own hybrid queue manager cluster environment.