Sterling B2B Integration

 View Only

Global Mailbox:  Components and deployments

By Scott Guminy posted Mon December 13, 2021 09:52 AM

  

This is part of a multi-part series on IBM Sterling Global Mailbox. 

What is Global Mailbox? 

IBM Sterling Global Mailbox helps companies address demands for high availability operations and redundancy with a robust and reliable data storage solution available across geographically distributed locations. It is an add-on to Sterling B2B Integrator and Sterling File Gateway. 

What components are used in Global Mailbox? 

A typical global mailbox production environment includes many components that facilitate redundancy and data replication.  Below is the recommended topology for a 2 Data Center deployment.  This article will discuss each required component. 

Components and setup are required for a two data center system.

Global Load Balancer 

Global Load Balancers are responsible to routing traffic to your B2Bi/SFG data centers.  The load balancer is responsible for monitoring the environment and sending traffic only to working data centers. Global Load Balancers can be configured in many different ways, but clients should stay connected to a data center for a period of time.  If clients switch between data centers too quickly, they may see inconsistent data depending on how quickly data is replicated across data centers. 

The Global Load Balancer is not provided with the solution. 

Local Load Balancer 

Local Load Balancers are responsible for routing traffic within the Data Center.  The local load balancer is responsible for monitoring individual B2Bi/SFG servers and the exposed protocols.  Traffic should only be sent to working servers within the data center.   

Monitoring of the individual servers can be done in several ways.  For Global Mailbox it is recommended to do a periodic GET of a special file called the health check file (sometimes called health check message).  For example, an FTP get of the health check message will fail if any of the components in that data center are not working correctly. The load balancer should not send any traffic to that server until the health check becomes successful.  See the Health Check Utility for information on configuring the health check file. 

The Local Load Balancer is not provided with the solution. 

DB (Relational Database) 

This is the relational database that is normally used for B2Bi/SFG.  It is still required when deploying B2Bi/SFG with Global Mailbox.  Each data center has a separate B2Bi/SFG cluster, each having a separate relational database.   

The relational database stores standard operational and configuration data such as:  Business Processes, Adapter Configurations, Trading Partner Configurations, Routing Visibility Events, etc. This data is not replicated between data centers.  Only Mailbox Data is replicated between data centers. 

B2Bi (or SFG)

B2Bi nodes include additional adapters and functionality to work with Global Mailbox.  Specifically, the protocol adapters for FTP, SFTP and Connect:Direct  and My File Gateway include support to use Global Mailboxes.   

These adapters support both a mix Global and Traditional Mailboxes.  When a Trading Partner or User is created, they are assigned a Virtual Root or a “root mailbox”.  If this Virtual Root is defined in Global Mailbox, all protocol actions for that user will be done in Global Mailboxes – with mailbox data stored in Cassandra.  When the Virtual Root is defined in Traditional Mailbox, all protocol actions for that user will done in Traditional Mailboxes – with mailbox data stored in the B2Bi/SFG relational database.  You cannot combine Global Mailbox and Traditional Mailbox for an individual User/Trading Partner.  

Global Mailbox Client Adapter (GMCA) 

This adapter is used by the protocol adapters and business process engine to access Global Mailbox functionality.  Every node which needs access to Global Mailboxes requires a GMCA.   

By default, these are deployed only to ASI nodes. Copies of this adapter can be created and deployed to Adapter Containers if needed. 

Global Mailbox Event Rule Server Adapter (ERA) 

This adapter is not shown on the diagram but is important in processing files (messages) added to Global Mailbox.  For each message which requires processing/routing an event is raised and placed on an MQ queue.  The event rule adapter consumes these events from the queue and triggers the business process for the message. This adapter also ensures processing status is up to date in Global Mailbox. 

Event Rule Adapters are deployed to the ASI nodes. There is no need to deploy these to Adapter Containers. 

GM (Global Mailbox Admin Node) 

When you add Global Mailbox to your B2Bi/SFG cluster, an additional process is created.  This process is called the GM Admin Node.  Sometimes it is referred to as the “GM Liberty Server”. This node is a WebeSphere Liberty Server which performs the following functions: 

  • Global Mailbox User Interface:  Global Mailboxes are managed with a different user interface with additional functionality beyond Traditional Mailbox.  This user interface knows how to manage the mailbox data stored in Cassandra. 
  • Global Mailbox Scheduler:  The scheduler runs periodic jobs to keep Global Mailbox functioning smoothly, such as Purge jobs. 
  • Global Mailbox Replicator: The replicator is responsible for replicating file content (payloads) between data centers). 
  • Aspera FASP:  The replicators use the Aspera FASP protocol to replicate payloads between data centers.  The FASP protocol includes 2 sub-protocols:  an SSH server for as a control layer, a UDP protocol for data transfer.  Both protocols run within this GM Admin Node. 

    Each B2Bi/SFG install has a GM Admin Node that is automatically started when you start B2Bi/SFG.

    WebSphere MQ 

    MQ is used to notify B2Bi/SFG when a message is added to a global mailbox.  This triggers processing of the message. To ensure resiliency, the recommended deployment includes an active and standby queue manger in each data center. 

    Shared File System 

    Each data center requires a resilient shared file system to be mounted on all B2Bi/SFG nodes in that data center.  This shared disk is used to store Global Mailbox file content (payloads).  The payloads are then replicated to the shared file system in other data centers by the replicator using the FASP protocol. 

    There are no strict requirements for this shared file system.  It just needs to support the read/write throughput required to meet your performance goals. 

    Cassandra 

    Cassandra is a NoSQL, replicated, fault-tolerate database.  Global Mailbox meta-data is stored in Cassandra.  Cassandra ensures this data is replicated both within and across data centers.  This meta-data includes the following: 

    • Virtual Roots 
    • Mailboxes and their hierarchy 
    • Mailbox permissions 
    • Messages 
    • Event Rules (used to trigger processing of messages) 
    • Processing Status (event status) 
    • Other related data 

      If a user has a Virtual Root defined in Cassandra, all of their mailboxes and related data will be stored in Cassandra and not in the relational database. 

      Each data center requires multiple Cassandra servers to ensure the database is always available.  The recommendation is to have 3 Cassandra servers per data center. 

      ZooKeeper 

      ZooKeeper is a distributed co-ordination service.  In a distributed computing environment, ZooKeeper helps co-ordinate activities across nodes and data centers. 

      Global Mailbox uses ZooKeeper for the following functions: 

      • Global locking: ensuring only one action can update an object at a given point in time.  For example, to ensure accurate extraction counters, the extraction counter is locked before updating to avoid any conflicting updates. 
      • Co-ordination of scheduled jobs: ensuring that a job is only running once in a DC, ensuring that a job is only running once globally, etc 

      Each data center requires multiple ZooKeeper servers to ensure the system is resilient.  There must be an odd number of servers across all data centers. 

      Smaller deployments 

      The diagram above shows a full, resilient, highly available production deployment of B2Bi with Global Mailbox. 

      To get started with Global Mailbox, you may want to use a smaller, less resilient deployment.  See Global Mailbox test environments for information on other deployments that use fewer resources.  These deployments have trade-offs in redundancy but are a great place to start to learn about Global Mailbox. 

      Here is an example learning deployment which requires a single node.  It is a great way to get started learning about Global Mailbox.

      A learning test environment consists of Sterling B2B Integrator, Global Mailbox, WebSphere MQ, Cassandra, and ZooKeeper.


      #SupplyChain
      #B2BIntegration
      0 comments
      21 views

      Permalink