This blog is a continuation of our original blog post about IBM Z Digital Integration Hub (zDIH), We encourage you to take the time to read it and familiarize yourself with the concepts of zDIH as well as how we have implemented it in our environment.

IBM zDIH provides real-time information sharing between z/OS core systems and hybrid cloud applications and has built-in support for industry-standard interfaces and connectors. You can find additional information about this support for the different interfaces and connectors at the following zDIH documentation sites:

• Establishing zDIH interfaces – to provide support for downstream applications on Hybrid cloud to consume information from zDIH caches via standard interfaces such as SQL/JDBC, REST and Kafka
• Establishing zDIH connectors – to provide support with zDIH’s built-in connectors to share information from zDIH caches in real-time to any JDBC compliant database, MongoDB or via Kafka events.

For example, one of the connectors is the zDIH MongoDB connector which provides support to efficiently share curated information from z/OS core applications to MongoDB database in an event-driven approach. This article will discuss our step-by-step approach to set up and configure the IBM zDIH MongoDB connector:

MongoDB setup

• We setup a local instance of MongoDB, which you can do on a local machine, container or virtual machine, following steps  from Mongo documentation
  (See https://www.mongodb.com/docs/manual/installation/). If you already have your own MongoDB server, you can use it’s port and access information in the Running the MongoDB Connector step below.
  
• We created a mongo server, with password protection on docker :

  docker pull mongodb/mongodb-community-server
  docker run --name mongod -d -e MONGO_INITDB_ROOT_USERNAME=user -e MONGO_INITDB_ROOT_PASSWORD=pass -p 27017:27017 mongodb/mongodb-community-server:<version-number>

• We had to make the MongoDB port public:

  sudo firewall-cmd --permanent --zone=public --add-port=<port>/tcp
  sudo firewall-cmd –-reload

• We decided to install MongoDB Compass (See https://www.mongodb.com/try/download/compass) which we used to query and analyze the database with a visual interface.
  
  
• We then created a connection from MongoDB Compass to the MongoDB deployment. Here is an example:

IBM zDIH Management Center Setup

The zDIH Management Center allows you to monitor and administer your zDIH clusters. You can also monitor the work submitted by client applications as well as work being consumed by connector applications such as the MongoDB connector. We set up a new instance of the zDIH Management Center instance to monitor how the connector applications operated and consumed our target zDIH maps.

Here's how you can do the same:

• Download the zdih-client-<version-number>.zip file from /<path-to_your-zdih-install directory>/ from USS
• Extract the zip on your preferred Management Center machine
• Go to /<extracted_path>/zdih210/management-center/bin and issue the appropriate command to start the Management Center:
  • zdih-mc start (Mac, Linux, Unix)
  • zdih-mc.cmd start (Windows)
• Once the Management Center is started it is available at localhost:8080 by default
• To add a zDIH server connection on your Management Center:
  • Go to cluster connections at the top -> add (big button on the left) -> enter your cluster name & server ip:port -> connect. If you require secure connection, you can use a configuration file.

See IBM zDIH documentation for the details about the administering the Management Center.

The initial setup steps are now complete. 

Running the MongoDB Connector

1. Start the MongoDB server present on your local or virtual machine using the command.

   sudo systemctl start mongodb

2. Connect to the MongoDB server from MongoDB Compass
3. Create the collection in the database in which you want the documents to be written from the zDIH cache
4. Configure the parameters in the yaml file with the desired values. (i.e server_address, cluster_name, cache_name, pipeline_name, mongo_connection_string, mongo_database, mongo_collection)

   # ============================================================================
     # zDIH  Log Process Default Configuration. Use this template for customization
   # ============================================================================
   # The host name or host address and port of the zDIH server to connect.
   
     # This host name or host address is MANDATORY. It has to be either specified
     # in the configuration yaml or as command line property
     # -Dserver_address=<server_address>
     #
     # The port is OPTIONAL. The default value is 5701.
     # To specify a port for zDIH server, use the format "<hostname>:<port>".
     #
     server_address: "xxx.xx.xxx.xx"  
   
     # The cluster name of the zDIH server to connect.
     # This field is OPTIONAL. The default value of cluster_name is "zdih"
     # To specify as command line option, use the property
     # -Dcluster_name=<cluster_name>.
     #
     cluster_name: "zdih"  
   
     # The name of the cache to be listened by the pipeline.
     # This field is MANDATORY.
     #
     cache_name: "SAMPLE"  
   
     # The name of the pipeline to be created.
     # This field is MANDATORY.
     #
     pipeline_name: ""  
   
     # The mongodb connection url string to configure the sink.
     # This field is MANDATORY.
     #
     mongo_connection_string: "mongodb://xxx.xx.xxx.xx" 
   
     # The database name to configure the sink.
     # This field is MANDATORY.
     #  
     mongo_database: ""
     
     # The collection name to configure the sink.
     # This field is MANDATORY.
     #
     mongo_collection: ""

5. Add the event-journal configuration in the zdih.xml located in /zdih/instances/<cluster-name>/config

   <map name= <Name of the Map/Cache given in configuration yaml> >
   
           <event-journal enabled="true">
               <capacity>10000</capacity>
               <time-to-live-seconds>0</time-to-live-seconds>
           </event-journal>
    </map>

6. Place the sample application jar and customized configuration file in the USS classpath as defined in the JCL that starts the server on the z/OS host(s) where the zDIH cluster runs. E.x /zdih/global/lib
7. Run the following command to start the MongoDB connector on the zDIH cluster host(s):

   java -cp {path where the jar is placed}/zdih-mongo-jar-with-dependencies.jar -Dmongo_connector_config_path={path where the config file is placed}/mongo_connector_yaml.yaml ibm.mongosample.SampleMongoDBSink

8. Check the status of MongoDB connection pipeline under ‘Jobs’ menu in the zDIH Management Center.
9. Once the MongoDB connection pipeline status is RUNNING, load the zDIH cache for which you enabled the MongoDB connector in the previous step.
10. View the document in the collection from MongoDB compass.

Notes from working closely with the software

• The updated version of zDIH only supports Java 11 and upward. Java 17 is recommended.
• The cluster-name refers to the name of the zDIH server being used which is started via JCL.
• The cache_name and pipeline_name provided will be seen under Maps and Jobs respectively in the Management center.

Conclusion

In this blog we’ve walked through the steps of connecting a MongoDB database and zDIH server. We covered setting up MongoDB on your local or virtual machine including downloading MongoDB Compass for ease via a GUI (Graphical User Interface), downloading and starting the zDIH Management Center, and starting the zDIH MongoDB connector pipeline to share information from zDIH caches to MongoDB collections. 

We hope to continue to explore additional zDIH connectors and share our findings in the future.

Resources/Links

-       MongoDB Compass

-       Install MongoDB

-       zDIH Documentation

•       Official zDIH MongoDB Connector Documentation

Author

Timothy Odom