IBM Z and LinuxONE - Software - Group home

IBM Z Common Data Provider and IBM Z Operations Analytics both provide data ingestion pipelines to the Elastic Stack platform. In most cases, only one ingestion kit is needed to use IZOA applications on Elastic Stack. However, if required, you can have both IBM Z Common Data Provider Elasticsearch ingestion kit and IBM Z Operations Analytics Elastic Stack ingestion kit working for the same Elastic Stack instance.

1. Install the IBM Z Common Data Provider Elasticsearch ingestion kit.
   1. From the IBM Z Common Data Provider /usr/lpp/IBM/zcdp/v2r1m0/DS/LIB directory, download the IBM Z Common Data Provider Elasticsearch ingestion kit, which is in the tar.gz file, in binary mode.
   2. Extract the Elasticsearch ingestion kit.
   3. Create a new directory in a location of your choice, for example <zcdp-pipeline-dir>/zcdp-ingestion-kit, and copy the Logstash configuration files that you need for your environment to the new directory.
   4. If needed, configure the files according to the instructions in step 3 in Preparing to send data to Elasticsearch in the IBM Z Common Data Provider documentation.
2. Install the IBM Z Operations Analytics Elastic Stack ingestion kit.
   1. Extract the IBM Z Operations Analytics Elastic Stack ingestion kit, which is in the file IZOA-IngestionKit-4.1.0.zip.
   2. Create a new directory in a location of your choice, for example <izoa-pipeline-dir>/izoa-ingestion-kit, and copy the Logstash configuration files that you need for your environment to the new directory.
   3. If needed, configure the files according to the instructions in step 1 in Deploying the Z Operations Analytics application on the Elastic Stack platform in the IBM Z Operations Analytics documentation.
      
3. Configure the pipelines.yml file by specifying the pipeline IDs and the pipeline configuration files (and other settings if needed). In this scenario, use the following code sample to configure the pipelines.yml file to specify both IBM Z Common Data Provider and IBM Z Operations Analytics pipelines:

   - pipeline.id: ibm_cdp

     path.config: "<zcdp-pipeline-dir>/zcdp-ingestion-kit/*.lsh"

   - pipeline.id: ibm_zoa

     path.config: "<izoa-pipeline-dir>/izoa-ingestion-kit/*.conf"
   For other considerations about configuring multiple pipelines, see Multiple Pipelines in the Logstash documentation.

4. Start Logstash without the -e and -f arguments. Otherwise the pipelines.yml file will be ignored.
5. Check the Logstash log after the Logstash is started with multiple pipelines for the first time. Verify that the configuration is correct and that the pipelines start successfully.