MQ

MQ

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Back to discussions
Expand all | Collapse all

Monitoring MQ with ELK (Elasticsearch) stack

  • 1.  Monitoring MQ with ELK (Elasticsearch) stack

    Posted Tue August 18, 2020 02:12 AM
    Edited by Bernard Pittens Tue August 18, 2020 02:32 AM
    Hi,
    We are looking for our queueing tool for the next 5 years. At this moment we use MQ appliances but we have to move to the cloud.
    We are testing with MQ on Linux.

    About monitoring/logging:
    In general we use the ELK stack for logging and log agregating at our company, and for file shipment we use the Filebeat agent which is installed at the MQ Server. Filebeat has lot of connectors(modules) and ibmmq is one of them  We managed to get it working, but its just the hello world config for ELK.  At this moment Filebeat is sending logs directly to elasticsearch.
    Is there any standard or best practise how we have to configure this for MQ?
    Some kind of link, maybe somebody has an example with a template or something ? 
    Maybe some field mapping or filter for logstash ?
    Kind regards

    ------------------------------
    Bernard Pittens
    Integration Engeneer
    Sligro Foodgroup B.V.
    Veghel
    ------------------------------


  • 2.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 12:19 AM

    MQ Appliance does not allow Filebeat agent to be installed. So option is to stream it to filer/syslog server and then stream it.

    MQ Appliance log : using log targets stream it to a logsys server and have Filebeat read from it.
    Qmgr Events: Using the sample Event management C code,  write the events to a file and have Filebeat read it.
    Queue manager logs : Currently IBM does not allow and if you plan to use Filebeat it gets difficult as there will be another layer of code to find deltas and then push them to elasticsearch.

    If MQ on Linux, it is easy to use Filebeat.
    - setup the queue manager to write qmgr logs as JSON, and then Filebeat reads it with no logstash transformation/grok. All key values show up exactly. In MQ appliance this is missing. 

    Note: We use elasticsearch for MQ todo all monitoring and alerting.



    ------------------------------
    om prakash
    WI
    ------------------------------

    Original Message


  • 3.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 02:18 AM
    Edited by Bernard Pittens Wed August 19, 2020 02:20 AM
    Hi,

    Thanks Om Prakash, no its not our intention to use ELK on the MQ Appliances because we are on V8.0.0.11 which is a custom edition for us. MQ V8 doesn't support syslog out of the box on MQ Appliance.
    Like you describe the Filebeat implementation on Linux we have a test environment which is configured just like that, so if this is best practice then its ok.

    Kind regards

    ------------------------------
    Bernard Pittens
    Integration Engeneer
    Sligro Foodgroup B.V.
    Veghel
    ------------------------------

    Original Message


  • 4.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 05:51 AM
    There is an article about the IBM MQ Module for Elastic Search
    https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-module-ibmmq.html

    ------------------------------
    Matthias Jungbauer
    ------------------------------

    Original Message


  • 5.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 06:52 AM
    Edited by Bernard Pittens Wed August 19, 2020 06:53 AM
    Thanks, yes I know, I have used this manual, together with the default steps which are explained in Kibana.
    So for logging this seems to be the best practice. Ok.


    ------------------------------
    Bernard Pittens
    Integration Engeneer
    Sligro Foodgroup B.V.
    Veghel
    ------------------------------

    Original Message


  • 6.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 10:15 AM
    Just a Note if you are using RDQM, Filebeat can be a bottleneck. As the Qmgr moves between nodes, pacemaker wil not be able to shutdown Filebeat and cause hang of the HA for QMgr.

    If you are looking for specific use case, please share. Willing to assist on ELK with MQ integration.

    ------------------------------
    om prakash
    WI
    ------------------------------

    Original Message


  • 7.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 02:55 PM
    Edited by Bernard Pittens Wed August 19, 2020 02:58 PM
    Hi om prakash,

    Yes, we are testing with RQDM and  I just checked it and we are able to switch by shutting one Linux machine A down, MQ switched without a problem to the other system B queuemanager, and when we started  system A , system A became the active Queuemanager again.
    Off course we needed to restart the filebeat process but this can be fixed in a startup script, so the pacemaker did not give any problem at least in this test.
    What do you mean exactly ?

    ------------------------------
    Bernard Pittens
    Integration Engeneer
    Sligro Foodgroup B.V.
    Veghel
    ------------------------------

    Original Message


  • 8.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 05:00 PM
    Having a pre and post script to switch failure will resolve the Filebeat shutdown and pacemaker.
    But if you use the move queue manager command from the bin folder.. you would see issue that is what I meant.

    ------------------------------
    om prakash
    WI
    ------------------------------

    Original Message


  • 9.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 09:24 AM
    Hi Bernard,

    I would look at using the JSON formatted MQ logs, that should make it a lot easier to ingest the data. I think you will be able to send the logs as-is and not require filters etc.

    https://www.ibm.com/support/knowledgecenter/SSFKSJ_9.2.0/com.ibm.mq.con.doc/q018792_.htm

    Regards,

    Martin 

    ------------------------------
    Martin Evans
    ------------------------------

    Original Message


  • 10.  RE: Monitoring MQ with ELK (Elasticsearch) stack

    Posted Wed August 19, 2020 02:59 PM
    Thanks for the tip Martin.

    ------------------------------
    Bernard Pittens
    Integration Engeneer
    Sligro Foodgroup B.V.
    Veghel
    ------------------------------

    Original Message