Sending IBM App Connect Enterprise log messages to an ELK stack

Introduction

• Elasticsearch: an open source search and analytics engine. It is able to achieve fast search responses because, instead of searching the text directly, it searches an index instead.
• Logstash: a light-weight, open-source, server-side data processing pipeline. It can receive data from multiple sources simultaneously, transform it, and then send it to a specific destination. It is often used as a pipeline for Elasticsearch.
• Kibana: an open-source data visualization and exploration tool. Kibana lets you visualize your Elasticsearch data. You can use it to build clear visualizations and dashboards. Kibana uses an index pattern to tell it which Elasticsearch indices to explore.
• Beats: open source ‘data shippers’ which can be installed as agents on servers to send operational data to Elasticsearch. Beats can be used for capturing audit data, log files, cloud data, availability, metrics, network traffic and windows event logs. Beats can send data directly to Elasticsearch or via Logstash, where data can be further processed and enhanced, before it is visualized in Kibana.