Cloud Pak for Business Automation

Did you ever got confused when trying to analyse IBM Liberty log files in JSON format?

Especially in Cloud environments like Kubernetes it is a common practice to write logs in a JSON format, in order to aggregate them later in logging backends.

But these backends are not always available, for example in development and test clusters. Obviously the JSON format is hard to read for a human.

{“@timestamp”:”2020–10–05T09:01:18.337Z”,”@metadata”:{“beat”:”filebeat”,”type”:”_doc”,”version”:”7.0.1"},”host”:{“name”:”workflow-designer-ums-deployment-7fd4d68d7-mb7cs”},”agent”:{“ephemeral_id”:”a82b2d8c-025b-4328-ac37–66921bdaf569",”hostname”:”workflow-designer-ums-deployment-7fd4d68d7-mb7cs”,”id”:”a2fe592c-f9f0–4c65-a136-f677f2524640",”version”:”7.0.1",”type”:”filebeat”},”log”:{“offset”:43805,”file”:{“path”:”/logs/application/liberty-message.log”}},”message”:”{\”type\”:\”liberty_message\”,\”host\”:\”workflow-designer-ums-deployment-7fd4d68d7-mb7cs\”,\”ibm_userDir\”:\”\\/opt\\/ibm\\/wlp\\/usr\\/\”,\”ibm_serverName\”:\”ums\”,\”message\”:\”CWWKF0011I: The ums server is ready to run a smarter planet. The ums server started in 126.196 seconds.\”,\”ibm_threadId\”:\”00000025\”,\”ibm_datetime\”:\”2020–10–05T09:01:17.338+0000\”,\”ibm_messageId\”:\”CWWKF0011I\”,\”module\”:\”com.ibm.ws.kernel.feature.internal.FeatureManager\”,\”loglevel\”:\”AUDIT\”,\”ibm_sequence\”:\”1601888477338_000000000004E\”}”,”log-type”:”application”,”input”:{“type”:”log”},”app_id”:””,”ecs”:{“version”:”1.0.0"}}

vs.

[2020–10–05T09:01:17.338+0000] 00000025 AUDIT CWWKF0011I: The ums server is ready to run a smarter planet. The ums server started in 126.196 seconds.

This blog shows an easy way to convert these log records into a human-readable format, using a small Node.js program. Node.js was an intuitive choice for working with JSON objects. But it would also be possible to use python for example.

Node.js is Open-Source software and can be downloaded from here: https://nodejs.org/

Note, that the blog only shows snippets of code. The complete program, as well as a guide to set it up, can be found on GitHub.

https://github.com/benjaminwendeibm/logformat



The implementation of the Log-Formatter can be split into 3 logical steps:

1. Read the JSON input line-wise from stdin
2. Parse it as an object (and do not fail if it’s no JSON)
3. Format it human-readable and print it to stdout

Read the JSON input line-wise from stdin

Reading from stdin can be easily implemented like shown below. Note that the processStdIn function must be called somewhere when the program starts. There is also a debug option, to show the input line, in case the program fails.

async function processStdIn(debug) {
  const rlIn = readline.createInterface({
    input: process.stdin,
    crlfDelay: Infinity
  });

  // read from stdin until program ends
  for await (const line of rlIn) {
    if(debug == true) {
      console.log("RAW: " + line)
    }
    processLine(line);
  }
}

Parse the line as object

In JavaScript the parsing of JSON can be done using the globally available JSON module. We just pass in the text and will get an object. There is also some error handling to ensure we are not failing if there is some random syntax issue or the input is no JSON at all.

function parseLine(line) {
  try {
    if(line.trim().startsWith("{") && line.trim().endsWith("}")) {
      return JSON.parse(line.trim());
    }
  }
  catch(e) {
    return null;
  }
  return null;
}

Format the output

The processLine function now uses the parsed JavaScript object and checks for a couple of well known property names, in order to recognise a certain log format. For each of these formats, the relevant properties will be written to stdout, using the ES6 template string syntax and a helper function called prop(). This helper is basically responsible to make the program fail-safe, as it will only return empty strings, if some property is missing.

function processLine(line) {
  var o = parseLine(line);
  if(o) {
    if(o['@metadata'] && o.message && o.message.startsWith("{") && o.message.endsWith("}")) {
      return // ignore filebeat lines as this doubles the log lines
    }
    if(o.type && o.type == "liberty_message") {
      console.log(`[${prop(o, "ibm_datetime")}] ${prop(o, "ibm_threadId")} ${prop(o, "loglevel")} ${prop(o, "message")}`)
      return
    }
    else if(o.type && o.type == "liberty_ffdc") {
      console.log(`[${prop(o, "ibm_datetime")}] ${prop(o, "ibm_threadId")} FFDC ${prop(o, "ibm_stackTrace")}`)
      return
    }
    else if(o.type && o.type == "liberty_trace") {
      console.log(`[${prop(o, "ibm_datetime")}] ${prop(o, "ibm_threadId")} ${prop(o, "ibm_className", "module")}.${prop(o, "ibm_methodName")} ${prop(o, "message")}`)
      return
    }
    else if(o['@metadata']) {
      console.log(`${prop(o, "message")}`)
      return
    }
    else if(o.level && o.ts && o.msg && o.logger) {
      console.log(`${prop(o, "level")} ${prop(o, "logger")} ${prop(o, "namespace")}/${prop(o, "name")}: ${prop(o, "msg")} ${prop(o, "error")}`)
      if(o.stacktrace) {
        console.log(`${prop(o, "level")} ${prop(o, "logger")} ${prop(o, "namespace")}/${prop(o, "name")}: ${prop(o, "stacktrace")}`)
      }
      return
    }
    else if(o.msg) {
      console.log(`[${prop(o, "time")}] ${prop(o, "file")} ${prop(o, "level")} ${prop(o, "msg")}`)
      return
    }
    else {
      console.log(line);
      return
    }
  }
  else {
    console.log(line);
    return
  }
}

I appreciate any comments or feedback, using the comment function below.