kafka-console-consumer.sh is one of the most useful tools in the Kafka user’s toolkit. But if your topic has Avro-encoded events, the output can be a bit hard to read.
You don’t have to put up with that, as the tool has a formatter plugin framework. With the right plugin, you can get nicely formatted output from your Avro-encoded events.
With this in mind, I’ve written a new Avro formatter for a few common Avro situations. You can find it at:
The README includes instructions on how to add it to your Kafka console command, and configure it with how to find your schema.
I won’t duplicate those instructions here, but I do want to briefly call out a few ways you can use it, depending on where your Avro schema is.
Schemas in Apicurio Schema Registry
If you keep your schema in an Apicurio schema registry the formatter can dynamically fetch the schemas it needs from the registry (just like your regular deserializer would).
Schemas in local files
If you keep your schema together with your application source code, and have it available in a local file, then the formatter can just read that local file directly.
Schemas in Event Endpoint Management
If you’re consuming events from someone else’s topic that you found in Event Endpoint Management, then the formatter can fetch any schemas it needs from the Catalog.
I hope you’ll find this useful.
Extra tips
Use a properties file and a shell alias
Save yourself some typing!
Instead of typing your schema registry URL and credentials in --property options every time you use kafka-console-consumer.sh, leave them all in a properties file and create an Avro-themed alias for kafka-console-consumer that points at your properties file.
Pair it with jq
The formatter outputs the parsed Avro-encoded events as a JSON string. This means that you can pipe it to something like jq for colour formatting and advanced parsing and processing.
This is very useful with large and complex event data, where you just want to view specific fields.