Maximo

 View Only
Expand all | Collapse all

Outbound Kafka message compression in 7.6.1.3

  • 1.  Outbound Kafka message compression in 7.6.1.3

    Posted Wed October 05, 2022 02:38 PM
    Hi,

    We've been using Kafka message producer in 7.6.1.2 and the message format is as documented in Message processing using a Kafka provider - IBM Documentation

    We've now installed 7.6.1.3 fix pack to our test environment and noticed that the outbound Kafka messages have changed. With 7.6.1.3 it seems that the whole of the message is compressed and the payload property contains the payload as an escaped JSON string.

    Are there any properties that can be used to configure the compression scheme of Kafka messages? We would prefer to keep the previous format to avoid changes in the kafka broker.

    ------------------------------
    Antti Kalliomaki
    ------------------------------



    #MaximoIntegrationandScripting
    #AssetandFacilitiesManagement
    #Maximo


  • 2.  RE: Outbound Kafka message compression in 7.6.1.3

    Posted Fri November 04, 2022 03:40 AM

    Received a response from IBM Support to this question:

    There is no setup for having a cleartext format in the queue. The change was done when we faced issues with bytes size in the topic for bigger messages.
    Dev team are not clear if you are trying to access the Kafka message directly from the Kafka broker or letting Maximo cron's access it. The latter is the way this flow is designed - and in that case the compressed message is uncompressed and pushed forward to its final destination.
    We would definitely want to consider this enhancement to Kafka processing for clear text messages in upcoming releases - but will not be able to backport it.


    In our pattern we're not consuming the message in Maximo again to push it to a final destination. The message is consumed directly from the broker by an external application.

    Our issue was solved by adapting the downstream components to the new message format / compression scheme.



    ------------------------------
    Antti Kalliomaki
    ------------------------------