IBM Event Streams and IBM Event Automation

 View Only
Expand all | Collapse all

Fetch history of messages from Event streams topic

  • 1.  Fetch history of messages from Event streams topic

    IBM Champion
    Posted Thu December 02, 2021 01:30 AM
    Hi ,

    I have a requirement to query all the history of messages in an event streams topic, is there a way to get all the messages in a topic in a single query using ACE V12?

    ------------------------------
    Thanks,
    Santhosh Ramanathan
    ------------------------------


  • 2.  RE: Fetch history of messages from Event streams topic

    Posted Thu December 09, 2021 09:06 AM
    Hello,

    It's not possible to get all messages from an event stream topic in one call using any of the provided ACE nodes, but, assuming you want to do this mid flow, you could make use of the kafka read node in ACE with some kind of transformation node before it to propagate out from and read each message in turn.
    Details about kafka in ACE can be found from this doc topic: https://www.ibm.com/docs/en/app-connect/12.0?topic=messages-using-kafka-app-connect-enterprise
    and in particular the kafka read : https://www.ibm.com/docs/en/app-connect/12.0?topic=nodes-kafkaread-node

    With a transformation node before the Kafka read node you could setup the topic, partition, and offset to read from in a loop and then propagate out to the kafka read and in a node after the Kafka read process the response before returning back to the transformation node and going to the next iteration of the loop to read from the next offset. You would then also need to wire up the no_match terminal of the kafka read node to then set something up to say that you've reached the end of the topic and so signify to end the loop.

    Thanks
    David

    ------------------------------
    David Coles
    App Connect Developer
    ------------------------------

    Original Message