IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Protobuf definition for subdocument not present

  • 1.  Protobuf definition for subdocument not present

    Posted Thu December 15, 2022 03:29 AM

    Hello,

    in the Integration Server, I have a publishable document type (I’ll call it “MainDoc”) that is associated with a connection of type “Universal Messaging” (UM) and has the encoding type “Protocol buffers”. MainDoc has a field of type “Document Reference”. I’ll call the referenced document type “SubDoc”.

    SubDoc is not publishable. It was generated from an XSD and contains very many fields that are deeply nested. Some are document references themselves, others are defined inline.

    In the node.ndf file for the MainDoc, there is the property protoBufDescriptor which describes the document structure for the purpose of communication with UM. I decompiled this value (using com.google.protobuf.DescriptorProtos.FileDescriptorSet#parseFrom) and dumped the result to the console.

    Thus I noticed that:

    1. The message type for MainDoc has a field “SubDoc” of type “TYPE_MESSAGE”
    2. There is a message type “SubDoc”
    3. The definition of the message type “SubDoc” contains no fields

    How can it happen that the definition of “SubDoc” is empty? I’d expect that the protobuf definition contains a complete definition of MainDoc, including all its subdocs and their fields.

    MainDoc works well, can be published and received in the IS, the data is transported correctly etc. But where is the protobuf definition for SubDoc??? (Actually, there are three fields in MainDoc whose definition is empty.)

    Thank you for any hints.


    #Universal-Messaging-Broker
    #webMethods
    #Integration-Server-and-ESB
    #Flow-and-Java-services


  • 2.  RE: Protobuf definition for subdocument not present

    Posted Thu February 23, 2023 06:36 PM

    Newer versions of IS include the human-readable protocol buffer definition in the package namespace, so there should be no reason to decompile the descriptor. For example, a document called test:mainDoc in package Default will contain two files in \packages\Default\ns\test\mainDoc, node.ndf and pdt.proto. The latter is the protocol buffer definition. I just ran a quick test in IS 10.11, and I see the subDoc in the pdt.proto file. If you have a newer version of IS, you may want to try the same test.

    As for missing some fields in the definition, that is expected because some field names are illegal in protocol buffers. This does not mean you cannot send and receive these fields. It just means they cannot be read directly via the protocol buffer APIs. This only impacts filtering on UM.


    #webMethods
    #Universal-Messaging-Broker
    #Integration-Server-and-ESB
    #Flow-and-Java-services


  • 3.  RE: Protobuf definition for subdocument not present

    Posted Fri February 24, 2023 02:30 AM

    Thank you @Derek_Rokicki for the info. I’m on an older IS version and hence have to decompile the definitions myself.

    But I’ve already solved the problem. The statement “they cannot be read directly via the protocol buffer APIs” is half true. The truth is that they can be read (otherwise the IS would not be able to read them as well), just with some tricks. The keyword here is “unspecified fields”.

    If you’re familiar with protobufs etc, it would be cool if you could answer my question from another thread. Thank you!


    #Integration-Server-and-ESB
    #Flow-and-Java-services
    #Universal-Messaging-Broker
    #webMethods