Hi everyone,

I'm working on implementing a feature to detect duplicate keys in JSON payloads within IBM API Connect. I've developed a JavaScript function that parses JSON strings to find duplicate keys, but I'm facing challenges integrating this into API Connect policies or actions. Does anyone have experience with custom validations or similar implementations in API Connect? Any guidance or references to documentation would be greatly appreciated.

Thank you!

Hi Soulaiman,
The actual parsing of JSON strings into a parsed object will not support duplicate keys.  The specification indicates the handling of duplicate keys is solution dependent, and I believe the last duplicate property will be what is in the parsed object in API Connect.  The API Gateway doesn't do implicit parsing so as long as your code is doing its own parsing of a JSON string I wouldn't see an issue to doing it.  What type of challenges are you encountering?
Regards,
Steve

Hello Steve,

First of all, thank you for the prompt response. The challenge I am encountering is that when I use apim.getvariable("message.body") inside of a gatewayscript, I get an already parsed object. This method of retrieval does not help in my case since I'm looking to work with the raw JSON string specifically to detect duplicate keys before it's parsed into an object, where duplicate keys would be inherently lost or overridden based on the last key-value pair encountered, as per JSON specification.

In your experience, is there a way within IBM API Connect to intercept the raw JSON payload before it's parsed? My goal is to apply my JavaScript function directly to the raw JSON string for duplicate keys detection. I suspect I might be missing something in terms of API Connect's capabilities for handling pre-parsed payloads or perhaps there's a specific policy or action that allows for such an operation that I'm not aware of.

Thank you in advance

Hi Soulaiman,

Which gateway type are you using?  the API Gateway or the v5 compatible Gateway?

If v5 compatible, the framework implicitly parses all JSON payloads.  However, you can still get the original unparsed request payload by using the apim.readInputAsBuffer function if you do so at the beginning of the policy before a message.body payload has been created.   If you're really wanting message.body where message.body was created by another policy, for example an invoke policy, then the v5 implicit parsing of the invoke JSON response will have already parsed that response payload before placing it in message.body and it will be too late to use your function.

If you're using the API Gateway, your current issue is your use of the apim.getvariable function.  As noted earlier, v5 implicitly parsed JSON (and XML) payloads, and the apim.getvariable function would thus always return a parsed payload in those cases.  In the API Gateway, the apim module is a "v5 compatibility" module that will allow GatewayScript written for v5 to run unchanged in the API Gateway.  Thus it is assumed that code following the apim.getvariable in your GatewayScript would expect the payload in message.body to be parsed as it was in v5, and so that is what it returns.  Instead, one approach would be to use a context.get('message.body').  This has no v5 compatibility assumptions, and if the body is actually in context as a buffer then that is how it will be returned.  One caveat on the use of context.get for message.body:

The API Gateway streams request.body by default.   It will also stream an invoke response written to somename.body, including message.body, by default.  In these cases, a context.get('message.body') will return null since your code is running while that response may still be in flight.   Since you don't want to do a parse policy, which will read a payload off of the stream but would then parse the JSON payload, you will need to either
1. use a context.message.body.readAsBuffer(function(error, buffer) {});  This would read the buffer from the stream and you can have your code use variable buffer, or buffer.toString(), in the asynchronous function to have your parser check for duplicates.
2. in your api set x-ibm-configuration.buffering  to a boolean of true.  This will disable streaming for the API, which would have performance impacts to the API, but context.get('message.body') will now have access to the buffer in context before your code starts and will return the buffer instead of null.
Regards,
Steve