DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

How to Increase JSON length for particular API service on DataPower

  • 1.  How to Increase JSON length for particular API service on DataPower

    Posted Tue July 26, 2022 09:09 AM
    Hello Team,

    Please guide  how to increase JSON length for one API service on DataPower.
    Please find below version of API connect and DataPower.

    API connect: V5
    DataPower: IDG.2018.4.1.6

    Thanks and Regards.

    ------------------------------
    Jyoti Yadav
    ------------------------------


  • 2.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Tue July 26, 2022 09:21 AM
    With the JSON Settings?   Consider implementing a custom policy and check the payload length manually.

    ------------------------------
    Joseph Morgan
    ------------------------------



  • 3.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed July 27, 2022 05:16 AM

    Hi Jyoti,

    You can the the increase the length of the payload by modifiying xmlmanger settings 

    update the xml manger of ther service with JSON Settings 

    either you can create a new xml manger specific to this service and edit the add new JSON Settings with the size limit that you would require





    hope this is what yiou are looking for.

    Thanks & Regards

    Avianash



    ------------------------------
    Avinash Mohan Vundale
    ------------------------------



  • 4.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed July 27, 2022 12:45 PM
    Avinash,

    Though that will work for a brief time for the JSON settings for the webapi XML Manager for API Connect, it will just be overwritten.  Jyoti will have to use a custom policy.


    ------------------------------
    Joseph Morgan
    ------------------------------



  • 5.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed July 27, 2022 09:18 AM

    Hi Jyoti,

    Just a friendly reminder that API Connect v5 is now out of support effective April 30 of this year and you'll need to be migrating to a supported API Connect version.  Some customers have purchased extended support, but the maximum v5 can be supported under that extended support is the end of this year.  The API Connect Migration tool is available to support your migration from v5 to v10, either in v5 compatible mode, or using the native DataPower API Gateway Service.

    Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 6.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed July 27, 2022 09:24 AM

    Hi Jyoti,

    yes we can do it to specific servic . We just need to create a new xml manger specific to this service and then you can modify the settings accordingly.

    Thanks

    Avinash 



    ------------------------------
    Avinash Mohan Vundale
    ------------------------------



  • 7.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed July 27, 2022 12:49 PM
    Avinash,

    Even then, Jyoti's request is to customize JSON limits by URI, which is not possible for either an XML Manager or JSON Settings (IBM - Are you listening??? Maybe a good RFE).

    That is, we cannot, with API Connect, customize the XML Manager nor the JSON Settings per URI, which is what needs to be done.

    So, the JSON Settings for the API Connect will need to be very forgiving, and a custom policy will have to be created to control the size limits in that environment.

    ------------------------------
    Joseph Morgan
    ------------------------------



  • 8.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed July 27, 2022 12:55 PM
    Hi Joseph,

    I belive Jyoti was asking about service that was hosted on Datapower which will invoke a service reciding on API connect. in this case we can do xml manger chnages as above and set the limit of jason paylod thus Jyoti seems able to do and achive the requirment.

    Thanks
    Avinash

    ------------------------------
    Avinash Mohan Vundale
    ------------------------------



  • 9.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed July 27, 2022 01:58 PM
    If that is the case, then, yes, Jyoti can setup a custom JSON settings object just for that one service.

    ------------------------------
    Joseph Morgan
    ------------------------------



  • 10.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed August 03, 2022 08:39 AM
    Hello Avinash,

    We have created mpgw service & configure static backend service. We have set xml manager service & within that service set json payload length. Policy set as default but still getting same json length exceed  error.

    Please guide us how we can resolve this issue. Created xml manager but still getting same issue.

    ------------------------------
    Akshay Sawant
    ------------------------------



  • 11.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed August 03, 2022 09:12 AM
    Akshay,

    So, you have something like this reference tree:

    MPGW ->  CustomXMLManager -> CustomJSONSettings

    And your JSON settings is set to some value.   You are getting JSON limits exceeded error.   What are you JSON length settings and what is the JSON payload size?

    ------------------------------
    Joseph Morgan
    ------------------------------



  • 12.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Wed August 03, 2022 12:33 PM

    Hi All,
    There is a lot to catch up on with this thread. 

    1. Although this is a DataPower forum this thread started out asking about API Connect v5 which was implemented as a MultiProtocol Gateway Service which references an XML Manager which references JSON settings.  The only place where the JSON settings is enforced in a MPGW would be on a HTTP Convert action with JSON encoding which produces a JSONx output.   APIC v5 uses this action always on the request rule for JSON requests and optionally on the response rule based upon an API property being set to true to enforce response JSON payload sizes.  If in GatewayScript, a JSON.parse(someJSONstring) doesn't use these limits as the parse is being done by the JavaScript engine which doesn't have knowledge of the DataPower JSON settings.  If any MPGW has a request or response type configured as JSON, then I believe this HTTP Convert with JSON encoding is happening "under the covers" in a similar manner as the request or response type would be configured as XML which would use the XML parser settings.  As noted, the XML Manager / JSON settings are service wide settings with no ability to customize how a particular HTTP Convert action within the service behaves, so it needs to be configured to the maximum sizes that would be allowed for all  requests coming into the service.   As for submitting RFEs, they are always welcomed and encouraged.  https://integration-development.ideas.ibm.com/?project=DPGWY for DataPower and https://integration-development.ideas.ibm.com/?project=APICONN for API Connect are links where you can submit your ideas.
    2. There was a comment about manually updating the XML Manager settings and a subsequent comment about them being overwritten.  Again I assume this is an API Connect concern about updating a v5 object such as the webapi XML Manager or JSON settings object, only to have them redeployed to the default settings when you would deploy a new DataPower Gateway or upgrade  an existing one.  In this case, to allow the updated settings to be preserved in this case, you should utilize a Gateway Extension that would deploy your updated XML Manager or JSON settings upon deployment/upgrade.  The Gateway Extension can be used in a couple ways, but its simplest form is a DataPower export that would contain DataPower objects that would be deployed after the base framework has been deployed, thus you could have the webapi xml manager and json settings objects as the only two object in this zip file.  You'd associate the zip file as the Gateway Extension using the API Connect CMC.
    3.  As for the most recent question, I'd add to Joseph's questions what specific error are you getting? You have the overall document size but also the value of a specific property that could also be an issue.  Also, what is meant in comment 10 by "Policy set as default"?
    Regards,
    Steve

    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 13.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Thu August 04, 2022 12:16 AM
    Hello Steve & Joseph,

    Thanks for your replay.
    Please find attachment for json payload configuration on DataPower & error screenshot.
    We are using 33.18 kb request payload size & getting "json exceed error".
    Please find attachment & advice for the same.

    ------------------------------
    Akshay Sawant
    ------------------------------



  • 14.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Thu August 04, 2022 03:47 AM
    Hello Akshay,

    Can you please check the length of the JSON paayload and modifiy the settings accordingly. I would suggest to give a buffer length as well. 
    then give a try also enable probes on this service and see whats the error you are getting it could be something else as well.

    Thanks & Regards
    Avinash


    ------------------------------
    Avinash Mohan Vundale
    ------------------------------



  • 15.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Thu August 04, 2022 10:12 AM
    Yes, so, the issue appears to be around the "Maximum value length for strings".  How many characters are in the "DecryptedGCM" string?

    Your maximum type length for String fields is set to 819,200.   What is the length of the string in "DecryptedGCM" and other string type fields in the JSON?




    ------------------------------
    Joseph Morgan
    ------------------------------



  • 16.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Fri August 05, 2022 01:06 AM
    Hello Joseph,

    Please find below information for the same

    How many characters are in the "DecryptedGCM" string

    => 33.18Kb

    What is the length of the string in "DecryptedGCM

    => 33.18Kb

    string type fields in the JSON?

    => Object






    ------------------------------
    Akshay Sawant
    ------------------------------



  • 17.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Fri August 05, 2022 11:00 AM
    Any way you can share the JSON with the group or with me privately? 
    There's something in there and I suspect if we can see the actual content we can help.
    Just obfuscate any sensitive data with the same number of characters.  That is, if you have a really long field, just put it in as:

    "{ "fieldName": "xxxxxxxxxxx.......all the way to the end but the same size.........xxxxxxxxxxxxxxxx" }"

    ------------------------------
    Joseph Morgan
    ------------------------------



  • 18.  RE: How to Increase JSON length for particular API service on DataPower

    Posted Fri August 05, 2022 03:13 PM
    Thanks for the content.  Does the processing policy decode the base64 within the "DecryptedGCM" value into JSON?

    ------------------------------
    Joseph Morgan
    ------------------------------