DataPower

 View Only
Expand all | Collapse all

Post processing error messages for API

  • 1.  Post processing error messages for API

    Posted Wed October 12, 2022 10:16 AM
    Hi - I need to do custom post-processing on all error messages generated by API Connect using Datapower API Gateway.
    I tested that I can not catch all errors with global error policy.
    I found out that on IDG.10.0.1.8 I can customize default API error rule and include custom steps before API Result Action.
    The issue is that when processing error the API Result Action is ignoring my response message (custom error) and instead generating its own error message.
    Is there a way to tell DataPower that it should process response as normal response - and not error? Some magic service variable?
    I tried
    • var://service/error-ignore
    • var://service/soap-fault-response
    But without luck.

    I actually managed to hack around this but it requires
    • function call from default error rule
    • function calling assembly
    • assembly throwing new error in main flow
    • custom error generation as default catch
    So it looks terrible and pollutes log with unnecessary error. On the other hand it proves that the processing can somehow switch back to straight response processing instead of error processing when using catch. So my question is how to switch the normal mode explicitly in gatewayscript/xslt?

    ------------------------------
    Grzegorz Abramczyk
    ------------------------------


  • 2.  RE: Post processing error messages for API

    Posted Fri October 14, 2022 09:14 AM

    Hi Grzegorz,
    This thread probably should be in the API Connect community, but I'll answer here.  I'm surprised that a global error policy is not able to catch all errors.  Which errors cannot be caught?  I'd think any error could be caught and you would update message.body which would be returned to the client.  That's a typical use case to customize the schema of the error message that is returned to the client.  You should open a PMR and include your global error policy for analysis.

    As for var://service/error-ignore and var://service-soap-fault-response, I wouldn't think those DataPower service variables would apply to an API Gateway Service.  You should work with variables in the API Gateway context, context.error, context.message.body, etc. 
    The API Result Action is simply going to take context.message.body and serialize it to the DataPower OUTPUT context.  Where does your custom error reside in the API Gateway context? 

    Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 3.  RE: Post processing error messages for API

    Posted Mon October 17, 2022 07:29 AM
    As for errors that do not get caught the error when parsing message is one example.

    I already had PMR but without much help.

    I experimented with API Connect context before moving to DP variables but without results.

    What I observed is that

    * When in "error processing" mode API Result Action ignores message and constructs own response (it determines if it should send json or xml, formats error message, sends response)
    * Going through throw/catch exits from "error processing" mode and causes message to be sent as response
    * No API connect or DP variable that I could found allows for exiting from "error processing" mode

    So does anybody know if there is any way to exit from "error processing" mode apart of throw/catch hack?



    ------------------------------
    Grzegorz Abramczyk
    ------------------------------



  • 4.  RE: Post processing error messages for API

    Posted Tue October 18, 2022 02:21 PM

    Hi Grzegorz,

    As I stated above, the only mechanism to modify the error response payload outside of a catch block is a global post-error policy.  Note if you catch an error, then the post-error policy will not see this error unless you rethrow the exception as the error is seen as "handled" in the catch.  You may consider that a "hack" but it is specifically designed that way, and has been that way for previous API Connect releases too.  This is what you are probably seeing.  You can use a global post-response policy that will be called for all responses that are not errors (or caught errors).  Perhaps you could set a context variable in your catch, or I would need to do a test to verify if the context.error object still exists in the post-response rule for a caught exception, check that in your post-response global policy and if set, do whatever additional processing you wish to do.  If not set, then you can leave message.body unchanged for any responses that had no errors.  Then the post-error global policy would handle all of the uncaught errors in a similar manner.

    Best Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 5.  RE: Post processing error messages for API

    Posted Tue October 25, 2022 04:32 AM
    I had some problems with my environment so I could not respond earlier.

    What I found out is (DP configuration commands referring to objects in parentheses)
    1. Catch/Default catch does not catch Parse Error (error from Parse Policy) as it shows default catch is not "catch-all" then I assume there may be more error that are not caught
    2. Global post-error policy is implemented the same way as default catch (assembly -> default-catch) it also does not catch Parse Error and likes
    3. Parse error (like any uncaught error is handled by API Collection Error Rule (api-collection -> error-rule)
    4. API Result (api-result) action from error rule is ignoring message and generating response based on error
    5. The only way to overcome this is as I wrote to function call -> assembly -> throw -> catch as part of default error rule - that way the api-result is thinking the error was caught and is sending the message that is prepared
    My main trouble is that the method in point 5 is convoluted and I wonder if there is easier way to do this.

    ------------------------------
    Grzegorz Abramczyk
    ------------------------------



  • 6.  RE: Post processing error messages for API

    Posted Tue October 25, 2022 10:00 AM

    Hi Grzegorz,

    After some testing I discovered that this design is to be compatible with v5.  In v5, the framework implicitly parsed the inbound payload based upon the provided content-type header.  If that parse of the request payload failed, then a "pre-flow" BadRequestError was thrown which was NOT catchable by the default catch.  You had to explicitly catch that error.  If that initial parse passed, then during the assembly if a subsequent policy, for example an invoke policy, received a response from a backend server that would not parse, then you would get a different type of error that was catchable, either in the default or specified catch.

    This behavior is repeated in the v10 API Gateway.  A parse failure throws a BadRequestError which is always catchable.  However, based upon the location of the parse, since the API Gateway doesn't do implicit parses and the API must specify explicit parse policies, the location of this policy would dictate which parse failure would be catchable by the default catch.  For example, here's my test API

    Policy 1 - Parse via content type

    Policy 2 - Set variable of message.body to "this text will not parse"

    Policy 3 - Parse via content type

    Passing in a content type of application/json with JSON data, the first policy passes but the third policy fails.  That is catchable in BOTH the BadRequestError and default catch.

    However, if the content-type is application/json with malformed JSON, the first policy fails.  This is catchable with the BadRequestError but NOT with a default catch.  

    I've not tried a global post-error policy, but I would like to think it would behave just as the catch did in my API.  To be on the safe side, I would recommend catching the BadRequestError explicitly.

    Best Regards,
    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 7.  RE: Post processing error messages for API

    Posted Wed October 26, 2022 11:33 AM
    Thank - you very much - I managed to catch BadRequestError with global error policy.
    To be sure I added handler for every "known" (suggested by API Connect UI) error, are there any errors that have similar special handling (not caught by default catch but caught by explicit catch)?

    Now I'm still struggling with next thing - catching errors from "pre" processing - I added prehook but it looks like I'm still not able to catch for example errors related to security (missing/invalid API Key) - this is important for me because I need to have specific format and signature over all error messages.

    Is there anything I can do to catch and handle those "special" errors?

    My prehook configuration looks like this

    global-policy: 1.0.0
    info:
      name: error-handler-pre
      title: Error Handler
      version: 1.0.0
      mode: after-builtin
    gateways:
      - datapower-api-gateway
    assembly:
      execute: 
        - gatewayscript:
            version: 2.0.0
            title: gatewayscript
            source: |-
              //
      catch:
        - errors:
            - ConnectionError
            - JavaScriptError
            - PropertyError
            - RedactionError
            - TransformError
            - RuntimeError
            - BadRequestError
            - SOAPError
            - OperationError
            - ValidateError
            - AssemblyRateLimitError
          execute: 
          - gatewayscript:
              version: 2.0.0
              title: gatewayscript
              source: |-
                context.message.body.write('Handled error - Pre');
                context.message.header.set('Content-Type', "text/plain");
        - default:
          - gatewayscript:
              version: 2.0.0
              title: gatewayscript
              source: |-
                context.message.body.write('Handled error - Pre Default');
                context.message.header.set('Content-Type', "text/plain");​


    ------------------------------
    Grzegorz Abramczyk
    ------------------------------



  • 8.  RE: Post processing error messages for API

    Posted Wed October 26, 2022 04:16 PM

    Hi Grzegorz, 

    Per https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=reference-error-cases-supported-by-assembly-catches, BadRequestError, UnauthorizedError, and ForbiddenError must be explicitly caught.  I don't see those errors in your sample catch.  Invalid/missing APIKey would be one of these UnauthorizedError (The API cannot be invoked based on the client ID and/or client secret provided, or id/secret were specified in the wrong location).  ForbiddenError would be for a disabled API.

    As for catching in the prehook, I'm not sure what your use case is.  Catching these errors and then not rethrowing them would allow the main assembly to continue which for an invalid/missing APIKey you wouldn't want to do.  Instead, you should consider a post-error global policy that would catch these errors to do any clean up, perhaps changing the schema of the error that is returned to the client, global logging, etc.

    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 9.  RE: Post processing error messages for API

    Posted Thu October 27, 2022 10:54 AM
    I had BadRequestError - the UnauthorizedError, and ForbiddenError are not suggested by API Connect UI so I missed it.

    As for catching pre errors - You are right - I was to fixated on assumption that main error handler is not catching those errors so I assumed it is only catch-able in pre section as it separate rule distinct from main processing rule.

    So to sum up to catch most (hopefully all) errors one should
    • Use global-policy-errors
    • Catch BadRequestError, UnauthorizedError, and ForbiddenError explicitly
    • Catch all other errors with default catch
    Am I right?

    ------------------------------
    Grzegorz Abramczyk
    ------------------------------



  • 10.  RE: Post processing error messages for API

    Posted Thu October 27, 2022 11:37 AM

    Hi Grzegorz,

    Yes, that would be my recommendation.

    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------