Issue Summary: Generic Validation Errors for Schema Validation Failures in APIs

We have observed that when backend responses fail JSON schema validation in API Connect (APIC), the error handling does not provide sufficient details in the error context (context.get("error.message")). Instead of capturing the complete schema validation error details, the error context logs a generic message:
"Validate: Internal Validation Error."

As per feedback from IBM this is the expected behavior, and I am not convinced with this response. I would like to know if anyone has faced similar issue and how they managed to resolve it.

Observations:

1. Error Context Behavior:

   • The error context only logs the generic message, making it difficult to identify the root cause.

2. Detailed Error in Gateway Logs:

   • While the error context logs a generic message, the DataPower Gateway pod logs capture the full error details. For example:

     
     20241118T233504.129Z [apiconnect][0x80c00010][xslt][error] apigw(apiconnect): tid(48256)[request][11.103.62.2] gtid(b4eccc98673bcf240000bc80): 
     Processing of 'temporary:///swagger/dhhs_esit_view-benefits-alpha-api_alpha.json' stopped: temporary:///swagger/dhhs_esit_view-benefits-alpha-api_alpha.json:484: 
     [JSV0003] Invalid array: the minimum number of items in the array must be at least 1 (got only 0).
     
     
     

     Additionally:

     
     
     20241118T233504.129Z [apiconnect][0x01d30003][multistep][error] apigw(apiconnect): tid(48256)[request][11.103.62.2] gtid(b4eccc98673bcf240000bc80): Schema Validation Error
     
     
     

3. Impact on Error Logging and Analysis:

   • We have a audit logging policy that gather the error information from the error context and stores the error information in a database. However, due to the generic error context being logged, the database contains insufficient details for production support teams to analyze and resolve the issue effectively.

Problem Statement:

The lack of specific error details in the error context (e.g., context.get("error.message")) during certain schema validation failures hinders efficient troubleshooting. Although the full error details are available in the DataPower logs, these are not directly accessible through the APIC error context.

I raised a PMR with IBM and below are the replies. They haven't provided me with any workable solution.

Reply - 1

Below is the response from our SME support team.

Unfortunately, the full error is not going to be in the response which is what you will get with the "context.get("error.message")". This is 'Working as Designed'. We do not guarantee that response will contain sufficient error information in the response. Log usually has more information but even that is not always the case.

Reply - 2

So this is the validate documentation. https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=policies-validate-datapower-api-gateway. It does not promise accurate error message. There is unfortunately no way to handle exception IF the exception handling relies on accurate error messages.

If the exception is just generic validate error, then you can either use a catch in assembly to catch validate error or check the status code in Gatewayscript. it should be a 422 Status code.

Hi Ravi,

This will be my last forum reply as I'll be retiring from IBM at the end of the month after 44 1/2 years of service.

If the request payload is validated and fails, the issue being with the client provided payload, a HTTP 422  is returned with the error detail.  The client provided data that does not match the anticipated schema and they will need this detail to fix their application to provide the correct schema of data.  However, if any other payload is validated and fails, the client has no idea what the payload is or what the schema should be.  For example, your API could have a map policy that is supposed to create a message to be sent to some backend server, but there is an error in the map policy and the payload is incorrect.  Another example, a backend server returns a response, but the schema in the API does not match that backend response.  In either case, a HTTP 500 Internal Server Error is returned to the client since it isn't the client's fault, but either the API or the backend server's fault.  As you note, the actual error detail is in the DataPower log, but the error object and the default error response in this case contain a more generic error.  It would be for the API developer to investigate, using the DataPower logs, but the client is not provided that detail which is why you're seeing the generic data.  The only answer I can think of in this case is a request for enhancement to provide some type of toggle that would allow the validate policy to set the error detail for these non request validation errors, but the current implementation is working as designed.  Of course, no promises that I can make as to if or when such an enhancement would be implemented.

I wish you and all of the forum participants the best in your use of DataPower and API Connect.  It has been my honor to provide some assistance to you through this forum these many years.

All the best ...
Steve Linn