API Connect

API Connect

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

  • 1.  Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Mon August 05, 2024 03:16 AM
    Edited by Kumar . Tue August 06, 2024 07:44 AM
    We have an API called TokenAPI(POST Method) to generate the Token.
     
    We have two other REST APIs(Both are GET), they should get the token from TokenAPI and set the token in the http header when calling the actual endpoints of these services.
    One API i.e. Test1API, we have to set http header as "apikey: <token>" and other i.e. Test2API "Authorization: Bearer <token>".
     
    Now, the token expire as 
    "expires_in": "7199"
    In the invoke of REST API, we configured the API Eddpoint of TokenAPI and then reading the token and set as a Authorization header by writing the
    gateway script.
    Test1 API is successfully called the TokenAPI and .js did it's job and Test1 API called it's actual backend with the http header i.e. apikey and got expected response.
    1) Coming to Test2 API, here we could see that token is coming and .js is working fine but the final response we are receiving is as below:
     
    {
        "fault": {
            "faultstring": "JSONThreatProtection[JSON-Threat-Protection]: Execution failed. reason: Expecting { or [ at line 1",
            "detail": {
                "errorcode": "steps.jsonthreatprotection.ExecutionFailed"
            }
        }
    }
     
    When I check APIC logs, there are two api calls under the same global-transaction-id. One is internal token call that is success and other one is TEST API2
    with failure 500. I am getting this error "steps.jsonthreatprotection.ExecutionFailed response" from the backend
     
    So, When I hit the TEST2 API by passing the header manually through postman/soap ui then I am getting valid
    response. But when I make Internal token call then I am getting this error.
     
     
    2) Also, Is there any way to store the token since expiration time of the token is 7199, we don't want to call Token API everytime. 
    Can we store the token in a cache or any variable to use it for further calls if we have receive any calls within the token expiration time?


  • 2.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Mon August 05, 2024 04:27 AM

    Hello All,

    Could anyone help me on this with your thoughts?



    ------------------------------
    Kumar
    ------------------------------



  • 3.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Tue August 06, 2024 03:08 AM

    Hello All,

    Can anyone please help me on this issue? I got stuck at this moment.



    ------------------------------
    Kumar
    ------------------------------



  • 4.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Tue August 06, 2024 05:56 PM

    Hi Kumar,
     Just looking at the error message, it would appear that some process is expecting a JSON payload and somewhere in its validation of that payload there is a validation error.  I'm not aware of a JSONThreatProtection policy within APIConnect, but a quick internet search showed that there is a competitive product that has a policy by that name. It is hard to diagnose your issue with just the information you've provided, but my best guess with the limited information is that API2 must be doing an invoke to some backend and passing a payload that is failing validation.  If you can execute API2 directly from the API Manager test tab, you can get a trace on the test tab and be able to look at the context for message.body for the invoke policy action to see exactly what is being sent.  Either it is malformed JSON or perhaps isn't JSON at all.  Unfortunately, the faultstring doesn't even provide an offset to know where the issue is.  If you can't use the test tab for API2, what type of platform are you using?  If you by chance have a virtual or physical DataPower, a packet capture to your API2 backend requesting tls session keys so the traffic can be decrypted would be very useful to see what was sent to the backend and hopefully you'd see the above response.
    Best Regards,
    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 5.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Wed August 07, 2024 08:08 AM

    Hi Steve,

    Thanks for your Inputs. I have tried through test tab and nothing is coming in the trace except the same error message.Now, we are only left with packet capturing and we will do.

    Meanwhile, Let me explain what we have been trying to achieve.

    Case 1)Test_API1 --> Invoke(Token API which is another API service hosted on same APIC Manager)  --> Gateway script(get access-token from json response and set apikey as http header)-->Invoke( Actual backend). 
    This scenario is working fine without any issues.

     Case 2)Test_API2-->Invoke(Token API which is another API service hosted on same APIC Manager)--> Gateway script(set Authorization Bearer as http header)-->Invoke( Actual backend)-->Parse

     This scenario is giving error. I though it would fail at response parse but it didn't fail. So, whatever the error response we are receiving that is clearly coming from the backend as per datapower and apic analytics logs.

    So, we have  created another Test_API2 with another version(2.0.0) there I only configured the Invoke with same backend that I have configured in Case2 Second Invoke. I hit the api endpoint by passing Authorization: Bearer Token manually then backend has given valid response.

    So, I am not getting what exactly am missing when we call Case2? 

    2) Also, Could you please tell us best way to store the token. Since the expiration time of the token is 7199, we don't want to call Token API every time. 
    Can we store the token in a cache or any variable to use it for further calls if we have receive any calls within the token expiration time?


    ------------------------------
    Kumar
    ------------------------------



  • 6.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Wed August 07, 2024 09:47 AM

    Hi Kumar, 
    For case 2, it seems that your backend is failing on the validation of the payload it is being sent.  You may have the same bearer token in both cases, but what is in message.body in both cases?  That is assuming of course that for the invoke policies of your two API2 APIs, that each invoke is doing a POST method.  The contents of message.body will be the request payload sent to your backend.  Perhaps you could have a GatewayScript prior to each invoke that would get message.body and simply log it for examination if you have difficulty doing the packet capture.

    As for how to store your token, I would suggest the use of a relatively new DataPower feature called distributed variables.  These variables are shared between multiple appliances all within the same peer group, thus the same token can be utilized on each appliance although it would have been obtained on the one appliance of the group that communicated with the token backend server.  You can read about distributed variables at https://www.ibm.com/docs/en/datapower-gateway/10.6?topic=administration-distributed-variables Note that the distributed variable feature was introduced in DataPower firmware 10.6.  I would suggest in your case using a xslt stylesheet, at least for saving your token, as the dp:set-dist-variable extension element (See https://www.ibm.com/docs/en/datapower-gateway/10.6?topic=elements-dpset-dist-variable) supports a time to live attribute (expire) that the equivalent GatewayScript capability doesn't support at theis time.  Since your token expires in 7199 seconds, you could set the distributed variable expire to something less, say 7170 seconds.  When you attempt on a subsequent transaction to get this distributed variable, if it is expired you'll not get a result which would tell you to hit your backend to get a new token and save a new token in the distributed cache for another 7170 seconds.

    Best Regards,
    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 7.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Thu August 08, 2024 03:14 AM
    Edited by Kumar . Thu August 08, 2024 08:01 AM

    Dear Steve,

    Thank you for your response.

    Case 2: We come to know that  there are some issues with api-probe settings and we have to fix it to see the trace.  Then we will log the message and  see what's actually coming.

    Coming to storing the token, Our DataPower is on 10.5.0.11 and APIC is on 10.0.5.7. Since you confirmed distributed values comes up with DataPower 10.6, Is there any way to achieve this in with APIC 10.0.5.7 and DP 10.5.0.11?



    ------------------------------
    Kumar
    ------------------------------



  • 8.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Wed August 14, 2024 04:58 AM

    Hello @Steve Linn,

    Clearing the content-type after first invoke, Now we were able to receive the expected response. 

    Coming to storing the token, As per the below information we can configure the Invoke with Time to Live and specifying the value of token expiry time. Can we consider this option?

    https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=invoke-configuring-policy-datapower-api-gateway



    ------------------------------
    Kumar
    ------------------------------



  • 9.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Wed August 14, 2024 09:14 AM

    Hi Kumar,

    The invoke policy caching options will only work if your request does not have an Authorization header.  Given it is returning a security token of some type I had made the assumption that you do have an Authorization header being sent to your backend, which is why I suggested the side call service, so yes, you could use this option on the invoke policy but you'd need two separate services, for example

    1. API 1 invoke sends a request to API2 with the invoke TTL specified.  Any request headers are send to API2 using your own named headers, NOT Authorization.
    2. API2 takes those named headers, builds the Authorization header, and contacts your real backend, gets the response and returns it to API1.

    Now API1's communication to API2 will be cached.  API2 will always communicate to your authentication server, but it will be called less frequently because of API1's invoke cache policy.

    Best Regards,
    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 10.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Thu August 15, 2024 02:05 AM
    Edited by Jirayu Kaewprateep Thu August 15, 2024 02:07 AM

    Dear Steve,

    May I know about the API1 and API2 endpoints, I assumed that they are on different internet address endpoints and that the re-negotiation message happened during the high-priority service request for the active job. 

    How long does the cache policy expire date time? After the maintenance services application can perform the same process negotiation with the same scenarios? Can we know when the cache policy is active? How do we identify the cache policy involved in communication from regular messages?

    It is a good service and I would like to study more about the API gateway more...

    Thank you

    - - -

    JSON dataset



    ------------------------------
    Jirayu Kaewprateep
    ------------------------------



  • 11.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Tue August 20, 2024 07:08 AM

    Hi Jirayu,
    Your current API is API1.  It is doing an invoke to some server that is returning an authentication token, and you are asking how to "cache" that token so you don't go to that server for every request.  I'm suggesting that instead of API1 using that authentication server as it's back, that you would use another service as your backend.  It could be a MPGW server, or it could be another API endpoint, which is what I was calling API2. This other service/API would interact with the authentication server and would return the token in its response.

    API1 has caching properties in its configuration, so you're in control.  If you choose a cache type of "Time to Live" then you provide a fixed time.  You could also choose a "Protocol" type which would then require that your MPGW/API2 return the Cache-Control response header that would control the cache duration.  Other headers may also apply, but see https://www.ibm.com/docs/en/datapower-gateway/10.5.0?topic=manager-defining-document-cache-policies for the rules.

    Specific answers to your questions:

    > How long does the cache policy expire date time?
    As I indicated, fixed with TTL, or by Cache-Control response and other headers if protocol type.

    > After the maintenance services application can perform the same process negotiation with the same scenarios?
    I'm not sure of your question here, please elaborate.

    > Can we know when the cache policy is active?
    There is a DataPower status provider that will show the currently active documents in the cache

    > How do we identify the cache policy involved in communication from regular messages?
    If you're asking if you'll know if the document read was from the backend or from the cache, generally you will not know.  You do an invoke to a http(s) backend and you get a response back.  Whether it came from the cache or was the actual backend call that was cached isn't known.

    Hope this helps,
    Best Regards,
    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 12.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Tue August 20, 2024 07:24 AM

    Dear Steve,

    Thank you for your answer, I question these because I have experience with the API-implemented architecture with different endpoints of API1 and API2.

    [Jirayu]: After the maintenance services application can perform the same process negotiation with the same scenarios?
    [Steve]: I'm not sure of your question here, please elaborate.

    [Jirayu]: After restarting our system for maintenance application previously working with API endpoint version 1 will try to connect to API endpoint version 2 because both are identified in the configuration and it creates concurrent sessions. Are there any limited of the number of concurrent sessions or a number of error retires?  

    [Jirayu]: Can we know when the cache policy is active?
    [Steve]: There is a DataPower status provider that will show the currently active documents in the cache

    [Jirayu]: That is a good tools monitoring status.

    [Jirayu]:  How do we identify the cache policy involved in communication from regular messages?
    [Steve]: If you're asking if you'll know if the document read was from the backend or from the cache, generally you will not know.  You do an invoke to a http(s) backend and you get a response back.  Whether it came from the cache or was the actual backend call that was cached isn't known.

    [Jirayu]: That is exactly my question do you have any option for update document or start query for new request for same resultset?

    Thank you

    - - -

    I am studying and learning from online courses and this is my hobbies

    Data Warehouse


    ------------------------------
    Jirayu Kaewprateep
    ------------------------------



  • 13.  RE: Getting "steps.jsonthreatprotection.ExecutionFailed" when we call another api(Invoke) inside the assembly

    Posted Tue August 20, 2024 02:44 PM

    Hi Jirayu,

    API2 should be an internal API only and never called by an external client.  If that API could be exposed, then perhaps having a different service endpoint such as a DataPower multi protocol gateway service accessible only as a local host IP would be preferable.  The purpose of this additional hop is to allow the document caching to work between API1 and its backend as the document cache feature will not cache any backend response where the request has an Authorization request header.  I don't understand you comment on how some application would change from connecting from API1 to API2.  As for concurrent sessions to a particular API, other than the resources of the DataPower appliance itself, there is no specific limit.  By error retries, are you asking about an authentication failure?  Only HTTP 200 responses will be cached, so continual failing requests will always be sent to the authentication server.

    As for cache management, you can administratively flush documents from the cache, and you can also remove documents from the document cache programmatically.  See https://www.ibm.com/docs/en/datapower-gateway/10.5.0?topic=functions-api-gateway-extension and specifically the document cache specific functions.  As for determining if the response is cached or not, the only thing I can think of is to test if some response headers are present on the actual backend response and not present on the cached response.  I'd think they would be the same, but perhaps for protocol based caching, one could be missing such as Age.  Just speculation on my part as testing would need to be done to verify that, thus my base assumption that you can't tell the difference.

    Best Regards,
    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------