API Connect

API Connect

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  How does the cache work in invoke policy

    Posted Mon September 09, 2024 03:52 AM
    Edited by Elshaday Teshome Mon September 09, 2024 03:54 AM

    I need to cache an access token from oauth endpoint and use the access token to make a call to a backend endpoint. How do I pass the cached access token to the next invoke policy? I am using API Connect saas and the basic plan.



    ------------------------------
    Elshaday Teshome
    ------------------------------



  • 2.  RE: How does the cache work in invoke policy

    Posted Thu September 12, 2024 10:03 AM
    In the Invoke policy where you are invoking token generation api to get the token, configure below fields
     
    Cache Type --> Time to Live
    Set Time to Live (value in seconds)
    Cache Key - Any name such as "ApiToken"
    Response object variable - Any name such as "TokenAPIResponse"
     
    Now, as a next step, before you invoke actual backend operation, populate the required token header using map/set policy or gateway script. For example
     
    message.headers.token = TokenAPIResponse.body.access_token
     
    The invoke policy behaves in such a way that, it checks for the cached data in "TokenAPIResponse" using key "ApiToken", if it is available then it will not call the token generation and proceed with next steps in the api assembly.
     
    You can verify whether your data is cached or not in datapower console and try to look for your token key "ApiToken", it will be displayed along with remaining TTL.



    ------------------------------
    uvaise odam
    ------------------------------



  • 3.  RE: How does the cache work in invoke policy

    Posted Thu September 12, 2024 11:00 AM

    Hi Elshaday,
    In addition to what Uvaise indicated, you can also use a protocol based caching where the request and response headers will dictate if the value is cached and what the expiration TTL would be.  See https://www.ibm.com/docs/en/api-connect/10.0.5.x_lts?topic=invoke-configuring-policy-datapower-api-gateway for a description of the invoke policy properties, and in particular, if using the protocol based caching, see the RFC specification at https://datatracker.ietf.org/doc/html/rfc7234.  Note that no caching will be attempted if your invoke request has an Authorization HTTP Request header.

    Having said all of that, re-reading your question, I'd assume that your request to the OAuth endpoint requires an Authorization header, so the response from that endpoint will not be cached by the invoke policy.  The only workaround I can think of would be to use a separate API to do your OAuth endpoint request as a "side" call from your actual API, particularly since you are on SaaS using a basic plan and don't have access to GatewayScript or XSLT policies where another mechanism could be considered.  For example,

    Some API -> Set Variable message.headers.myAuthorization to whatever you would want to provide to the OAuth endpoint and exclude the Authorization header if present -> Invoke to OAuth API  that specifies caching

    OAuth API -> Set Variable message.headers.Authorization to the value in message.headers.myAuthorization -> Invoke to OAuth endpoint excluding the myAuthorization header and no caching

    So your "Some API" will have your OAuth API response cached, and hopefully will not call the OAuth API except when the cache has expired.  Do you have the same Authorization header for all of your requests?  If not, then you should also specify a cache key in your invoke policy properties to provide additional information that uniquely identifies this OAuth request from another, ie a user/client id, so you'd now have multiple responses cached, one for each user/client id.

    Best Regards,
    Steve Linn



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 4.  RE: How does the cache work in invoke policy

    Posted Thu September 12, 2024 03:50 PM

    Thank you both for your responses.

    The OAuth endpoint I am calling does not require an Authorization header; it only needs the client ID and secret in the body as x-www-form-urlencoded. However, the backend endpoint does require an Authorization header.

    I've implemented the API as suggested, but the OAuth token response is not being cached. When I hit my API 20 times in under 15 minutes, I receive an "unauthorized" error due to a rate limit on how many times I can request an access token from the OAuth endpoint.

    I'm not sure where to check the cached values in API Connect SaaS. I am still new to API Connect.

    I've included my assembly code below. Perhaps you could help identify anything I might have missed.

    Assembly Overview:

    • Set Variable: Configures the headers and body for the OAuth request.
    • Invoke (Token Request): Calls the OAuth endpoint to get the access token, with caching set (cache-key: cachedToken, cache-ttl: 86300).
    • Map: Converts the response from the OAuth request from string to object.
    • Set Variable: Extracts the access token from the response.
    • Set Variable: Sets headers for the backend call, including the Authorization token.
    • Invoke (Backend Request): Calls the backend endpoint.

    Could you please advise why the OAuth token isn't being cached? 

    assembly:
        execute:
          - set-variable:
              version: 2.0.0
              title: Token Req Variables Set
              actions:
                - set: message.headers.Content-Type
                  value: application/x-www-form-urlencoded
                  type: any
                - set: message.body
                  value: >-
                    client_id=121212&client_secret=4556545
                  type: any
              
          - invoke:
              version: 2.3.0
              title: Request Token
              backend-type: json
              header-control:
                type: blocklist
                values: []
              parameter-control:
                type: allowlist
                values: []
              http-version: HTTP/1.1
              timeout: 60
              verb: POST
              chunked-uploads: true
              persistent-connection: true
              cache-response: time-to-live
              cache-ttl: 86300
              stop-on-error: []
              websocket-upgrade: false
              cache-putpost-response: true
              keep-payload: false
              target-url: https://oauthedpoint/tokens
              cache-key: cachedToken
              output: accessToken
     
          - map:
              inputs:
                input:
                  schema:
                    type: string
                    $$byref: true
                    $$title: input
                  variable: accessToken.body
                  content: text/plain
              outputs:
                output:
                  schema:
                    type: object
                    $$byref: true
                    $$title: output
                  variable: message.body
                  content: application/json
              actions:
                - set: output
                  from: input
              version: 2.1.0
              title: map
              options:
                mapEmulateV4EmptyJSONObject: false
                mapEmulateV4DefaultRequiredProps: false
                mapEnablePostProcessingJSON: false
                mapReferenceLimit: 1
          - set-variable:
              version: 2.0.0
              title: Message Body Set
              actions:
                - set: token
                  value: $(message.body)
                  type: any
              description: Set the message body of the token request to a variable 
          - set-variable:
              version: 2.0.0
              title: Set request parameters
              actions:
                - set: message.headers.Content-Type
                  value: application/json
                  type: string
                - set: message.headers.client_id
                  value: 12122
                  type: string
                - set: message.headers.client_secret
                  value: 121212121
                  type: string
                - set: message.headers.Authorization
                  value: Bearer $(token.data.access_token)
                  type: string
              
          - invoke:
              version: 2.3.0
              title: Request inventory
              backend-type: json
              header-control:
                type: blocklist
                values: []
              parameter-control:
                type: allowlist
                values: []
              http-version: HTTP/1.1
              timeout: 60
              verb: GET
              chunked-uploads: true
              persistent-connection: true
              cache-response: protocol
              cache-ttl: 900
              stop-on-error: []
              websocket-upgrade: false
              target-url: >-
              output: ''
              cache-key: ''
              inject-proxy-headers: false
              
        catch:
          - errors: []
            execute: []
      activity-log:
        enabled: true
        success-content: activity
        error-content: payload
      catalogs:
        UAT:
          properties: {}


    ------------------------------
    Elshaday Teshome
    ------------------------------



  • 5.  RE: How does the cache work in invoke policy

    Posted Fri September 13, 2024 05:07 PM

    Hi Elshaday,
    So the policy in question is Invoke (Token Request): Is there any chance there is a Cache Control header in your request? or perhaps the OAuth server is providing a Cache-Control response header indicating no-cache or no-store?  Try using the header control of the invoke policy.  By default, anything that is message.headers will be sent to the backend but could also play a part in the caching decision.  Either use an allowlist with only Host and Content-Type (those will be the only two headers sent to the backend) or a blocklist that blocks Cache-Control and Authorization (which will send all but those headers). 

    Best Regards,

    Steve



    ------------------------------
    Steve Linn
    Senior Consulting I/T Specialist
    IBM
    ------------------------------



  • 6.  RE: How does the cache work in invoke policy

    Posted Wed January 22, 2025 11:45 AM

    Hi @Elshaday Teshome

    Where did you set  cache-key: cachedTokencache-ttl: 86300?

    Could you please guide me to config it? I am very appreciate that!



    ------------------------------
    Nguyen Thao
    ------------------------------



  • 7.  RE: How does the cache work in invoke policy

    Posted Sun January 26, 2025 02:35 AM

    Hi Nguyen Thao.

    Can I show you my approach?

    Here is source of invoke:

    - invoke:
      version: 2.3.0
      title: invoke for cached token
      backend-type: json
      header-control:
        type: blocklist
        values: []
      parameter-control:
        type: allowlist
        values: []
      http-version: HTTP/1.1
      timeout: 60
      verb: POST
      chunked-uploads: true
      persistent-connection: true
      cache-response: time-to-live
      cache-putpost-response: true
      cache-ttl: 86300
      stop-on-error: []
      websocket-upgrade: false
      target-url: $(targetUrl)
      cache-key: cachedKey
      decode-request-params: false
      output: cachedResponse

    The chachedResponse variable contains response of invoke, and you can reach and use it by context.get('cachedResponse') ;

    Best Regards,

    Sergey



    ------------------------------
    Sergey Paponov
    ------------------------------



  • 8.  RE: How does the cache work in invoke policy

    Posted Thu February 06, 2025 10:32 PM

    Thanks a lot for your kind support @Sergey Paponov

    I successfully cached my response. However, I encountered a peering issue where the cached response only appears on one DataPower instance. My system consists of three DataPower appliances, and I am still looking for a way to configure data peering between them.

    If you have any suggestions on how to achieve this, I would highly appreciate your help.

    Thank you.



    ------------------------------
    Nguyen Thao
    ------------------------------