API Connect

API Connect

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.


#API Connect
#Applicationintegration
#APIConnect
 View Only
  • 1.  Rate Limiting Setting for Different Consumer Organizations

    Posted Sun March 17, 2024 09:23 PM
    Edited by Guo Jun Qiao Sun March 17, 2024 09:29 PM

    Requirement:

    We have requirement to set rate limiting for different consumer organizations in APIC. Here's a breakdown of the requirements:

    • Rate Limit Target: Consumer Organization AD Groups (Active Directory Groups). Each Consumer Organization has multiple AD accounts associated with their individual AD Group to access the APIs from APIC. 
    • Rate Limit per Consumer Organization AD Group: 500k monthly requests. The 500K limit should apply to all the accounts owned by each Consumer Organization AD group. 
    • Rate limit Reset Period: on 1st day of each month (12:00 AM) we want to reset rate limit for each Consumer Organization.
    • Additional Rate Limit: 4 requests per 2 minutes for accessing one API. This is to limit all the Consumer Organizations from accessing this API. 

    Here are some queries for which we need your assistance.

    • Is implementing rate limiting based on AD group feasible instead of setting it to an individual customer account? Given the customer's specific needs, what would be the most efficient and scalable way to implement rate limiting based on AD group membership?
    • The rate limit has to be reset to all the Consumer Organizations on the first day of each month (12:00 AM). The current APIC feature supports resetting for days and weeks. Can you suggest an option that meets the client's requirements through the other options(external script, etc.)? 
    • The customer is required to onboard new consumers in the future. Considering the high number of Consumer Organizations (20+) and the overall rate limit volume, what approach would ensure optimal performance and scalability? Applying the rate limit through global policies, etc.
    • Does API Connect offer built-in functionality to send email notifications when a rate limit is reached? Can you suggest alternative solutions if email notifications for rate limit thresholds aren't available in API Connect?



    ------------------------------
    Guo Jun Qiao
    ------------------------------



  • 2.  RE: Rate Limiting Setting for Different Consumer Organizations

    Posted Mon March 18, 2024 02:33 AM

    Rate limits are based on application subscriptions. They're not per user or per consumer org.

    You could look at creating an application per ad group or something like that, but the apic rate limits would remain per application, you'd need to create the apps per group.

    20 consumer orgs is not high, that's tiny, I have more than that on my test system ;-) 20,000 would be a large number of consumer orgs.

    Email notifications on rate limits is not currently possible but it's being looked at - I'd suggest raising an rfe.

    Id strongly recommend you use apic artifacts such as consumer orgs and apps how they're intended or you'll likely have issues down the line.

    eg a consumer org only exists to share application credentials with other people. If you wouldn't tell someone your app keys then you shouldn't be in the same consumer org as that is literally the primary purpose of a consumer org.



    ------------------------------
    Chris Dudley
    ------------------------------



  • 3.  RE: Rate Limiting Setting for Different Consumer Organizations

    Posted Mon March 18, 2024 02:36 AM

    Consumer org membership cannot be externalised to AD groups either.

    that must be managed internally inside apic, but you could write scripts to make the internal membership match what you want.



    ------------------------------
    Chris Dudley
    ------------------------------



  • 4.  RE: Rate Limiting Setting for Different Consumer Organizations

    Posted Mon March 18, 2024 04:21 AM

    Hi Chris,

    What do you mean to   make the internal membership match what we want? Do you mean match internal membership with Consumer organization AD group? 

    Is it possible to share some sample scripts to make the internal membership match what we want? 



    ------------------------------
    Guo Jun Qiao
    ------------------------------



  • 5.  RE: Rate Limiting Setting for Different Consumer Organizations

    Posted Mon March 18, 2024 04:29 AM

    APIC is only going to use its internal list of who is in what consumer org. It is not possible to delegate that to external AD groups. Essentially your external consumer org AD groups are meaningless to APIC and cannot be used for anything directly.

    That means you need to make that internal list match your external server - maybe through some kind of cron that runs nightly to make sure that the membership is correct - create new users, remove old ones, etc... That script can use the APIC Provider API/CLI in order to modify the membership of consumer orgs. I'm afraid I don't have any examples, its not something I've ever tried to do.

    Remember if you're putting more than 10 users in a consumer org you are probably doing it wrong - consumer orgs only exist to share credentials, so it is a bit of a give away if you have too many users in a single one.

    You can always group consumer organizations together into "Groups" which can then be used for setting product visibility.

    A reminder - rate limits apply at the application level. A subscription is a link between a given application and a specific product plan.

    If you want to raise an RFE for externalising the consumer org membership to AD groups then please do.



    ------------------------------
    Chris Dudley
    ------------------------------



  • 6.  RE: Rate Limiting Setting for Different Consumer Organizations

    Posted Mon March 18, 2024 08:37 PM

    Hi Chris,

    Thank you. Let me discuss with team to see if any further questions on this rate limit. 



    ------------------------------
    Guo Jun Qiao
    ------------------------------



  • 7.  RE: Rate Limiting Setting for Different Consumer Organizations

    Posted Mon March 18, 2024 09:55 PM

    Hi Chris,

    When storing analytic data, is it possible to enable request body and disable response body? 



    ------------------------------
    Guo Jun Qiao
    ------------------------------



  • 8.  RE: Rate Limiting Setting for Different Consumer Organizations

    Posted Tue March 19, 2024 01:03 AM

    The log level settings are just activity, headers, payload, so out of the box, no you can't.

    However you can use custom code in the API to customise that, of even use custom logstash code to redact out the response bodies if you dont need them.

    Bodies are just stored as long strings - they arent parsed, and they considerably increase the overheads in storage and memory on analytics, so unless essential we generally advise against payload storage.



    ------------------------------
    Chris Dudley
    ------------------------------