API Connect

When protecting our backend and distributing a fair amount of capacity across API consumers in the REST world we limit the amount of API calls , as each call represents a certain resource. In the world of GraphQL however this doesn't work, because a call can have a tiny impact and tiny response or it can be huge.

To solve this problem we have to take a different approach, rather than an amount of calls per time unit you get a "budget" per time unit. 

With the same star wars API used in part 1 of this series we now imagine that we are the owner of the backend for that GraphQL endpoint, it's build on a collection of data sources and it happens to be that Species and Planet information are more expensive for us to collect (it might be because we rely on an API of external party, or because the backend containing that information is just more expensive to host)

What we do is we give it the respective nodes a higher weight
Now let's create a plan where we set the total rate limits in the product.
First we create the product and then we add the SWAPI API , we then go and edit the GrapQL Rate Limits for this Plan


To get an idea of field and type cost, have a look at this video

Now lets see it in action , a developer makes the following query 

{
	allFilms(first: 10) {
		edges{node{title}}
		films {
			title
			episodeID
			vehicleConnection(last: 10) {
				vehicles {
					name
					model
				}
			}
			speciesConnection(last: 1000) {
				species {
					name
					designation
					classification
					personConnection(last: 10) {
						people {
							name
							gender
						}
					}
				}
			}
		}
	}

But he gets a 429 - Too Many Requests response
He can also see in the response headers what his ratelimit is, how much of it is remaining and in what time it will get reset.

x-ratelimit-limit: name=graphql-field-cost,10000;name=graphql-type-cost,10000;
x-ratelimit-remaining: name=graphql-field-cost,0;name=graphql-type-cost,0;
x-ratelimit-reset: name=graphql-field-cost,2;name=graphql-type-cost,2;

Now that we have this failing request, the developer wants to know why he can send a request to the /cost endpoitn, in part 1 of these series we enabled the /cost endpoint so people can see the cost of their query before executing it.

We send in the same query and get the following response 

{
	"query": {
		"query": "{allFilms {edges {node {title}} films {title episodeID vehicleConnection(last:10) {vehicles {name model}} speciesConnection(last:1000) {species {name designation classification personConnection(last:10) {people {name gender}}}}}}}",
		"operationType": "query"
	},
	"request": {
		"fieldCost": 20053,
		"typeCost": 210152,
		"typeCounts": {
			"Film": 20,
			"FilmSpeciesConnection": 10,
			"FilmVehiclesConnection": 10,
			"FilmsConnection": 1,
			"FilmsEdge": 10,
			"Person": 100000,
			"Root": 1,
			"Species": 10000,
			"SpeciesPeopleConnection": 10000,
			"Vehicle": 100
		},
		"fieldCounts": {
			"Film.speciesConnection": 10,
			"Film.vehicleConnection": 10,
			"FilmSpeciesConnection.species": 10,
			"FilmVehiclesConnection.vehicles": 10,
			"FilmsConnection.edges": 1,
			"FilmsConnection.films": 1,
			"FilmsEdge.node": 10,
			"Root.allFilms": 1,
			"Species.personConnection": 10000,
			"SpeciesPeopleConnection.people": 10000
		}
	},
	"rateLimits": {
		"assemblyRateLimits": {
			"graphql-design-request": [
				{
					"rate": 5,
					"interval": 3,
					"unit": "second",
					"hardLimit": true
				}
			],
			"graphql-field-cost": [
				{
					"rate": 10000,
					"interval": 10,
					"unit": "second",
					"hardLimit": true
				}
			],
			"graphql-type-cost": [
				{
					"rate": 10000,
					"interval": 10,
					"unit": "second",
					"hardLimit": true
				}
			]
		},
		"assemblyCountLimits": {
			"graphql-subscription-field-cost": [
				{
					"count": 0,
					"hardLimit": true
				}
			],
			"graphql-subscription-type-cost": [
				{
					"count": 0,
					"hardLimit": true
				}
			],
			"graphql-subscription-websocket": [
				{
					"count": 0,
					"hardLimit": true
				}
			]
		},
		"rateLimits": {
			"default": [
				{
					"rate": 0,
					"interval": 1,
					"unit": "second",
					"hardLimit": false
				}
			]
		},
		"name": "default-plan"
	}
}

From here we can see where the problem sits, what are rate limits are and we decide to reduce the amount of speciesConnections we request from a 1000 to 10
 speciesConnection(last: 1000) {
to
 speciesConnection(last: 10) {
we make another call to the cost endpoint and see that we are now within the limits of our rate limit and we can make a couple of calls without running into the rate limit.

This way the developer  is forced to make more sensible queries, if a developer turns out to have a justified reason for needing a higher rate limit you can always create an extra plan in the product that requires subscription approval and only provide it to that developer.


To read more on the cost specification that we have implemented and open sourced go here
https://ibm.github.io/graphql-specs/cost-spec.html