How event-driven architecture is changing APIs

Event-driven architectures (EDAs) are peaking in popularity as developers look for ways to make their applications run faster because businesses demand real-time data to make real-time decisions. While EDA isn’t a new idea, it is something that is recently becoming more common in API conversations and architectures.

I caught up with Przemek Kulik, VP of product management at Software AG, to discuss why EDA is so important for API management. This is an excerpt from that interview.

Mirza Salihagic (MS): In brief, what is an event-driven architecture?

Przemek Kulik (PK): An event-driven architecture is a model for application design that focuses on production, detection, consumption and reaction to events vs. responding to requests. It isn’t new, but it is gaining momentum because of what developers are being asked to do – and where other technologies are falling short. It adds speed, responsiveness and resilience to anything that developers are building – be it applications, business processes or APIs.

MS: And how would you describe EDA to someone who hasn’t ever heard of it?

PK: I spoke to an analyst recently who described it in the following way. It’s like when you are on a road trip with your kids in the back seat. They keep asking you, “Hey Dad, are we there yet?” “No, not there yet.” Five seconds later, they ask, “Are we there yet?” “No, we’re not there yet.” And so on.

This is how a REST API works. They ask, you answer. You get frustrated and annoyed.

Instead, your kids can ask you, “Hey Dad, can you tell us when we’re there?” You agree, and for a few hours you have blissful silence in your car until you arrive at your destination and tell your kids, “We’re here!”

That’s the difference between traditional API architectures and event-driven ones – the former which handles information synchronously and the latter which handles it asynchronously.

MS: OK, so why would an EDA be better for APIs?

PK: A lot of it comes down to speed. If you think about a REST API, it responds in just one millisecond to whatever query you have. You ask for data and in one millisecond you get the data back in. You would think that’s enough. But the fact is that first you need to call it to get the data back, so you need to add the time between two of your calls to get the total time to data. And you need to keep calling an API to always be up to date. That’s a lot of calls, constantly happening.

If you compare that to event-driven patterns, the data will be delivered to you when it becomes available. So instead of constantly making calls and waiting on the response, you’re always up to date.

MS: Are there any specific use cases for event-driven APIs?

PK: There are a lot of use cases, and every industry could probably benefit from them. But the two industries that seem to be driving this are the finance and gaming industries, where milliseconds really matter. If you think about the stock exchange, a millisecond could translate to millions in gains or losses.

MS: How do EDAs relate to microservices?

PK: Microservices is one way that teams are increasing the speed of applications and how things are delivered. But when we are talking about microservices, we are mostly focusing on the application layer. But often times the data delivery behind this application is still using the “old” patterns. I query, and I get a response.

So you’ll see that data integration patterns didn’t keep pace. There were different attempts to make it faster and better going from REST, through OData to GraphQL and what not. But what developers have been coming back to recently is to use event-driven patterns, pub/sub, messaging, event streaming, Kafka and so on, because they provide scale and better speed.

MS: Will all new API projects take an event-driven approach?

PK: It will not be the only architecture people will use. If you take a look at microservices when they first appeared, it’s not like one day all other models disappeared and everything that we were building where microservices. We’re seeing the same idea with event-driven APIs. There will be a mixture of architectures. In fact, a common pitfall among developers is to jump on whatever’s new and then decide if it is makes sense as they try to make it work.

When developers are considering EDAs, they need to remember that they’re not right for everything. They’re good for some things. But the decision needs to be made consciously.

Where it will become problematic is in the touch points between these different approaches. There’s nothing wrong with API management, there’s nothing wrong with integration, and there’s nothing wrong with EDA. But making sure they all work together is complicated.

MS: What is the potential impact of EDA on API management?

PK: It’s important to note that “API” has become a business term to mean a way of interacting with a system or application. So things that aren’t technically APIs are sometimes referred to as APIs.

Traditionally APIs (SOAP, REST and GraphQL) have been synchronous – the request-reply approach. It’s been well established, and developers know how to work with it. They know how to build things that way. But sometimes it’s not fast enough. Even if the APIs themselves are very fast, the calls take time, which sometimes just isn’t good enough. So then developers start think through their options.

They then get to this notion of an asynchronous API. What that means in the business sense is a way of interacting with the system or application asynchronously. So you can have a system that acts as a client of another system that says, “Hi, I’m interested in this data. Can you let me know when it arrives?”

That means you don’t need to query every millisecond. You just sit there and wait for the data to arrive. When a purchase order is sent for a given product, notify me at this address. When a stock price changes, notify me. These are asynchronous patterns.

MS: Does the way that APIs are being built need to change?

PK: The reality is that a vast majority of APIs that are built today are synchronous, and developers can’t rewrite and implement async versions over night. So they need a way of actually bridging this divide.

It’s similar to what happened between SOAP and REST. SOAP became a legacy way of doing web services, and REST became this new, fancy and fast way of doing services. But sometimes developers didn’t know how to rewrite something from SOAP to REST, or they didn’t own the code to rewrite it. They needed a piece of technology to help them do this.

In this particular case, this technology was found within API management. When it comes to making synchronous APIs act in asynchronous ways, it’s a similar situation. Sometimes developers don’t have the time or access that they need, so how do they make something synchronous act and behave in asynchronous way?

The solution is also a piece of technology – either API management or integration, depending on what you want to do and how you want to implement it. (And you know Software AG’s platforms can definitely deliver that.)

MS: So what is the role of API management when implementing event-driven architectures?

PK: Developers need to make all of their APIs work together. If you have some that are asynchronous and some that are synchronous, you have a problem. How do they communicate and collaborate? Do I just scrap the synchronous APIs that I have used for years?

API management work with the natively asynchronous APIs to talk to each other over a WebSocket or through callbacks or webhooks. API management could and should help building such native patterns here.

But perhaps more importantly, API management technology can help making my synchronous APIs behave in an asynchronous way. Imagine I have a REST API, which is request-reply, and it will never change because that’s how REST work. But my SAP® system sends out a million messages with some information. How can I make this SAP event or message trigger the API?

Well, I can sit down and code, or I can use an integration or API management platform to do this. I can point an API gateway at this system saying, “Hey SAP, can you call the API in the gateway?”

That’s how API management can bridge these two worlds. I can mediate between these two different technologies, translating one into another - in both directions if needed. I can have my API gateway react to SAP sent event and trigger a REST API in turn, or I can give a REST endpoint to my clients for something that is asynchronous in the backend.

MS: What are the most commonly used async API mechanisms?

PK: WebSocket is being adopted because it feels natural for developers because it is essentially HTTP-based, just like REST. So the learning curve is relatively flat. It capable of working asynchronously instead of request-reply you just open up a pipe to the other side and through this pipe you can keep talking in an asynchronous way. So WebSocket is definitely one.

You also have webhooks for callbacks. This is a mechanism that is easiest described like when someone is doing his subscription was saying hey, I’m interested in this data. I don’t want to call you every second to get this data. This is my address. Can you send this data to me when it’s available?

There’s definitely more to come – messaging and event streaming included.