In the "42" series, I collaborate with the IBM Liberty Cloud Native Architect and Chief Advocate, Emily Jiang, to examine some of the cloud native technology pop culture to separate facts from fiction, and provide a better understanding on when to use them.

In Part 1, we answered the question on "What is Serverless?". In this part, we will answer the question on "When should we use Serverless?"

Serverless is a technology that allow developers to deploy services that can be used on demand without having to pre-deploy the service ahead of time. After an application was turned to a serverless function, it registers a service with an associated end point. The end point was known to the Serverless system to instantiate the service whenever the endpoint is invoked by a caller. After a period of inactivity, the serverless system will stop the service (thus “scale to zero”) and free up the resources allocated for other services.

As a consumer of the serverless service (e.g., AWS Lambda or IBM Cloud Functions), this is great news, because you will only be charged for the running of the service. However, as a provider of the serverless service, the provider has to ensure there is spare capability laying around so the system can handle any concurrent traffic as demanded by the customers. 

Let’s say you’re intrigued by the promise of serverless but due to whatever reasons, cannot use an existing cloud-based serverless services.  You will then have to build out your own serverless infrastructure.  Within Kubernetes (CNCF including Red Hat OpenShift platform), there is a native serverless framework (Knative). Let’s look at what they are.

Knative

Knative is an open-source enterprise-level solution to build Serverless and Event Driven Application. It has 2 major components – Knative serving and Knative eventing. Knative serving is the serverless backbone.  Once a Kubernetes service (pod) has been registered as a Knative service, it will get an endpoint URL that Knative will use to trigger the pod to be started. The primary Knative Servicing resources are Services, Routes, Configurations, and Revisions. These resources are used to define and control how your serverless workload behaves on the cluster.

Knative eventing is an event backbone based on the CloudEvent specification.  It allows Knative services to produce and consume events.  Knative events can also be used as a way to trigger the Knative services to be started. Kantive Eventing uses standard HTTP Post requests to send and receive events. This sounds great in theory but let’s consider the followings before deciding if Knative is still the right solution.

Considerations

The cost of Serverless is not zero (“free”).

As mentioned before, as a provider of serverless system (Knative), the provider must make sure there are spare capacity to handle the demand of the customers.  In addition, Knative rely on Istio/Ingress, Kafka, and other subsystems to ensure it can listen to the events. Recommendation is 6 cores for the 1st worker node, and 2 cores for each additional worker node.

·     There is a “cold start” penalty associated with serverless services.

Knative serving rely on Istio/Ingress to listen to the request and start the
service.  There is a “cold start” time of 10 – 15s depending on the current state of the system – this means even if your pod can start instantly, there is still a delay before your pod can start.  However, there are ways to reduce the “cold start” time, see https://developer.ibm.com/articles/reducing-cold-start-times-in-knative/.

·     Knative has a limit on the number of serverless services. 
There is a perception that with Knative, you can add as many services as you need and it will not affect the capacity of the cluster.  For example, you need to run 10,000 services, but if you know you will only need to support 1,000 concurrent services at a time, you can build a serverless system with just enough capacity to run 1,000 services.  On the other hand, because of Knative leverages Iptables as the backbone of the networking stack, there is a global limit of 10,000 ip addresses per Kubernetes cluster.  Reality is there is an upper limit of about 3,350 Knative services per k8s cluster due to the implementation of Knative and its dependency on iptables (see link). This means once the limit is reached, the only recourse is to add a new cluster.

Given the above considerations, it does not mean we should not use Knative, we just need to realize Knative is not magic, and it is still bound by some real-world limitations.

Serverless presents a different paradigm to deploy and manage your solution.  However, as with any technology decisions, there are always pros and cons.

Comparison between Serverful and Serverless

As you see, if you decide to go with Serverless, you will need to understand the hidden cost and potential benefits. To take advantage of Serverless cost model, your application must start up fast. We will explore how to reduce start-up time in the next article. Stay tuned!

Why “42”?

For those science fiction fans who have read the The Hitchhiker’s Guide to the Galaxy by Douglus Adams, the number “42” is the “Answer to the Ultimate Question of Life, the Universe, and Everything” as calculated by the supercomputer Deep Thought after thinking about it for 7.5 million years.  We are hoping this series can help answer some of your lifelong questions about cloud native technologies and discuss the proper places to use them.