Kubernetes Probes – Liveness, Readiness and Startup

motoskia

2 years ago

Recently we have seen about pod requests and limits, now let’s see about another important and interest feature on Kubernetes. Which is nothing but Probes (Health Check). If you have worked on docker, you may know about health checks, which will help you to check the container status. As like that, here in Kubernetes we have certain condition which helps to make sure only we have healthy pods in our cluster. In additional this given option to find the startup duration, is pod ready for service, or live to serve traffic.

Types of Probes

Kubernetes gives you three types of health checks probes:

Liveness
Startup
Readiness

Liveness Probe

Liveness probes allow you to automatically determine container application (running in a container inside Pod) is healthy and also indicates whether the container is running. If app is unhealthy, Pod will be marked as unhealthy. If a Pod fails health-checks continuously, the Kubernetes terminates the pod and starts a new one.

Sometimes, Liveness probes could help to find a deadlock, where you could see an application is still running, but unable to make any progress. Restarting a container in such a state can help to make the application to available.

Also configure a health check that periodically checks if your application is still alive. Containers can be restarted if the check fails.

Use Cases

Use Liveness Probe to remove unhealthy pods.
If you’d like your container to be killed and restarted if a probe fails, then specify a liveness probe, and specify a restartPolicy of Always or OnFailure.
Use a liveness probe for a container that can fail with no possible recovery.
If your container cannot crash by itself when there is an unexpected error occur, then use liveness probes. Using liveness probes can overcome some of the bugs the process might have.

Best Practices

The best way to implement the liveness probe is for application to expose a /health HTTP endpoint. When receiving a request on this endpoint, the application should send a 200-399 response if it is considered healthy.

Readiness Probe

Readiness probes are used to determine when a container is ready to accept requests. Kubernetes makes sure the readiness probe passes before allowing a service to send traffic to the pod.
Unlike a liveness probe, a readiness probe doesn’t kill the container. If the readiness probe fails, Kubernetes simply hides the container’s Pod from corresponding Services, so that no traffic is redirected to it.
Liveness probes do not wait for readiness probes to succeed. If you want to wait before executing a liveness probe you should use initialDelaySeconds or startupProbe.
A side-effect of using Readiness Probes is that they can increase the time it takes to update Deployments.

Use Cases

Use Readiness Probe to detect partial unavailability.
If you’d like to start sending traffic to a Pod only when a probe succeeds, specify a readiness probe.
If you want your container to be able to take itself down for maintenance, you can specify a readiness probe that checks an endpoint specific to readiness that is different from the liveness probe.

Best Practices

The recommended way to implement the readiness probe is for your application to expose a /ready HTTP endpoint. When it receives a request on this endpoint, the application should send a 200-399 response if it is ready to receive traffic.

Startup Probe

These probes are very similar to liveness probes, however, while liveness probes run constantly on a schedule, startup probes run at container startup and stop running once they succeed.
They are used to determine when the application has successfully started up.
Startup probe has higher priority over the two other probe types. Until the Startup Probe succeeds, all the other Probes are disabled.
Startup probes are useful in situations where your app can take a long time to start, or could occasionally fail on startup.
Rather than set a long liveness interval, you can configure a separate configuration for probing the container as it starts up, allowing a time longer than the liveness interval would allow.

Use Cases

Use Startup Probe for applications that take longer to start.
When your app needs additional time to startup, you can use the Startup Probe to delay the Liveness and Readiness Probe.
If your container needs to work on loading large data, configuration files, or migrations during startup, you can use a startup probe.

Possible options to check:

Kubelet can check a Pods health in three ways. Each probe must define exactly one of these mechanisms:

HTTP

We define a Port number along with the URL. Kubernetes pings this path, and if it gets an HTTP response in the 200 or 300 range, it marks the app as healthy. Otherwise, it is marked as unhealthy.
HTTP probes are the most common type of custom probe.
Even if your app isn’t an HTTP server, you can create a lightweight HTTP server inside your app to respond to the liveness probe.

TCP

Kubernetes tries to establish a TCP connection on the specified port. If it can establish a connection, the container is considered healthy; if it, can’t it be considered unhealthy.
TCP probes come in handy if you have a scenario where HTTP probes or Command probes don’t work well.
For example, a gRPC or FTP service is a prime candidate for this type of probe.

Command

Kubernetes runs a command inside the container. If the command returns with exit code 0, then the container is marked as healthy. Otherwise, it is marked unhealthy.
This type of probe is useful when you can’t or don’t want to run an HTTP server, but can run a command that can check whether or not your app is healthy.

Facts

While Liveness probe detects failures in an app that are resolved by terminating the Pod (i.e. restarting the app), Readiness Probe detects conditions where the app may be temporarily unavailable.
Liveness probe passes when the app itself is healthy, but the readiness probe additionally checks that each required back-end service is available. This helps you avoid directing traffic to Pods that can only respond with error messages.
By combining liveness and readiness probes, you can instruct Kubernetes to automatically restart pods or remove them from backend groups. If your app has a strict dependency on back-end services, you can implement both liveness and readiness probes.
By default, the probe will stop if the application is not ready after three attempts. In case of a liveness probe, it will restart the container. In the case of a readiness probe, it will mark pods as unhealthy.
In many applications, the /health and /ready endpoints are merged into a single /health endpoint because there is no real difference between their healthy and ready states.

Demo: