10 Kubernetes Health Checks Every DevOps Must Use
Kubernetes health checks explained: startup, liveness and readiness probes, HTTP, TCP, exec and gRPC checks, readiness gates and PDBs, with YAML.
Kubernetes health checks tell the cluster whether a container has started, whether it is still working, and whether it should receive traffic. The kubelet runs startup, liveness and readiness probes using exec, HTTP, TCP or gRPC checks. Readiness gates, Pod Disruption Budgets and the API server's own health endpoints complete the picture.
This guide covers ten health checks you can configure today, with YAML for each, default values from the Kubernetes documentation, and the mistakes that turn a probe into the cause of an outage. It sits inside our Kubernetes architecture guide, which explains where the kubelet and API server fit.
Key takeaways
- Startup, liveness and readiness probes answer three different questions. Use all three only when your application needs all three.
- A failed liveness probe restarts the container. A failed readiness probe only removes the Pod from Service endpoints. Mixing them up is the most common probe bug.
- Default probe timing is
periodSeconds: 10,timeoutSeconds: 1,failureThreshold: 3andsuccessThreshold: 1. - Native gRPC probes are stable since Kubernetes v1.27, so gRPC services no longer need a helper binary.
- Liveness checks should never depend on a database or another service. If the database goes down, every Pod restarts at once.
- Pod Disruption Budgets and readiness gates protect availability during drains and rollouts, which probes alone do not cover.
The ten health checks at a glance
| # | Health check | Question it answers | What happens on failure |
|---|---|---|---|
| 1 | Startup probe | Has the app finished starting? | Container is killed after the failure threshold |
| 2 | Liveness probe | Is the process stuck? | Container restarts |
| 3 | Readiness probe | Can it serve traffic right now? | Pod removed from Service endpoints |
| 4 | HTTP GET check | Does an endpoint return 200 to 399? | Probe counts a failure |
| 5 | TCP socket check | Does the port accept connections? | Probe counts a failure |
| 6 | Exec check | Does a command exit with 0? | Probe counts a failure |
| 7 | gRPC check | Does the gRPC health service report SERVING? | Probe counts a failure |
| 8 | Readiness gates | Has an external system marked the Pod ready? | Pod stays not ready |
| 9 | Pod Disruption Budget | Can this Pod be evicted safely? | Voluntary eviction is refused |
| 10 | Control plane health endpoints | Is the API server live and ready? | Load balancer or monitoring flags the node |
The three probe types
1. Startup probe
A startup probe protects slow-starting containers. While it runs, the kubelet holds back the liveness and readiness probes. Once it succeeds, it stops running and the other probes take over. If it never succeeds within failureThreshold times periodSeconds, the kubelet kills the container and the restart policy applies.
Use it for Java applications that warm caches, services that run migrations at boot, or anything whose start time varies a lot. Without it, people set a long initialDelaySeconds on the liveness probe, which also delays detection of real hangs later.
startupProbe:
httpGet:
path: /healthz
port: 8080
periodSeconds: 10
failureThreshold: 30 # allows up to 300 seconds to start
2. Liveness probe
A liveness probe detects a process that is running but no longer doing useful work, such as a deadlock or an exhausted thread pool. When it fails failureThreshold times in a row, the kubelet restarts the container.
Keep the check local to the process. A liveness endpoint that queries the database makes every replica restart when the database has a brief problem, and the restarts add load at the worst moment. Return success if the process can answer at all.
livenessProbe:
httpGet:
path: /livez
port: 8080
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
If you are unsure whether a container needs a liveness probe, leave it out. A process that crashes on its own already gets restarted by the restart policy. The probe only helps when the process hangs without exiting.
3. Readiness probe
A readiness probe decides whether the Pod receives traffic. It runs for the whole life of the container. On failure, the EndpointSlice controller removes the Pod's IP from every Service that selects it, and the Pod keeps running. When the check passes again, the Pod goes back into rotation.
This is the right place for dependency checks, with care. If the app cannot serve any request without its cache, include the cache. If it can serve a degraded response, do not, because a shared dependency failure would mark every Pod unready and the Service would have no endpoints at all.
readinessProbe:
httpGet:
path: /readyz
port: 8080
periodSeconds: 5
failureThreshold: 2
Readiness also matters during rollouts. A Deployment counts a new Pod as available only after it is ready (plus minReadySeconds if set), so a meaningful readiness probe is what stops a rolling update from pushing broken Pods into service. Our guide to Kubernetes deployment strategies shows how this interacts with maxSurge and maxUnavailable.
The four check mechanisms
Each probe uses exactly one mechanism. The choice depends on what your container exposes.
4. HTTP GET check
The kubelet sends an HTTP GET to the Pod IP on the given port and path. Any status code from 200 to 399 counts as success. You can set scheme: HTTPS and custom headers. The kubelet does not verify the certificate for HTTPS probes.
readinessProbe:
httpGet:
path: /readyz
port: http
httpHeaders:
- name: X-Probe
value: kubelet
Using a named port (port: http) means the probe follows the container port definition if the number changes.
5. TCP socket check
The kubelet tries to open a TCP connection to the port. If the connection opens, the check passes. This suits databases, message brokers and other services without an HTTP endpoint. It only proves the port is listening, not that the service can answer requests, so prefer a protocol-level check when one exists.
livenessProbe:
tcpSocket:
port: 5432
periodSeconds: 15
6. Exec check
The kubelet runs a command inside the container. Exit code 0 is success. This is flexible, for example running pg_isready or checking that a file exists, but each run starts a new process. On nodes with many Pods and short periods, exec probes add CPU overhead. Keep the command small and set a realistic timeoutSeconds.
readinessProbe:
exec:
command: ["pg_isready", "-U", "postgres", "-h", "127.0.0.1"]
periodSeconds: 10
timeoutSeconds: 3
7. gRPC check
For services that implement the gRPC health checking protocol, the kubelet can call the health service directly. A response of SERVING is success. Native gRPC probes became stable in Kubernetes v1.27; before that, teams shipped the grpc_health_probe binary and called it from an exec probe.
livenessProbe:
grpc:
port: 9090
service: liveness # optional service name
periodSeconds: 10
The gRPC probe does not support TLS or authentication, so the health service must answer on a plaintext port that the kubelet can reach.
Checks beyond container probes
8. Pod readiness gates
Readiness gates let an external controller add its own condition to a Pod's readiness. You list a condition type under spec.readinessGates. The Pod is ready only when all containers are ready and every listed condition is True.
The AWS Load Balancer Controller uses this to keep a Pod out of rotation until it is registered and healthy in the ALB target group, which avoids dropped requests during rolling updates.
spec:
readinessGates:
- conditionType: "example.com/lb-registered"
Something has to set that condition through the Pod status API. If nothing does, the Pod never becomes ready, which is an easy mistake to make when copying manifests between clusters.
9. Pod Disruption Budget
A PodDisruptionBudget is not a probe, but it is a health guarantee. It limits how many Pods of an application can be down at once during voluntary disruptions such as kubectl drain or node upgrades. The eviction API refuses evictions that would break the budget.
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: web-pdb
spec:
minAvailable: 2
selector:
matchLabels:
app: web
Budgets rely on readiness. A Pod that is not ready already counts as disrupted, so poor readiness probes make drains stall. Our node maintenance tips cover how drains and budgets work together.
10. Control plane health endpoints
The API server exposes /livez and /readyz (the older /healthz is deprecated). Load balancers in front of the control plane and monitoring tools should use these. Add ?verbose to see each individual check.
kubectl get --raw='/readyz?verbose'
kubectl get --raw='/livez?verbose'
On nodes, check conditions such as Ready, MemoryPressure and DiskPressure with kubectl describe node. Node health and Pod health are separate signals, and an alert on one should not be read as a statement about the other.
Probe settings and their defaults
| Field | Default | Notes |
|---|---|---|
initialDelaySeconds | 0 | Prefer a startup probe to a long delay |
periodSeconds | 10 | Minimum is 1 |
timeoutSeconds | 1 | Raise this for slow endpoints instead of tolerating flapping |
successThreshold | 1 | Must be 1 for liveness and startup probes |
failureThreshold | 3 | Consecutive failures before action |
terminationGracePeriodSeconds (probe level) | Pod value | Lets a liveness failure kill faster than a normal shutdown |
A complete example
This Deployment uses all three probes with separate endpoints, which is the pattern most teams settle on.
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
spec:
replicas: 3
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: registry.example.com/web:1.4.2
ports:
- name: http
containerPort: 8080
startupProbe:
httpGet: { path: /healthz, port: http }
periodSeconds: 5
failureThreshold: 24
livenessProbe:
httpGet: { path: /livez, port: http }
periodSeconds: 10
timeoutSeconds: 2
readinessProbe:
httpGet: { path: /readyz, port: http }
periodSeconds: 5
failureThreshold: 2
Common probe mistakes
- Same endpoint for liveness and readiness. A dependency outage then restarts Pods instead of just draining them. Use separate paths.
- Timeouts left at 1 second. Garbage collection pauses or a busy node can push a healthy response past one second, causing restarts under load.
- Liveness probe checks the database. One external failure becomes a cluster-wide restart loop.
- Expensive health endpoints. A probe every few seconds across hundreds of Pods adds real load. Health handlers should be cheap.
- No probes at all. Pods receive traffic the moment the container starts, before the app is listening.
Writing good health endpoints
Probes are only as useful as the endpoints behind them. A few rules make the difference.
| Endpoint | Should check | Should not check |
|---|---|---|
/livez (liveness) | The process can handle a request; event loop or worker threads respond | Databases, caches, other services |
/readyz (readiness) | Startup work is done; local resources ready; critical dependencies reachable if the app cannot work without them | Optional dependencies; slow full self-tests |
/healthz (startup) | Initial loading finished | Anything that can hang for longer than the startup budget |
- Return quickly, well inside
timeoutSeconds. Cache the result of expensive checks for a few seconds. - Do not require authentication on probe paths, but do not expose sensitive details in the response either.
- Log probe failures with a reason, so
kubectl logsexplains why a Pod went unready. - During shutdown, start failing readiness before the process exits so the Pod leaves Service endpoints first. A short
preStopsleep gives endpoint updates time to spread.
lifecycle:
preStop:
exec:
command: ["sh", "-c", "sleep 10"]
Probes for common workloads
| Workload | Startup | Liveness | Readiness |
|---|---|---|---|
| HTTP web API | HTTP on /healthz if start is slow | HTTP on /livez | HTTP on /readyz |
| gRPC service | gRPC if start is slow | gRPC health service | gRPC health service with a separate service name |
| PostgreSQL or MySQL | Exec with the client tool, generous threshold | TCP or exec | Exec query such as pg_isready |
| Queue worker with no port | Exec checking a ready file | Exec checking a heartbeat file age | Often not needed (no Service traffic) |
| Batch Job | Usually none | Rarely; use activeDeadlineSeconds instead | None |
Debugging a failing probe
kubectl describe pod web-6c9f7d8b5-abcde # look for "Liveness probe failed" events
kubectl get events --field-selector reason=Unhealthy
kubectl logs web-6c9f7d8b5-abcde --previous # logs from before the last restart
kubectl get pod web-6c9f7d8b5-abcde -o jsonpath='{.status.containerStatuses[0].restartCount}'
A rising restart count with Liveness probe failed events usually means the timeout or threshold is too tight, or the liveness check is testing something it should not. Our list of common Pod failures and fixes and the Pod troubleshooting techniques go further on CrashLoopBackOff and readiness problems.
Learn health checks hands-on
Probes, Pod Disruption Budgets and rollout behaviour are core topics in the CKAD training, where you configure them in live labs. The course runs in our Kharadi, Pune classroom and live online across India.
Frequently asked questions
What is the difference between liveness and readiness probes?
A liveness probe decides whether to restart a container. A readiness probe decides whether the Pod receives traffic from Services. A failed readiness probe leaves the container running; a failed liveness probe restarts it.
When should I use a startup probe?
Use one when your container takes a long or variable time to start. It holds back liveness and readiness probes until the app is up, so you do not need a long initial delay on the liveness probe.
What are the default probe values in Kubernetes?
The defaults are initialDelaySeconds 0, periodSeconds 10, timeoutSeconds 1, successThreshold 1 and failureThreshold 3.
Does Kubernetes support gRPC health checks?
Yes. The grpc probe type has been stable since Kubernetes v1.27. It calls the standard gRPC health checking service and treats SERVING as success. It does not support TLS.
Should a liveness probe check the database?
No. If the database fails, every replica restarts at once. Keep liveness checks local to the process and put dependency checks, if any, in the readiness probe.
What happens if a readiness probe fails?
The Pod's IP is removed from the endpoints of every Service that selects it. The container keeps running and returns to rotation when the probe passes again.
What is a Pod readiness gate?
A readiness gate adds an extra condition to Pod readiness that an external controller sets. The Pod is ready only when its containers are ready and the condition is True.
Is the API server /healthz endpoint deprecated?
Yes. Use /livez and /readyz instead. Add ?verbose to see the result of each individual check.
References
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0