15 Common Kubernetes Pod Failures and Fixes
15 common Kubernetes pod failures, from CrashLoopBackOff and ImagePullBackOff to OOMKilled and Pending, with the kubectl checks and fixes for each.
Most Kubernetes pod failures fall into four groups: the pod cannot be scheduled (Pending), the image cannot be pulled (ImagePullBackOff), the container cannot be created (CreateContainerConfigError), or the process keeps exiting (CrashLoopBackOff, OOMKilled). Start every diagnosis with kubectl describe pod and kubectl logs --previous.
This page lists 15 failures you will meet on real clusters, with the status you see, the usual causes, the commands that confirm the cause, and the fix. Commands were checked against the kubernetes.io debugging documentation. For background on the components involved (scheduler, kubelet, container runtime), see 12 Kubernetes architecture components.
Key takeaways
- The
Eventssection ofkubectl describe podnames the cause for most scheduling, image and volume failures. kubectl logs --previousshows output from the crashed container, which is usually what you need for CrashLoopBackOff.- Exit code 137 means the process received SIGKILL, often from the out-of-memory killer. Exit code 1 is an application error, and 127 means the command was not found.
- A pod that is Running but shows
0/1ready is failing its readiness probe and gets no Service traffic. - Many failures come from manifests (wrong image tag, missing Secret, bad probe), so validate YAML in CI before it reaches the cluster.
The five commands you need first
# Status, restarts and node for every pod in a namespace
kubectl get pods -n shop -o wide
# Events, container states, exit codes and probe results
kubectl describe pod checkout-7d9f8 -n shop
# Logs from the current and the previous (crashed) container
kubectl logs checkout-7d9f8 -n shop -c app
kubectl logs checkout-7d9f8 -n shop -c app --previous
# Recent events in the namespace, newest last
kubectl get events -n shop --sort-by=.lastTimestamp
# Attach a debug container to a running pod (ephemeral containers)
kubectl debug -it checkout-7d9f8 -n shop --image=busybox:1.36 --target=app
kubectl debug with ephemeral containers is stable since Kubernetes 1.25. It helps when the app image has no shell.
Quick reference table
| # | Status or symptom | Stage | Most common cause | First check |
|---|---|---|---|---|
| 1 | CrashLoopBackOff | Running | App exits on start | logs --previous |
| 2 | ImagePullBackOff / ErrImagePull | Image pull | Wrong name, tag or credentials | describe Events |
| 3 | OOMKilled (exit 137) | Running | Memory limit too low or leak | describe Last State |
| 4 | Pending: Insufficient cpu/memory | Scheduling | Requests larger than free capacity | describe Events |
| 5 | Pending: taints or affinity | Scheduling | No matching node | describe Events |
| 6 | Pending: unbound PVC | Scheduling | No StorageClass or PV | get pvc |
| 7 | CreateContainerConfigError | Create | Missing ConfigMap or Secret key | describe Events |
| 8 | Stuck in ContainerCreating | Create | Volume mount or CNI problem | describe Events |
| 9 | RunContainerError / exit 127 | Start | Bad command or entrypoint | describe State |
| 10 | Running but 0/1 ready | Running | Readiness probe failing | describe Events |
| 11 | Restarts from liveness probe | Running | Probe too strict or slow start | describe Events |
| 12 | Init:CrashLoopBackOff / Init:Error | Init | Init container fails | logs -c init-name |
| 13 | Evicted | Running | Node under memory or disk pressure | describe pod and node |
| 14 | Stuck in Terminating | Deletion | Finalizer or unreachable node | get pod -o yaml |
| 15 | Rejected by admission (forbidden) | Admission | Pod Security or policy violation | ReplicaSet events |
1. CrashLoopBackOff
The container starts, exits, and the kubelet restarts it with an increasing delay. The kubernetes.io pod lifecycle page says the back-off grows exponentially (10s, 20s, 40s and so on) and is capped at five minutes.
Causes: an unhandled exception at startup, a missing environment variable, a database the app cannot reach, or a command that finishes immediately (for example a script run in a Deployment, which expects a long-running process).
Confirm: run kubectl logs POD --previous and check the exit code under Last State in kubectl describe.
Fix: fix the application error or configuration. If the workload is meant to run once, use a Job instead of a Deployment.
2. ImagePullBackOff and ErrImagePull
The kubelet could not pull the image. ErrImagePull is the first failure and ImagePullBackOff is the waiting state between retries.
Causes: a typo in the image name or tag, a tag that was never pushed, a private registry without credentials, registry rate limits, or a node with no route to the registry.
Confirm: the Events line shows the registry's reply, such as "not found", "unauthorized" or "toomanyrequests".
Fix: correct the reference, or create a pull secret and reference it:
kubectl create secret docker-registry regcred -n shop \
--docker-server=registry.example.com \
--docker-username=ci-bot --docker-password="$REG_TOKEN"
spec:
imagePullSecrets:
- name: regcred
Pin images by tag or digest rather than latest, so you know exactly what was meant to run. Our post on image tagging and versioning covers this.
3. OOMKilled (exit code 137)
The container used more memory than its limit, and the kernel's out-of-memory killer stopped it. kubectl describe shows Reason: OOMKilled and Exit Code: 137 under Last State.
Causes: a limit set below real usage, a memory leak, or a runtime (such as the JVM) sizing its heap without regard to the container limit.
Fix: measure actual usage with kubectl top pod (requires metrics-server), raise the limit to fit, and configure the runtime to respect container limits. If usage climbs forever, it is a leak and a higher limit only delays the crash.
4. Pending: insufficient CPU or memory
The scheduler found no node with enough unreserved capacity for the pod's requests. Events show a message like 0/3 nodes are available: 3 Insufficient memory.
Fix: lower inflated requests, add nodes, or let a cluster autoscaler add them. Remember that scheduling uses requests, not live usage, so a cluster can look idle and still refuse pods. See 12 Kubernetes scaling strategies.
5. Pending: taints, node selectors or affinity
Events say something like node(s) had untolerated taint or didn't match Pod's node affinity/selector.
Fix: compare the pod's nodeSelector, affinity rules and tolerations with node labels and taints:
kubectl get nodes --show-labels
kubectl describe node worker-2 | grep -i taints
Either label the right nodes, add a toleration, or relax the rule.
6. Pending: PersistentVolumeClaim not bound
The pod uses a PVC that has no volume yet. Events show pod has unbound immediate PersistentVolumeClaims.
Confirm: kubectl get pvc -n shop shows Pending. Describe the PVC to see why.
Fix: make sure a StorageClass exists (or set a default one), that its provisioner is running, and that the requested access mode and size are supported. Our Kubernetes storage options guide explains StorageClasses and access modes.
7. CreateContainerConfigError
The kubelet cannot build the container's configuration, usually because a referenced ConfigMap, Secret or key does not exist in the pod's namespace.
Confirm: Events say secret "db-creds" not found or couldn't find key PASSWORD.
Fix: create the object in the same namespace, fix the key name, or mark the reference optional: true if the app can run without it.
8. Stuck in ContainerCreating
The pod is scheduled but the container never starts. Events usually point to a volume mount (MountVolume.SetUp failed, Multi-Attach error) or pod networking (failed to set up sandbox container).
Fix: for a Multi-Attach error on a ReadWriteOnce volume, make sure the old pod using it has terminated, and check the CSI driver pods. For sandbox errors, check that the CNI plugin pods are healthy on that node.
9. RunContainerError, CreateContainerError and exit code 127
The runtime created the container but could not start the process. Exit code 127 means the shell could not find the command. Exit 126 means the file exists but is not executable.
Fix: check command and args in the manifest against the image's real paths. Run the image locally to confirm: docker run --rm -it --entrypoint sh IMAGE. Minimal images may have no shell, so a sh -c wrapper will fail there.
10. Running but not ready (0/1)
The container runs, but its readiness probe fails, so the endpoints controller removes the pod from Service endpoints. Users see errors or the Service has no backends.
Confirm: Events show Readiness probe failed with an HTTP code or timeout.
Fix: check that the probe path and port match what the app serves, and that the app's dependencies are reachable. Probe a lightweight endpoint, not one that calls every downstream service.
11. Restarts caused by the liveness probe
A liveness probe that fails restarts the container, which can look like a crash loop even when the app is fine but slow to start.
Fix: add a startupProbe for slow-starting apps. Liveness checks wait until the startup probe succeeds.
startupProbe:
httpGet:
path: /healthz
port: 8080
failureThreshold: 30
periodSeconds: 10
livenessProbe:
httpGet:
path: /healthz
port: 8080
periodSeconds: 10
timeoutSeconds: 2
This gives the app up to 300 seconds to start before liveness checks begin.
12. Init container failures
Status shows Init:Error, Init:CrashLoopBackOff or stays at Init:0/1. App containers never start until every init container succeeds.
Fix: read the init container's own logs with kubectl logs POD -c INIT_NAME. Common causes are a migration script failing or a "wait for database" loop that never sees the service.
13. Evicted
The kubelet evicted the pod because the node ran low on memory, disk or inodes. The pod shows Status: Failed and Reason: Evicted with a message naming the resource.
Fix: set memory requests close to real usage (pods using more than their requests are evicted first), clean up node disk (container logs and unused images), and check kubectl describe node for MemoryPressure or DiskPressure conditions. Evicted pod objects stay until deleted or garbage collected.
14. Stuck in Terminating
The pod was deleted but never goes away. Causes include a finalizer that a controller never removes, a node that is unreachable so the kubelet cannot confirm shutdown, or an app ignoring SIGTERM until the grace period ends.
Confirm: kubectl get pod POD -o yaml and look at metadata.finalizers and the node's status.
Fix: fix or restore the controller that owns the finalizer, or recover the node. A forced delete (kubectl delete pod POD --grace-period=0 --force) removes the API object without confirming the container stopped. The Kubernetes docs warn this can leave processes running on the node, so use it only when you understand the risk, especially for StatefulSets.
15. Rejected by Pod Security admission or a policy engine
No pod appears at all. The Deployment exists but its ReplicaSet cannot create pods. kubectl describe rs shows forbidden: violates PodSecurity "restricted:latest" or a message from a policy engine such as Kyverno or OPA Gatekeeper.
Fix: update the pod spec to meet the policy (for the restricted profile: run as non-root, drop all capabilities, set a seccomp profile, disallow privilege escalation) instead of loosening the namespace label.
securityContext:
runAsNonRoot: true
seccompProfile:
type: RuntimeDefault
containers:
- name: app
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
A repeatable troubleshooting order
- Read the pod status column. It tells you which stage failed.
- Read Events in
kubectl describe pod. Scheduling, image, volume and probe failures are named there. - Read logs, including
--previous, for anything that started and exited. - Check the owner object (ReplicaSet, StatefulSet, Job) if no pod exists.
- Check the node: conditions, disk, and system pods for CNI and CSI.
- Fix the manifest in Git, not with live edits, so the fix survives the next deploy.
To catch restart loops before users do, add a restart alert like the one in 10 monitoring alerts every DevOps team should configure. To practise explaining these failures in interviews, try our Kubernetes scenario-based interview questions. For log collection across pods, see 12 Kubernetes logging tools.
The CKA training includes hands-on troubleshooting labs, in our Kharadi, Pune classroom or live online.
Frequently asked questions
What is the difference between CrashLoopBackOff and ImagePullBackOff?
ImagePullBackOff means the image was never downloaded, so the container never ran. CrashLoopBackOff means the image pulled and the container started, but the process keeps exiting.
How do I see logs of a crashed container?
Run kubectl logs POD --previous, adding -c CONTAINER for multi-container pods. It prints output from the last terminated instance of that container.
What does exit code 137 mean in Kubernetes?
The process was killed with SIGKILL (128 + 9). When the reason is OOMKilled, the container exceeded its memory limit. It can also follow a failed graceful shutdown.
Why is my pod stuck in Pending?
The scheduler cannot place it. Events will say why: not enough CPU or memory for the requests, taints or affinity with no matching node, or an unbound PersistentVolumeClaim.
How long is the CrashLoopBackOff delay?
The kubelet restarts with an exponential back-off starting at 10 seconds and capped at five minutes, per the Kubernetes pod lifecycle documentation. It resets after the container runs successfully for a while.
Is force deleting a stuck pod safe?
Not always. A forced delete removes the API object without confirming the container stopped, so the process may keep running on the node. Fix finalizers or the node first.
How do I debug a container that has no shell?
Use kubectl debug to attach an ephemeral container with tools such as busybox, targeting the app container so you can see its processes.
References
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0