If you are managing the Kubernetes cluster or trying to set your own, you may notice some pods in evicted status. This happens when the pods run with lack of resources like CPU or memory or due to some application error, then Kubernetes restart these evicted pods, but still when you run the kubectl get pod command, you will see the evicted pods sometimes.
Before we see how to clear all evicted pods, we let see how to check the pods and get the evicted. To list down all the pods, in particular namespace (here, foxutech), you can run the following command:
# kubectl get pod -n foxutechAs you know this will list all the pods on the foxutech namespace.
Why Evicted Pod matters?
Let’s consider, you have too many evicted pods in your cluster, this can lead to network load in each pod, even though it is evicted, it is connected to the network and, This will be problematic if you are using cloud Kubernetes cluster as this will block an IP address, which can lead to exhaustion of IP addresses too if you have a fixed pool of IP addresses for your cluster.
Also, when we have too many pods in Evicted status, it becomes difficult to monitor the pods by running the kubectl get pod command as you will see too many evicted pods, which can be a bit confusing at times.
Delete Evicted Pods
We can use the kubectl delete pod command to delete any pod in Kuberenetes. But with this command, we need to provide the pod name to delete any particular pod.
If you have one or two pods to delete, you can easily do that, by first running the kubectl get pod command:
# kubectl get pod -n foxutech
NAME READY STATUS RESTARTS AGE
web-server-df45976b8-6d8mc 2/2 Running 0 8h
web-server-df45976b8-z26qj 2/2 Running 0 8h
web-server-df45973b8-z16qj 2/2 Evicted 0 8h
nginx-deployment-5d59d67564 1/1 Running 0 8h
nginx-deployment-5r58d67383 1/1 Running 0 8h
nginx-deployment-5h52d63383 1/1 Evicted 0 8h
Then using the pod name to delete the pod, like below:
# kubectl delete pod nginx-deployment-5h52d63383 -n foxutechThe above command will delete the pod with name nginx-deployment-5h52d63383 in foxutech namespace and will release all the resources held by that pod.
What If?
As this is just one or two pods, we can easily delete it, what if there is more than 10+ pods in evicted state? Well, no problem again as we are using Linux it simplifies this kind of operations, with just single command.
# kubectl get pod -n foxutech | grep Evicted | awk '{print $1}' | xargs kubectl delete pod -n foxutechIn the above command, we are searching for the pods with status Evicted and clearing it using kubectl delete pod command for all of them.
You can also use the above command to delete pods in any particular status like Running, Evicted, CrashLoopBackOff, etc.
If in your environment is frequently getting evicted, use this command and automate via some cron or some your preferable way. Also try to apply namespace resource quotas and limit range, this will help to manage pods resource consumption.
Check our Kubernetes Troubleshooting Series on https://foxutech.com/category/kubernetes/k8s-troubleshooting/
You can follow us on social media, to get some short knowledges regularly.
- Facebook: https://www.facebook.com/foxutech/
- Instagram: https://www.instagram.com/foxutech/
- YouTube: Foxutech
- Twitter: https://twitter.com/foxutech
- Medium: FoxuTech – Medium
