Kubelet at its pod limit
The node runs as many pods as its maxPods.
ImpactNo new pod fits on the node
Can it be seen coming?Yes
RupturaNot covered yet
How to detect it
Running pods near capacity.
Metrics and signals to watch:
kubelet_running_podskube_node_status_capacity pods
Can you see it coming?
Yes Precursor: Pod count growing toward maxPods.
What it hits next
scheduling of new pods.
What to do
Add nodes or raise maxPods.
Sources
- kubernetes-mixin KubeletTooManyPods
Where Ruptura stands
Ruptura does not cover this failure yet. The card is here so you know it exists and how to detect it.
Related failures
- Node not ready
- Node flaps between Ready and NotReady
- Node under memory pressure
- Node under disk pressure
- Node under PID pressure
See it coming instead of reading about it
Ruptura is an open-source operator for Kubernetes and OpenShift that predicts the failures in this catalog that can be predicted, detects the others, and sends them to Alertmanager. Watch an OOM crash predicted 21 minutes early · ★ GitHub
Ruptura is an open-source operator for Kubernetes and OpenShift that predicts the failures in this catalog that can be predicted, detects the others, and sends them to Alertmanager. Watch an OOM crash predicted 21 minutes early · ★ GitHub