Container throttled by its CPU limit
The container hits its CPU quota and waits for most periods.
ImpactLatency rises, probes time out, requests queue
Can it be seen coming?Partly
RupturaCovered
How to detect it
Share of throttled periods stays high.
Metrics and signals to watch:
container_cpu_cfs_throttled_periods_totalcontainer_cpu_cfs_periods_total
Can you see it coming?
Partly Precursor: CPU usage rising toward the limit.
What it hits next
callers of the workload.
What to do
Raise the CPU limit or reduce the workload's CPU demand.
Sources
- kubernetes-mixin CPUThrottlingHigh
Where Ruptura stands
Ruptura reports this failure as the cpu_throttling failure mode. A pre-registered lab test is not published yet.
Related failures
- Memory leak ends in an OOM kill
- Memory growth accelerates toward the limit
- Sudden memory jump past the limit
- Container killed by OOM again and again
- Container crashes at start, again and again
See it coming instead of reading about it
Ruptura is an open-source operator for Kubernetes and OpenShift that predicts the failures in this catalog that can be predicted, detects the others, and sends them to Alertmanager. Watch an OOM crash predicted 21 minutes early · ★ GitHub
Ruptura is an open-source operator for Kubernetes and OpenShift that predicts the failures in this catalog that can be predicted, detects the others, and sends them to Alertmanager. Watch an OOM crash predicted 21 minutes early · ★ GitHub