DNS resolution fails
Lookups time out or return SERVFAIL.
ImpactCalls between services fail across namespaces
Can it be seen coming?Partly
RupturaPlanned
How to detect it
SERVFAIL or timeout rate high.
Metrics and signals to watch:
coredns_dns_responses_totalkernel DNS events
Can you see it coming?
Partly Precursor: CoreDNS saturated or its upstream failing.
What it hits next
every workload that resolves names.
What to do
Check CoreDNS pods, their limits and upstream resolvers.
Sources
- CoreDNS CoreDNSErrorsHigh
Where Ruptura stands
This failure is on the Ruptura roadmap. It is not reported yet.
Related failures
- Service with no ready endpoint
- Calls to a dependency fail
- Network retransmits rise
- NetworkPolicy blocks a needed call
- Route or Ingress has no healthy backend
See it coming instead of reading about it
Ruptura is an open-source operator for Kubernetes and OpenShift that predicts the failures in this catalog that can be predicted, detects the others, and sends them to Alertmanager. Watch an OOM crash predicted 21 minutes early · ★ GitHub
Ruptura is an open-source operator for Kubernetes and OpenShift that predicts the failures in this catalog that can be predicted, detects the others, and sends them to Alertmanager. Watch an OOM crash predicted 21 minutes early · ★ GitHub