Every flight instructor knows the debrief where a student says, “It just felt wrong on short final.” The useful reply is never “try harder”. It is a question: what did the airspeed do, what did your hands do, and what changed from the last approach? Diagnosis is a skill built from symptoms, ordered questions and a calm voice. The same skill sits at the centre of KubernetesOps, the informal name for looking after Kubernetes clusters, where, according to the Ops Atlas, fault-finding is the biggest block of the CKA blueprint at thirty percent. This post works through five common symptoms the way an instructor would, with a problem, a likely cause and a fix.
The background facts come from the Atlas’s page on KubernetesOps, and every scenario below is an invented composite for teaching, not a report on any real system.
Symptom one: new pods sit in Pending while nodes look quiet
The story: a team adds a service and it never starts. The dashboard shows machines at a fraction of their processing power, yet the scheduler reports nothing fits. The student, in effect, is being told the runway is full when the apron looks empty.
Likely cause: reservations. The scheduler places work according to what each workload asked for, not what it consumes. Machines can be fully booked on paper while idling in practice. Fix: compare requested capacity with measured usage, then trim reservations toward observed peaks plus a margin. The Atlas suggests running the vertical autoscaler in recommendation-only mode to gather suggestions without it changing anything.
Symptom two: a container restarts again and again
The story: an application starts, dies, starts again, with longer gaps each time. It resembles a student who keeps aborting the same takeoff roll for a reason nobody has named yet.
Likely cause: the process itself is failing, or its health probe is set up wrongly. The Atlas offers one rule worth printing: a liveness check should only test the process itself, never outside services. If it calls a database that is briefly slow, the platform may restart a perfectly healthy application and make the outage wider. Fix: read the container’s last logs, then simplify the probe so that it answers only whether this process is alive.
Symptom three: a node drain hangs during maintenance
The story: an operator marks a machine for maintenance, and work refuses to leave. In the training analogy, the aircraft will not move because someone has parked in front of the chocks.
Likely cause: a protective budget set so strictly that not a single copy may be offline, or two budgets that both select one pod. The Eviction API answers with 429 when a request is refused for now, and a 500 can point to overlapping budgets. Fix: adjust the budget so at least one copy may be down, retry politely, and recall that forcing deletion bypasses the protection and should be agreed with the owner first.
Symptom four: a rollout stalls and then reports failure
The story: a new version appears, some copies start, then nothing happens for ten minutes and an error appears. Here the Atlas’s default is a 600-second progress deadline.
Likely cause: new copies never become ready, perhaps because of a missing setting, an unreachable dependency or a tight reservation. Fix: inspect the new copies’ events, correct the configuration, and re-deploy. Treat the deadline as a clue about when the clock started, not a verdict.
Symptom five: everything breaks right after an upgrade
The story: workloads cannot be created, or network traffic between machines fails right after a new version is rolled out. It is the equivalent of a student reporting the aircraft behaved differently after the shop returned it.
Likely cause: a removed or changed feature that nobody noticed, or an admission webhook that rejects requests by default when it is down. The Atlas discusses an outage where a label disappeared from control-plane machines during a version move, and a networking component that selected on that label stopped routing traffic. Fix: read release notes for removals before upgrading, and keep such webhooks out of the system namespace and away from the networking add-on, so a failure there cannot trap the cluster.
A short triage order you can memorise
When several symptoms appear together, resist the urge to chase the loudest. Instructors teach a scan: aviate, navigate, communicate. A cluster version of the scan might read as follows.
- Is the control plane answering at all, and how quickly? Slow responses colour every other symptom.
- Are machines ready, and has anything changed on them recently?
- Are workloads placed, or waiting for capacity?
- Are containers healthy, and what do their last messages say?
- What changed in the past day: a version, a setting, a certificate, a budget?
Write the answers down in that order. The pattern often emerges before you finish, and the notes become the first draft of the debrief.
The debrief habit
Instructors close a lesson with notes, and operators can do the same: symptom, cause, fix, and what will prevent a repeat. Over a few months those notes turn into a runbook that a newcomer can follow.
If you are weighing what training costs in flying, two posts here are worth a look. One covers sensible ways to lower the cost of training, and another explains why the choice of training aircraft changes the bill. Both reinforce an idea that applies equally to clusters: choose your equipment for the job, and measure before you spend.

