# Kubernetes Workload Readiness

> Kubernetes will keep a badly-configured workload running in a way that looks healthy. Most of
> the fields below exist to make the difference between "the pod is up" and "the service is
> working" visible before customers find it.

**Workload:** _______________  **Namespace:** _______________
**Owner (person):** _______________  **Date:** _______
**Cluster / environment:** _______________

## 1. Shape

| | |
|---|---|
| Workload type | Deployment / StatefulSet / Job / CronJob / DaemonSet |
| Replicas (min / max) | |
| Stateless | yes / no — if no, what state and where |
| Can two instances run at once safely | yes / no |
| Startup time to ready | |

## 2. Probes — the three, and what each means

| Probe | Endpoint | What it checks | Timing |
|---|---|---|---|
| Startup | | | |
| Liveness | | | |
| Readiness | | | |

- **Liveness** answers *should this container be restarted*. It must check only the process
  itself.
- **Readiness** answers *should this pod receive traffic*. It may check dependencies.
- **Startup** protects a slow-starting application from being killed before it is up.

🔴 Pointing liveness at a check that includes a dependency creates a cluster-wide restart storm
the moment that dependency is briefly slow. The database hiccups; every pod fails liveness; every
pod restarts; the database is now handling a thundering reconnect. Keep them separate.

**Slow-start applications:** startup probe configured? yes / no
(Without one, a long boot looks like a liveness failure and the pod never gets to start.)

## 3. Resources

| | Request | Limit | Basis |
|---|---|---|---|
| CPU | | | measured / guessed |
| Memory | | | measured / guessed |

- Requests set from observed usage: yes / no — date measured: _______
- Memory limit == request (guaranteed class)? yes / no
- Runtime told about the container memory limit: yes / no

> A CPU limit throttles; a memory limit kills. Setting memory requests too low gets pods
> scheduled onto nodes that cannot really hold them; setting them far too high wastes a cluster.
> Both are corrected by measurement, not by convention.

## 4. Disruption and scheduling

| | |
|---|---|
| PodDisruptionBudget (minAvailable / maxUnavailable) | |
| Anti-affinity so replicas are not on one node | yes / no |
| Topology spread across zones | |
| Tolerations / node selectors, and why | |
| Priority class | |

**What happens when a node is drained?** _______________
**What happens when a node dies without warning?** _______________

If both answers are "the replicas move", check that the disruption budget and anti-affinity
actually allow it — a single-replica workload with a strict budget blocks node maintenance
entirely.

## 5. Shutdown

| | |
|---|---|
| SIGTERM handled gracefully | yes / no |
| terminationGracePeriodSeconds | |
| preStop hook (e.g. brief sleep to drain endpoints) | |
| In-flight requests completed | yes / no |

There is a genuine race at shutdown: a pod can stop receiving traffic slightly after it starts
terminating, because endpoint removal propagates asynchronously. A short preStop delay is the
standard mitigation and is worth having wherever dropped requests matter.

## 6. Configuration and secrets

| | |
|---|---|
| Config source | ConfigMap / env / mounted file |
| Secrets source | |
| Secrets encrypted at rest in etcd | yes / no |
| Rotation picked up without a redeploy | yes / no |
| No secrets in the manifest or in git | confirmed |

## 7. Security context

| | |
|---|---|
| runAsNonRoot | yes / no |
| Read-only root filesystem | yes / no |
| Capabilities dropped | |
| allowPrivilegeEscalation | |
| Service account: dedicated, minimal RBAC | yes / no |
| Automount of the service account token needed | yes / no |

## 8. Network

| | |
|---|---|
| NetworkPolicy applied (default deny?) | |
| Ingress path, TLS termination | |
| Egress required to | |
| Service type | |

By default, everything in a cluster can talk to everything. A namespace with no network policy is
a flat network, and the first lateral movement in an incident goes wherever it likes.

## 9. Observability and operations

- [ ] Logs to stdout, collected centrally
- [ ] Metrics scraped, dashboard exists
- [ ] Alerts route to a human who can act
- [ ] HPA configured on a metric that reflects real load
- [ ] Runbook for the three most likely failures
- [ ] Someone would notice if the workload were scaled to zero

## 10. Before production

- [ ] Rolling update tested — no dropped requests observed
- [ ] Pod killed deliberately; recovery verified
- [ ] Node drained deliberately; behaviour verified
- [ ] Resource limits verified under load, not at rest
- [ ] Rollback of a bad image tested

## Sign-off

| | Name | Date |
|---|---|---|
| Built by | | |
| Platform review | | |
| Approved for production | | |