KCSA Kubernetes Trust Boundaries, Persistence and Denial of Service
The Kubernetes threat model is the map of where trust changes inside a cluster, how an attacker who gets in keeps a foothold, and how a workload or client can exhaust shared resources. This lesson covers the cluster's trust boundaries and data flows, with the API server as the hub and etcd behind it, the weak default trust between control plane and nodes, the shared-kernel boundary between containers and their node, persistence through static Pods, controllers, admission webhooks and stolen credentials, and denial of service with the limits, quotas, PID controls, priority rules and API Priority and Fairness that counter it.
On this page10 sections
- What trust boundaries are in a Kubernetes cluster
- The API server as the hub of every data flow
- Control plane to node traffic: the kubelet and proxy connections
- The container to node boundary and sandboxed runtimes
- Persistence on a node: static Pods and mirror Pods
- Persistence through controllers and admission webhooks
- Persistence through stolen credentials
- Denial of service on a node: requests, limits and process IDs
- Namespace exhaustion and preemption abuse: ResourceQuota, LimitRange and PriorityClass
- Protecting the API server: API Priority and Fairness
- Identify the trust boundaries in a Kubernetes cluster and the controls that defend each one
- Explain why the API server is the hub of every data flow and why direct etcd access bypasses all its checks
- Describe the default weaknesses in control plane to node traffic and how --kubelet-certificate-authority and Konnectivity address them
- Recognise persistence through static Pods, owning controllers and mutating admission webhooks, and remove each correctly
- Compare how long stolen service account tokens and client certificates stay valid, and how to revoke each
- Choose between limits, LimitRange, ResourceQuota, PID limits, PriorityClass controls and API Priority and Fairness for a given denial-of-service risk
What trust boundaries are in a Kubernetes cluster
A trust boundary is any point where a request, data or code passes from a less privileged or less trusted component to a more privileged one, so the receiving side must authenticate, authorize or otherwise check what arrives. Threat modelling a cluster starts by drawing those boundaries, because they are where an attacker tries to cross from what they control to what they want.
The main boundaries in a Kubernetes cluster are:
- Client to API server: every kubectl user, CI system, controller and Pod calling the Kubernetes API. Nothing a client sends is trusted until the API server has checked it.
- API server to etcd: the store of all cluster state, Secrets included.
- Control plane to node: the API server reaching each kubelet for logs, exec, attach and port-forward, and proxying to Pods and Services.
- Container to node: a containerised process and the host kernel it shares with every other container on that node.
- Workload to workload: Pods in different namespaces or tenants talking over the Pod network.
- Cluster to outside: image registries, cloud provider APIs, identity providers and the internet.
A boundary and a control are different things. Namespaces, private networks, RBAC rules and NetworkPolicies are controls that defend boundaries; they are not boundaries in themselves. A namespace, for example, is a naming and policy scope, while the boundary it helps enforce is the privilege change between one tenant's identity and another's resources.
| Boundary | What crosses it | Main defences |
|---|---|---|
| Client to API server | API requests from users, Pods and controllers | Authentication, authorization (RBAC), admission control, audit logging, TLS |
| API server to etcd | All reads and writes of cluster state | Mutual TLS, etcd reachable only from the API server, encryption at rest |
| Control plane to node | kubelet API calls, proxied traffic | Kubelet authentication and authorization, verifying the kubelet's certificate, Konnectivity or a trusted network |
| Container to node | System calls to the shared kernel, host mounts | Pod Security Standards, seccomp, AppArmor or SELinux, sandboxed runtimes |
| Workload to workload | Pod network traffic | NetworkPolicy, mutual TLS between services |
The API server as the hub of every data flow
The kube-apiserver is the hub of every Kubernetes data flow: users, controllers, the scheduler, kubelets and Pods all read and write cluster state through it, and it is the only component that talks to etcd directly. The scheduler, controller manager and kubelets never open etcd themselves; they watch and update objects through the API.
A request to create a Pod shows where the checks sit. The client's call arrives at the API server and crosses the most important boundary in the cluster. The API server then:
- authenticates the caller (client certificate, bearer token, OIDC token and so on);
- authorizes the verb on the resource, normally with RBAC;
- runs admission control: mutating admission (which may change the object), then validating admission such as Pod Security Admission;
- persists the accepted object in etcd and records the request in the audit log if auditing is configured.
Later hops in the same flow (the API server writing to etcd, the scheduler watching for unscheduled Pods, the kubelet reading the Pods bound to its node) are internal flows between components that already authenticate each other, normally over TLS. They are not where an untrusted request is judged.
Because every control is applied in the API server, direct access to etcd is one of the most damaging compromises. Whoever can reach the etcd endpoint with a client certificate it accepts can read and write any object with no RBAC check, no admission and no API audit events. Encryption at rest still protects the resources it is configured for, as long as the encryption keys are not stored with etcd, but everything else is exposed. Treat etcd's network access and its client certificates as control plane secrets, and allow only the API servers to reach it.
Control plane to node traffic: the kubelet and proxy connections
Traffic from the control plane to the nodes is the weakest default boundary in Kubernetes: the API server does not verify the kubelet's serving certificate unless told to, and its proxy connections to nodes, Pods and Services are plain HTTP by default. Node-to-control-plane traffic is the opposite: kubelets and Pods reach the API server over HTTPS, with the cluster CA to verify it and their own credentials to authenticate.
The API server opens connections to each kubelet to serve kubectl logs, exec, attach and port-forward. By default it does not check the certificate the kubelet presents, so on an untrusted or public network a man-in-the-middle can read or alter that traffic, including the commands and output of an exec session. Several kubelet-related flags look similar but protect different directions:
| Setting | Where | What it proves |
|---|---|---|
--anonymous-auth=false | kubelet | Unauthenticated callers are rejected by the kubelet |
--client-ca-file | kubelet | The caller is genuine: the kubelet verifies client certificates, such as the API server's |
--kubelet-client-certificate and --kubelet-client-key | kube-apiserver | The API server's own identity when it calls the kubelet |
--kubelet-certificate-authority | kube-apiserver | The kubelet endpoint is genuine: the API server verifies the kubelet's serving certificate against this CA bundle |
The direction is what separates them. Three of the settings let the kubelet judge who is calling it, which is kubelet hardening (covered in Domain 2). Only the CA bundle configured on the API server lets the caller judge the kubelet, so it is the one that protects this hop against interception.
The API server can also proxy requests to nodes, Pods and Services (for example a path ending in /proxy/). Those connections default to plain HTTP, neither authenticated nor encrypted. Adding the https: prefix to the Pod or Service name in the proxy URL turns on TLS, but the API server does not validate the endpoint's certificate and presents no client credentials. The hop is encrypted, yet there is no guarantee of who is on the other end, so it is not safe across an untrusted network.
When nodes sit on a network the control plane cannot trust, Kubernetes offers the Konnectivity service: a Konnectivity server in the control plane network and agents on the nodes that open the connections outward, so control plane to node traffic travels through that tunnel. SSH tunnels are the older, deprecated alternative.
The container to node boundary and sandboxed runtimes
The boundary between a container and its node is weaker than the boundary between two virtual machines because every container on a node runs on the same Linux kernel. A container is an ordinary process confined by namespaces, cgroups, capabilities, seccomp and a mandatory access control profile; a kernel vulnerability, a privileged container or a sensitive hostPath mount can carry an attacker straight through that confinement to the node.
Every one of those confinement mechanisms is itself a kernel feature. The kernel is therefore the single component that all workloads on the node depend on at once, and a flaw in it, or a workload given enough privilege to reach past it, affects them all. That is why threat models put the container boundary below a hypervisor boundary.
For untrusted code, Kubernetes supports sandboxed runtimes that put another kernel layer between the workload and the host: gVisor runs a user-space kernel that intercepts system calls, and Kata Containers runs each Pod in a lightweight virtual machine. They are exposed through RuntimeClass objects, each naming a handler configured in the container runtime, and a Pod selects one with spec.runtimeClassName.
| Control | What it does | Does it remove the shared kernel? |
|---|---|---|
| RuntimeClass with gVisor or Kata Containers | Runs the Pod under a sandboxed runtime | Yes: the workload talks to a separate kernel layer |
seccomp RuntimeDefault | Filters the system calls a container may make | No: it reduces the attack surface of the same kernel |
| AppArmor or SELinux profile | Limits files and operations the process can touch | No |
| Dedicated node pool (nodeSelector, taints) | Keeps untrusted Pods on separate nodes | No: it limits the blast radius to those nodes |
What an attacker does after crossing this boundary (container escape techniques, lateral movement) belongs to the sibling topic on compromised containers and privilege escalation.
Persistence on a node: static Pods and mirror Pods
A static Pod is a Pod the kubelet runs directly from a manifest file on the node's local disk, without the API server or scheduler, which makes it a favoured way for an attacker with root on a node to keep a workload running. The kubelet watches the directory set by staticPodPath in its configuration (commonly /etc/kubernetes/manifests; kubeadm runs the control plane itself this way) and starts, restarts and keeps any Pod manifest it finds there, even while the API server is unreachable.
So that static Pods are visible, the kubelet creates a mirror Pod for each one on the API server. The mirror is read-only from the cluster's point of view:
kubectl deleteon the mirror succeeds, but the static Pod keeps running and the kubelet recreates the mirror moments later.- Its
ownerReferencespoint at the Node object rather than a workload controller, and its name ends with the node's hostname, which is a quick way to recognise one. - Changes made through the API never reach the kubelet's copy, because the file on disk is the only source of truth for that Pod.
The only way to stop a static Pod is to delete its manifest file from the node's static Pod directory (or change the kubelet's configuration). An attacker does not even need a node shell to plant one: any Pod that can mount the host's /etc/kubernetes through a hostPath volume can write a manifest there. Closing that route means enforcing at least the Baseline Pod Security Standard on namespaces that have no need for host access. Baseline forbids hostPath volumes, host namespaces and privileged containers, the known routes to the node, while typical application Pods pass it unchanged.
Persistence through controllers and admission webhooks
An attacker who can create workloads persists by creating a controller, because the controller recreates its Pods whenever they are deleted; removing the foothold means deleting the top-level owner, not the Pod. Pods created by a controller carry an ownerReferences entry, which tells responders where to look.
| Object | How it brings Pods back | Persistence value to an attacker |
|---|---|---|
| Deployment | Manages a ReplicaSet, which replaces missing replicas | Keeps a fixed number of attacker Pods running |
| DaemonSet | Runs one Pod on every eligible node, including nodes added later | Reaches every node; with tolerations for every taint, the control plane nodes too |
| CronJob | Creates a new Job, and so new Pods, on its schedule | Periodic re-execution that survives Pod and Job deletion |
| StatefulSet | Keeps a set number of named replicas | Fixed replica count, no automatic reach to new nodes |
Only objects that hold a Pod template create Pods. Objects that refer to a workload without a template, such as a HorizontalPodAutoscaler adjusting its target's replica count, act on the controller rather than producing Pods, so following ownerReferences upwards is the reliable way to find the source. The counters are tight RBAC on workload creation (especially in kube-system) and audit logs that show who created the owning object.
A stealthier, cluster-wide persistence point is an admission webhook. The API server calls every mutating admission webhook whose rules match a request and applies the patches it returns, before the object is stored. An attacker who can create or modify mutatingwebhookconfigurations (API group admissionregistration.k8s.io) can point one at a service they control and inject a backdoor container into every new Pod. Because the change is made inside the API server, the manifests in source control still look clean; only the stored objects and the webhook configuration show it.
- Mutating webhooks see and can change every matching object.
- Validating webhooks see every matching object and can allow or deny it, but cannot change it. They still leak data and can block the cluster, so their configurations need the same protection.
The Kubernetes RBAC good practices treat write access to webhook configurations as highly privileged. Restrict it to cluster administrators and audit every change to these objects.
Persistence through stolen credentials
Stolen credentials let an attacker return without any workload at all, and how long they stay useful depends on the credential type: bound service account tokens expire and die with their Pod, while legacy token Secrets and client certificates can last until someone deliberately revokes trust in them.
| Credential | Lifetime | How to revoke it |
|---|---|---|
| Projected (bound) service account token from the TokenRequest API | Time-limited, with an audience, bound to its Pod; the kubelet refreshes it while the Pod runs | Expires on its own; rejected once the Pod it is bound to is deleted |
Legacy token in a kubernetes.io/service-account-token Secret | No expiry and not bound to any Pod | Delete that Secret: the API server checks the Secret still exists (--service-account-lookup, on by default) |
| Client certificate signed by the cluster CA | Until the certificate expires | No revocation list; only rotating the CA it was signed by |
| OIDC token from an identity provider | Short-lived ID token | Disable the user at the identity provider and let tokens expire |
Pods have received projected tokens by default for several releases, and since Kubernetes v1.24 no legacy token Secret is created automatically for a ServiceAccount. Legacy Secrets still exist where someone created one by hand or where they survive from older clusters; Kubernetes also labels auto-generated ones that go unused as invalid and later removes them. A legacy token is not tied to any Pod, so its validity depends only on the Secret that holds it. Removing that Secret is the narrowest revocation: it invalidates one token. Broader measures on the ServiceAccount or the signing key work too, but they also invalidate every other credential that depends on them, so incident responders weigh the blast radius before choosing one.
Client certificates are the hardest case. Kubernetes has no certificate revocation list for them, so a certificate is valid until it expires or until its signing CA is no longer trusted. A certificate with the organization system:masters is worse still: members of that group bypass RBAC entirely. The group membership is written into the certificate, and the authorizer grants that group everything without consulting any binding. RBAC rules are also purely additive, so there is no RBAC change that can take the access away. Closing that foothold means rotating the cluster CA and reissuing the certificates it signed, which is why system:masters certificates should never be issued to people; give administrators credentials from an identity provider and grant access through RBAC instead.
Denial of service on a node: requests, limits and process IDs
A node-level denial of service happens when one container consumes so much CPU, memory or process IDs that its neighbours, the kubelet or the host itself cannot work, and the counters are container limits plus the kubelet's PID limits and reservations.
Requests tell the scheduler how much a container needs and guarantee it that share; limits cap what it may actually use. A container with requests but no limits can grow beyond its request into all the node's spare capacity. When a container sets a limit but no request, Kubernetes uses the limit as its request. A LimitRange can supply default limits where teams forget them.
| Resource | Type | Over the limit |
|---|---|---|
| Memory | Incompressible | The kernel's out-of-memory killer ends the process; the container reports OOMKilled and restarts per its restart policy |
| CPU | Compressible | The container is throttled to its limit through a CFS quota; it is not killed, and neighbours keep their share |
Requests and limits also set a Pod's quality-of-service class (Guaranteed, Burstable or BestEffort), which roughly predicts eviction order when the node runs short; the kubelet itself ranks Pods by whether their usage exceeds their requests, then by Pod priority, then by usage relative to requests. Node-pressure eviction responds to memory, disk and PID pressure (memory.available, nodefs.available, pid.available and similar); CPU is not an eviction signal, so a CPU-hungry container without a limit simply competes for CPU rather than being evicted. Scheduling decisions happen only when a Pod is placed; contention after that is handled on the node, by throttling, the OOM killer and eviction.
Process IDs are easy to exhaust with a fork bomb without touching CPU or memory limits, and running out destabilises the whole node. The kubelet has two PID controls, and only one of them caps a Pod:
podPidsLimitin the kubelet configuration (--pod-max-pids) caps the PIDs available to each Pod, so a fork bomb hits its own ceiling. A Pod spec cannot set this itself.pid=<number>insystemReservedandkubeReservedsets PIDs aside for the operating system and Kubernetes daemons, so they can still start processes when Pods together use many. It is a reservation for the daemons, not a ceiling on any workload.
A pid.available eviction threshold acts later still: once the node as a whole is short of PIDs, the kubelet evicts Pods to recover. It is a backstop that reacts to node-wide pressure rather than a limit set in advance.
Namespace exhaustion and preemption abuse: ResourceQuota, LimitRange and PriorityClass
A ResourceQuota caps the total resources and object counts a namespace may consume, a LimitRange sets per-container defaults and bounds, and together they stop one tenant exhausting a shared cluster. The RBAC good practices warn that anyone allowed to create objects can create enough of them, or large enough ones, to cause a denial of service, including filling etcd.
| Object | Scope | Typical use against DoS |
|---|---|---|
| ResourceQuota | Namespace total | Aggregate requests.cpu, limits.memory, storage, and object counts such as pods, count/configmaps, count/secrets |
| LimitRange | Each container, Pod or claim in a namespace | Default requests and limits, plus minimum and maximum values per object |
Two behaviours of ResourceQuota matter in practice:
- It is enforced at admission. Each create request, and each update that changes resource usage, is checked against the remaining quota and rejected if it would exceed it. A quota therefore acts only on future requests: resources that already exist are left as they are, even if usage now sits above the hard limit, and further creations are refused until usage falls below it.
- A CPU or memory quota makes requests or limits mandatory. Once a quota covers
limits.cpuorlimits.memory, every new Pod must declare them or admission rejects it. A LimitRange in the same namespace can supply default values: its admission step fills in missing requests and limits before the quota check runs, so Pods that declare none are still admitted.
Preemption is a scheduling route to denial of service. A PriorityClass gives Pods a priority value; when a pending Pod cannot be placed, the scheduler evicts lower-priority Pods to make room (the default preemptionPolicy is PreemptLowerPriority). A tenant allowed to use a high class can therefore displace other tenants whenever the cluster is full. Controls:
preemptionPolicy: Neveron a class keeps its Pods ahead in the scheduling queue but stops them evicting anyone.- A ResourceQuota with a
scopeSelectoronPriorityClasscaps how many Pods, or how much CPU and memory, a namespace may use at that priority. Because quota is set per namespace, each namespace can get its own allowance for the class, including none. - The ResourceQuota admission configuration's
limitedResourcescan make a class usable only in namespaces that have a matching quota.
Protecting the API server: API Priority and Fairness
API Priority and Fairness (APF) protects the API server from overload by sorting every incoming request into a priority level with its own share of concurrency and its own queues, so one noisy client cannot starve the others. It is enabled by default and stable.
- A FlowSchema matches requests by subject (user, group or ServiceAccount) and by resource or verb, and assigns them to a priority level. It also defines how requests are split into flows, for example per user.
- A PriorityLevelConfiguration holds a share of the server's total concurrency and the queuing settings for the requests assigned to it.
- Inside a level, fair queuing spreads capacity across flows, so one flow's burst queues behind its own requests instead of everyone's.
Kubernetes ships default FlowSchemas and levels, including protected levels for system components and leader election and an exempt level for system:masters. Administrators can add their own FlowSchemas to route a particular user, group or ServiceAccount to a level with a smaller share. Requests in that level are queued when it is busy and rejected (HTTP 429) only when its queues are full, while the other levels keep the shares they had.
The older --max-requests-inflight and --max-mutating-requests-inflight flags are blunt server-wide caps: lowering them reduces total work but cannot protect important requests from unimportant ones. With APF enabled, their sum sets the server's total concurrency, which APF then divides among priority levels. Despite the shared word, APF priority levels are unrelated to PriorityClass, which orders Pods for the scheduler.
Tip. KCSA questions on this topic are mostly short definitions and scenarios from a threat modeller's or incident responder's point of view. Expect to identify which hop on a request's path is a trust boundary, which component talks to etcd and what direct etcd access bypasses, and which API server setting protects traffic to kubelets or what a proxy hop does and does not guarantee. Persistence questions describe something that keeps coming back or keeps working after a clean-up and ask what is causing it or what removes it: a static Pod, an owning controller, a mutating webhook, a legacy token or a system:masters certificate. Denial-of-service questions ask what happens when a container passes a CPU or memory limit, which kubelet setting contains a fork bomb, how ResourceQuota and LimitRange interact, how preemption can be abused, and how to contain a noisy API client. Distractors are real Kubernetes controls that act on the wrong resource, direction or scope, and some scenarios add an operational constraint that a heavier fix would break.
- A trust boundary is where data or requests pass to a more privileged component; namespaces, RBAC and NetworkPolicies are controls that defend boundaries, not boundaries themselves.
- The API server is the only client of etcd, so direct etcd access skips authentication, RBAC, admission and audit logging.
- By default the API server does not verify the kubelet's serving certificate; --kubelet-certificate-authority fixes that, and Konnectivity protects traffic across untrusted node networks.
- API server proxy connections are plain HTTP by default; an https: prefix encrypts them but does not validate the endpoint's certificate.
- Containers share the node's kernel; a RuntimeClass running gVisor or Kata Containers is the control that adds a separate kernel layer.
- A static Pod survives kubectl delete of its mirror Pod; only deleting its manifest on the node removes it.
- Controller-owned attacker Pods come back until the Deployment, DaemonSet or CronJob that owns them is deleted; mutating webhook configurations are a cluster-wide persistence point.
- Deleting a legacy token Secret revokes that token; system:masters certificates bypass RBAC and cannot be revoked short of rotating the cluster CA.
- Memory over the limit is OOMKilled, CPU over the limit is throttled; podPidsLimit caps a Pod's PIDs while systemReserved and kubeReserved PIDs protect daemons.
- ResourceQuota is checked at admission and never removes existing Pods; priority-scoped quotas stop preemption abuse and FlowSchemas contain noisy API clients.
Check yourself: 3 questions on this topic
Exam-style KCSA questions on Trust boundaries, data flow, persistence and denial of service, answered right here with an explanation after each. Want more? KCSA Kubernetes Threat Model practice questions.
Frequently asked questions
What are the main trust boundaries in a Kubernetes cluster?
The main Kubernetes trust boundaries are client to API server, API server to etcd, control plane to node (the API server calling kubelets and proxying to Pods and Services), container to node (the shared host kernel), workload to workload over the Pod network, and the cluster to external systems such as registries and cloud APIs. At each one, the receiving side must authenticate, authorize or otherwise check what arrives.
Why is direct access to etcd so dangerous in Kubernetes?
The kube-apiserver is the only component meant to talk to etcd, and it applies authentication, RBAC authorization, admission control and audit logging before any data is stored. Someone who can reach etcd directly with an accepted client certificate reads and writes cluster state with none of those checks and leaves no API audit events. Encryption at rest still protects the resources it covers if its keys are kept separate from etcd.
How do you remove a static Pod that keeps coming back after kubectl delete?
A static Pod is run by the kubelet from a manifest in its static Pod directory (staticPodPath, often /etc/kubernetes/manifests), and the Pod seen through the API is only a mirror. Deleting the mirror with kubectl does not stop it, and the kubelet recreates the mirror. Remove it by deleting the manifest file on that node, then close the route the attacker used, for example hostPath mounts blocked by the Baseline Pod Security Standard.
How do mutating admission webhooks give an attacker persistence?
The API server sends every matching request to each registered mutating admission webhook and applies the changes it returns before storing the object. An attacker who can create a MutatingWebhookConfiguration can point it at a service they run and, for example, inject a backdoor container into every new Pod in the cluster, including Pods created from clean manifests. Restrict write access to webhook configurations to administrators and audit changes to them.
Can you revoke a Kubernetes client certificate?
Kubernetes does not support certificate revocation lists for client certificates, so a certificate signed by the cluster CA stays valid until it expires or the CA is no longer trusted. If the certificate's organization is system:masters, its holder bypasses RBAC completely, so removing role bindings does nothing. Closing that access requires rotating the cluster CA and reissuing the certificates it signed, which is why system:masters certificates should never be given to people.
What is the difference between a Kubernetes CPU limit and a memory limit when a container exceeds it?
CPU is compressible: a container that tries to use more than its CPU limit is throttled to that amount and keeps running, so its neighbours keep their share. Memory is incompressible: a container that exceeds its memory limit is ended by the kernel's out-of-memory killer, reported as OOMKilled and restarted according to its restart policy.
How do you stop a fork bomb in one Pod from taking down a Kubernetes node?
Set podPidsLimit in the kubelet configuration (the --pod-max-pids flag) to cap the process IDs each Pod can use, so a fork bomb hits its own ceiling. Add pid reservations in systemReserved and kubeReserved so the operating system and Kubernetes daemons can still start processes. A pid.available eviction threshold is only a node-level backstop and does not cap any single Pod.
What does API Priority and Fairness do in Kubernetes?
API Priority and Fairness is the kube-apiserver feature, on by default, that classifies each incoming request with a FlowSchema and assigns it to a priority level with its own share of concurrency and its own queues, with fair queuing between flows inside a level. It stops one noisy client, such as a controller flooding the server with list requests, from starving other clients, which the server-wide --max-requests-inflight limits cannot do.
Source
This lesson covers the "Kubernetes Threat Model" domain of the official KCSA exam guide. Vendors revise their guides — check the source for the current version.
- KCSA certification page and curriculum — The Linux Foundation
Sign up free to mark lessons complete, bookmark topics and track your exam readiness.