KCSA Container Networking, Client Security and Storage Security
Container networking, client security and storage security are the parts of Kubernetes cluster component security that protect the Pod network, the credentials clients use to reach the API server, and the volumes where workloads keep data. On the network side, this KCSA lesson explains the flat Pod network, the CNI plugin as a privileged component, encrypting Pod traffic and limiting access to the cloud metadata API. On the client side, it covers kubeconfig files, TLS verification, revocation, short-lived OIDC tokens, kubeadm admin files and kubectl proxy. On the storage side, it covers hostPath volumes, PersistentVolume creation rights, reclaim policies, access modes and volume encryption.
On this page10 sections
- What container networking, client and storage security cover
- The flat Kubernetes network model: any Pod can reach any Pod
- The CNI plugin: a privileged component that wires every Pod
- Encryption in transit: Pod traffic is plaintext unless you add it
- Protecting the cloud instance metadata API from Pods
- kubeconfig files: a portable identity that must be trusted and verified
- Client credentials: short-lived tokens, individual identities and revocation
- Admin credentials and kubectl proxy
- Volumes and hostPath: keep Pods away from the node's filesystem
- Persistent storage: PV creation rights, reclaim policies, access modes and encryption
- Explain the flat Kubernetes network model and why namespaces, ClusterIP Services and selectors do not restrict which Pods can connect
- Describe what the CNI plugin does, why its binaries and configuration are root-equivalent, and how it differs from kube-proxy, the scheduler and CoreDNS
- Compare CNI node encryption, service mesh mutual TLS and application TLS for encrypting Pod traffic, and explain why node encryption does not prove workload identity
- Restrict Pod access to the cloud instance metadata API with an egress network policy and explain why RBAC and token settings do not
- Protect kubeconfig files and client credentials: verify the API server's CA, review untrusted files, prefer short-lived OIDC tokens and revoke access through RBAC
- Distinguish kubeadm's admin.conf from super-admin.conf and explain the risk of exposing kubectl proxy
- Secure storage by limiting hostPath and PersistentVolume creation, choosing reclaim policies and access modes, and encrypting volume data through StorageClasses
What container networking, client and storage security cover
This part of Kubernetes cluster component security protects the three things that sit between the control plane and the workloads: the Pod network that carries application traffic, the client credentials people and tools use to reach the API server, and the volumes where Pods keep data. Each has a default that is more open than many teams assume.
| Area | What it is | The default to know | Main controls |
|---|---|---|---|
| Container networking | The CNI plugin and the Pod-to-Pod network it builds | Every Pod can reach every other Pod; traffic between Pods is not encrypted | NetworkPolicy enforced by the CNI, CNI node encryption, service mesh mutual TLS, egress limits on the cloud metadata API |
| Client security | kubeconfig files, kubectl and the credentials they carry | A kubeconfig file is a portable identity; issued client certificates cannot be revoked | Short-lived OIDC tokens, individual least-privilege identities, CA verification, trusted files only, break-glass admin credentials |
| Storage | Volumes, PersistentVolumes, claims and StorageClasses | hostPath exposes the node; deleted claims can leave data behind; API server encryption does not cover volume data | Pod Security Standards, PV creation kept with admins, reclaim policies, encrypting StorageClasses, narrow access modes |
The control plane components themselves (API server, controller manager, scheduler, etcd) and the node components (kubelet, container runtime, kube-proxy) are covered in their own KCSA lessons. How to write NetworkPolicy rules and how the Pod Security Standards are enforced are covered in the Kubernetes Security Fundamentals domain; this lesson uses them only where networking, client access or storage depends on them.
The flat Kubernetes network model: any Pod can reach any Pod
By default every Pod in a Kubernetes cluster can open a connection to every other Pod, on any node and in any namespace, because the network model gives each Pod its own IP address that the whole cluster can reach without NAT. Nothing blocks that traffic until a network policy does.
The Kubernetes network model requires the network plugin to provide three things: every Pod gets a unique cluster-wide IP, Pods on any node can talk to Pods on every other node directly, and node agents can reach the Pods on their node. This makes service discovery and routing simple, and it also means the network is flat and allow-all out of the box.
Namespaces are not a network boundary
A namespace scopes object names, RBAC permissions, quotas and Pod Security labels. It does not separate traffic. A namespace boundary stops a user from listing or editing another team's objects, but a packet sent to a Pod IP is never checked against namespaces at all. Separating the API objects is therefore not the same as separating the traffic; only a network policy that the CNI plugin enforces does the second.
Services control exposure, not callers
| Service setting | What it actually controls | What it does not control |
|---|---|---|
type: ClusterIP | The Service gets a virtual IP reachable only from inside the cluster (every Pod and node) | Which namespaces or Pods may call it; every Pod in the cluster can |
selector | Which Pods are the backends that receive the traffic | Which Pods are allowed to be clients |
DNS name (<service>.<namespace>.svc.cluster.local, where cluster.local is the default cluster domain) | How the Service is found by name | Access; the namespace in the name is not a permission check |
NodePort / LoadBalancer | Exposure on node addresses or an external load balancer | Authentication of callers |
To limit who can call a workload you need network policies on the backend Pods, enforced by a CNI plugin that supports them, plus authentication in the application itself. A NetworkPolicy object has no effect if the installed plugin does not enforce it.
The CNI plugin: a privileged component that wires every Pod
The Container Network Interface (CNI) plugin is the component that gives each new Pod its network interface and IP address and connects it to the node's network; the container runtime calls it every time it creates a Pod sandbox. Kubernetes defines the network model but does not implement it; the plugin the operator installs does.
| Component | Networking job |
|---|---|
| CNI plugin | Creates the Pod's interface, allocates its IP, sets up routes; many plugins also enforce NetworkPolicy and offer node-to-node encryption |
| kube-proxy | Programs iptables, IPVS or nftables rules that implement Service virtual IPs; it does not create Pod interfaces or allocate Pod IPs |
| kube-scheduler | Chooses a node for the Pod; it reserves no IP address |
| CoreDNS | Answers name lookups for Services and Pods; it configures no interfaces |
Why the CNI files are as sensitive as root
On each node the plugin's executables live in /opt/cni/bin and its configuration in /etc/cni/net.d (the usual default paths). The runtime runs those binaries as root for every Pod it starts. Anyone who can write to those directories can therefore run their own code as root on the node and reroute or capture every Pod's traffic. Only root should be able to modify them, and changes should be detected.
The kubelet's credentials, the runtime socket and static Pod manifests are sensitive too, but they live in other locations; node hardening is covered in the node security lesson.
Choosing a CNI plugin is a security decision: check that it enforces NetworkPolicy (including egress rules), whether it can encrypt traffic between nodes, and that it comes from a trusted, maintained source.
Encryption in transit: Pod traffic is plaintext unless you add it
Kubernetes does not encrypt application traffic between Pods; it travels across the Pod network exactly as the application sent it unless the CNI plugin, a service mesh or the application itself adds encryption. The TLS that Kubernetes configures protects control plane connections: clients to the API server, the API server to etcd, and the API server to kubelets (encrypted, but the kubelet's serving certificate is verified only when the API server is given the kubelet CA with --kubelet-certificate-authority). None of those connections carries the traffic Pods send to each other.
| Option | How it works | Changes Pods or code? | Proves which workload is calling? |
|---|---|---|---|
| CNI node-to-node encryption (for example WireGuard or IPsec) | The plugin encrypts Pod traffic as it leaves one node and decrypts it on the other | No; transparent to Pods | No; the keys belong to nodes, so the receiver learns only which node sent the packet |
| Service mesh mutual TLS | Proxies (traditionally sidecar containers in each Pod) set up mTLS using certificates issued per workload, often tied to the service account | Sidecar meshes add containers to Pods; no application code change | Yes; each side presents a workload certificate that policy can check |
| Application TLS | Each service serves and calls over HTTPS/TLS itself | Yes; every application must change | Only if the application also checks client certificates |
API server EncryptionConfiguration | Encrypts chosen API resources (such as Secrets) at rest in etcd | - | Not a network control at all |
Encryption is not workload identity
Two different requirements are often confused. Encrypting traffic between nodes is a confidentiality requirement, and CNI encryption meets it. Letting a service accept calls only from one named workload, proved cryptographically, is an identity requirement: it needs per-workload credentials, such as the certificates a mesh issues for mutual TLS, plus authorization rules that check them. Node encryption cannot meet it, because its keys identify nodes, and many unrelated Pods share a node. When a scenario adds constraints, compare them with the "Changes Pods or code?" column above. Some meshes also offer modes without sidecars, using per-node proxies.
Protecting the cloud instance metadata API from Pods
On cloud-hosted clusters a Pod can usually reach the provider's instance metadata API at the link-local address 169.254.169.254, and that endpoint can hand out temporary credentials for the node's cloud identity, so the Kubernetes documentation recommends restricting Pod access to it. A compromised Pod that reads those credentials can act in the cloud account with the node's permissions.
Pod traffic leaves through the node, so by default it reaches the same metadata endpoint the node uses. The endpoint is a cloud service, not a Kubernetes one: it answers HTTP requests about the instance and returns the instance role's credentials.
Which controls fit
Because the threat is a network connection the Pod opens to a cloud address, the fitting controls act on that connection or on what the endpoint will return:
- Egress NetworkPolicy. A policy that selects the workload Pods and allows egress to
0.0.0.0/0with anexceptentry of169.254.169.254/32stops them reaching the endpoint, provided the CNI plugin enforces policy. Direction matters: egress rules filter connections a selected Pod opens, ingress rules filter connections that arrive at it. - Cloud provider settings. Requiring session tokens, limiting how many network hops a metadata response may travel, and using the provider's workload identity feature so Pods never need the node's credentials.
Kubernetes API controls do not reach this path. RBAC decides which Kubernetes API requests a subject may make, and service account token settings govern the Kubernetes credential mounted in a Pod; neither filters packets to a cloud address.
Practical notes on ipBlock
ipBlock is intended for addresses outside the cluster. How a plugin treats Pod IPs that fall inside a broad CIDR such as 0.0.0.0/0 varies, so allow in-cluster destinations with Pod or namespace selectors where you need them. Once a Pod is selected by any egress policy, everything not allowed is denied, so make sure the policies also allow what the workload really needs, such as DNS to the cluster DNS Pods.
System components that genuinely need the metadata API (for example a cloud controller or node agent) can be allowed by a separate policy that selects only them.
kubeconfig files: a portable identity that must be trusted and verified
A kubeconfig file tells kubectl and other clients which clusters exist, which credentials to use and which pairing of the two (a context) to use by default, so whoever holds the file holds those credentials and must be treated like a password.
| Section | Contains | Security relevance |
|---|---|---|
clusters | server URL, certificate-authority or certificate-authority-data (the CA's public certificate), optionally insecure-skip-tls-verify | Decides whether the client verifies the API server; holds no private keys |
users | A client certificate and private key, a bearer token, or an exec credential plugin | The actual secret: embedded keys and tokens are a stolen identity; exec runs a local program |
contexts | A cluster + user pair and a default namespace | A convenience only; the default namespace restricts nothing |
The secret in a client kubeconfig is the users entry: whoever has a copy can authenticate as that user. Cluster-side secrets and permissions stay on the cluster, and every request is authorized by the API server, never by kubectl. Keep the file readable by its owner only and avoid copying it between machines.
Verify the API server: never skip TLS verification
TLS gives encryption and server authentication. insecure-skip-tls-verify: true keeps the encryption but drops the authentication, so kubectl will send its credentials to anyone in the network path who answers as the API server. When kubectl reports x509: certificate signed by unknown authority, the correct fix is to add the cluster's CA certificate as certificate-authority-data (or a certificate-authority file path). The error says the client cannot verify the server, so the fix belongs on that side of the trust relationship: give the client the CA, rather than removing the check. (Current API server releases no longer offer an insecure HTTP port.)
Use kubeconfig files only from trusted sources
The Kubernetes documentation warns to inspect an unfamiliar kubeconfig carefully before using it. A crafted file can make kubectl run local programs through an exec credential plugin under users, and can point certificate and key fields at local files. The risk lands on the machine that runs kubectl, with the privileges of the person running it. Review first the entries that execute something or read local paths; descriptive fields are lower risk, though the server URL still decides where your credentials go.
Client credentials: short-lived tokens, individual identities and revocation
The safest client credentials are short-lived and tied to one person with least-privilege RBAC, because Kubernetes cannot revoke an individual client certificate and a stolen long-lived credential keeps working until it expires.
| Credential | How the API server validates it | How access is cut off |
|---|---|---|
| X.509 client certificate | Accepts any unexpired certificate signed by its client CA; user name from the subject's Common Name, groups from Organization | No revocation list: the old certificate stays valid until it expires. Remove the RBAC bindings for its user name and for its groups, and give the person a new user name |
OIDC ID token (usually fetched by an exec plugin) | Checks the token's signature against the provider's published signing keys and its expiry, locally; it does not ask the identity provider about each request | Disabling the identity provider account stops new tokens; a token already cached stays valid until it expires. Removing the user's RBAC bindings closes that window at once |
| Static bearer token | Compared with a token list configured on the API server | Requires changing API server configuration; avoid for people |
| Service account token | Signed by the cluster; current tokens are short-lived and bound to a Pod | Delete the service account or its bindings; for workloads, not people |
There are no User objects
Kubernetes has no User resource to delete: a "user" is just the name and groups an authenticator reports. A leaked certificate keeps authenticating until it expires, so cutting it off means leaving its identity with no permissions. Removing the user-name bindings is enough only if the certificate's groups (its Organization fields) have no bindings of their own and none of them is system:masters, which bypasses RBAC. Because the old certificate still carries the old name, any binding later created for that name would empower it again, which is why the person moves to a new user name.
Limiting damage from a stolen kubeconfig
- Short-lived credentials: an exec plugin that obtains tokens from an OIDC identity provider means a stolen file holds no long-lived secret, and the provider can refuse future tokens.
- Individual, least-privilege identities: one identity per person with narrow RBAC limits the blast radius and allows revoking one person without affecting others. Shared admin credentials do the opposite.
- Know what does not help: controls that protect the connection or change client defaults do nothing against an attacker who connects normally with a valid stolen credential.
Admin credentials and kubectl proxy
Full-control credentials should be revocable and handed out rarely, and local helpers such as kubectl proxy must stay bound to localhost, because both turn one credential into broad access for whoever holds or reaches it.
kubeadm: admin.conf versus super-admin.conf
| File | Identity's group | How it gets full control | Can access be withdrawn? |
|---|---|---|---|
/etc/kubernetes/admin.conf | kubeadm:cluster-admins | A ClusterRoleBinding to the cluster-admin ClusterRole | Yes; remove the binding |
/etc/kubernetes/super-admin.conf (current kubeadm releases) | system:masters | The API server authorizes this group before RBAC is consulted | No binding change can stop it; it lasts until the certificate expires |
The difference that matters is how access can be withdrawn, not how the files connect. Keep super-admin.conf offline as a break-glass credential, hand out admin.conf only when necessary, and give day-to-day administrators their own identities bound through RBAC. The RBAC good practices likewise advise against putting users in system:masters.
kubectl proxy
kubectl proxy runs a local HTTP listener that forwards requests to the API server using the credentials of the person who started it; the listener itself asks callers for nothing. By default it binds to 127.0.0.1 and accepts only localhost host headers, which keeps it personal. Widening --address to all interfaces and --accept-hosts to any host turns it into a network service: anyone who can reach the port acts with the starter's identity, and the API server sees only that identity. Prefer giving each tool its own credential, or reach a remote dashboard through an authenticated route instead.
Volumes and hostPath: keep Pods away from the node's filesystem
A hostPath volume mounts a file or directory from the node into a container, which can expose kubelet credentials, the container runtime socket and other Pods' data, so the Baseline and Restricted Pod Security Standards disallow it.
| Volume type | What the Pod gets | Allowed by Baseline? |
|---|---|---|
hostPath | A path on the node's filesystem | No |
emptyDir | A fresh scratch directory created for the Pod and removed with it | Yes |
configMap / secret / projected | Keys from API objects projected as files | Yes |
persistentVolumeClaim | Storage bound through a claim; the Pod spec names no node path | Yes (see the next section for the catch) |
When a hostPath is genuinely needed
Some system agents, for example a node monitoring agent that reads one host directory, really do need node files. The documentation advises scoping a hostPath to the specific file or directory required, never a parent of it, and mounting it readOnly: true when the agent only reads, in a namespace with a policy that permits it. The hostPath type field only checks (or creates) what kind of path exists; it adds no protection. Settings that let mounts propagate between container and host need a privileged container and widen exposure further. An agent that only reads data never needs write access to it.
Persistent storage: PV creation rights, reclaim policies, access modes and encryption
Securing persistent storage means keeping PersistentVolume creation with administrators, choosing reclaim policies with data left behind in mind, using the narrowest access mode, and getting encryption at rest from the storage backend through the StorageClass.
Who may create PersistentVolumes
PersistentVolumes (PVs) are cluster-scoped; PersistentVolumeClaims (PVCs) are namespaced requests for storage. The Kubernetes RBAC good practices warn that permission to create arbitrary PVs includes creating hostPath-backed PVs, which give any Pod that mounts them access to the node's filesystem. This also defeats Pod Security admission: it checks the Pod's own spec.volumes, sees an allowed persistentVolumeClaim volume and does not look through the claim to the PV behind it. So a namespace label enforcing a Pod Security level does not, on its own, keep node files away from anyone who can also create PVs. Users should request storage through PVCs and administrator-controlled StorageClasses.
Reclaim policies and data remanence
| Policy | When the claim is deleted | Security consequence |
|---|---|---|
Retain | The PV moves to Released and keeps its data; it still carries the old claim reference and is not rebound automatically | Protects against accidental deletion, but sensitive data persists and must be wiped deliberately before the storage is reused |
Delete | The PV and its backing storage asset are deleted | Data is gone; this is the default for dynamically provisioned volumes when a StorageClass sets no reclaimPolicy |
Recycle (deprecated) | A basic scrub, then the volume is made available again | Deprecated in favour of dynamic provisioning |
Dynamically provisioned PVs inherit their StorageClass's reclaimPolicy. The reclaim policy is the only StorageClass setting that decides what happens to the data when a claim is deleted; the others govern binding and resizing.
Access modes
| Mode | Who can mount it |
|---|---|
ReadWriteOnce | Read-write by a single node; several Pods on that node can still use it |
ReadWriteOncePod | Read-write by a single Pod in the whole cluster (stable since Kubernetes 1.29; CSI volumes only) |
ReadOnlyMany | Read-only by many nodes |
ReadWriteMany | Read-write by many nodes |
The "once" in ReadWriteOnce is per node, which is weaker than it sounds: two Pods scheduled to the same node can both write. Access modes are the field that limits how many consumers can mount a volume; security context settings govern file ownership and permissions inside the container instead.
Encryption at rest for volume data
The API server's EncryptionConfiguration encrypts chosen API resources before they are written to etcd. Data a Pod writes to a volume goes from the node to the storage system and never passes through the API server, so it is not covered, however the configuration is set. Volume encryption comes from the storage backend or CSI driver, usually configured with StorageClass parameters (for example an encryption flag and a customer-managed key ID). To make encryption automatic, make an encrypting StorageClass the default: claims that name no class get it, so the guarantee is set once by administrators rather than in each manifest. Remember that EncryptionConfiguration only ever encrypts API objects, so naming storage-related resources there protects object metadata in etcd, not the bytes on disk. Applications may add their own encryption too, but it is not the only route.
Tip. KCSA questions on this topic are mostly short scenarios about default behaviour and which control fits. Networking items test the flat Pod network (whether a cross-namespace or cross-node connection succeeds, what ClusterIP and selectors really restrict), which component assigns Pod IPs, why the CNI directories are sensitive, and how to encrypt Pod traffic under constraints such as no code changes or no sidecars, including the gap between node encryption and workload identity. Expect a question on why and how to block Pods from the cloud metadata API, with near-miss controls in the wrong direction or layer. Client items cover what a kubeconfig holds, insecure-skip-tls-verify and the unknown-authority error, reviewing an untrusted kubeconfig, limiting damage from a stolen file, what remains after a client certificate leaks or an OIDC account is disabled, admin.conf versus super-admin.conf, and exposed kubectl proxy listeners. Storage items cover hostPath and Pod Security Standards, scoping a necessary hostPath, PV creation rights, reclaim policies, access modes and where volume encryption is configured. One item may ask you to select two answers.
- The Pod network is flat: any Pod can reach any Pod across nodes and namespaces until a NetworkPolicy, enforced by the CNI plugin, restricts it.
- ClusterIP keeps a Service off external networks but is reachable from every Pod; the selector chooses backends, not allowed clients.
- The container runtime runs the CNI plugin from /opt/cni/bin with config from /etc/cni/net.d as root, so write access to those paths is root on the node.
- Kubernetes does not encrypt Pod-to-Pod traffic; use CNI node encryption, mesh mutual TLS or application TLS. Node encryption proves the sending node, not the workload.
- Block Pods from 169.254.169.254 with an egress NetworkPolicy ipBlock except entry; the metadata API can hand out the node's cloud credentials.
- A kubeconfig is a portable identity; never use insecure-skip-tls-verify, supply certificate-authority-data, and inspect untrusted files for exec plugins.
- Client certificates cannot be revoked and there are no User objects; remove the bindings for the certificate's user name and groups, and prefer short-lived OIDC tokens with individual least-privilege identities.
- super-admin.conf is in system:masters and bypasses RBAC, so keep it offline; admin.conf's access comes from a removable ClusterRoleBinding.
- Baseline blocks hostPath in a Pod, but the right to create PersistentVolumes can reopen it through a hostPath PV and a claim.
- Retain leaves data on Released volumes; ReadWriteOncePod limits a volume to one Pod; volume encryption comes from the StorageClass and CSI driver, not EncryptionConfiguration.
Check yourself: 3 questions on this topic
Exam-style KCSA questions on Container networking, client security and storage, answered right here with an explanation after each. Want more? KCSA Kubernetes Cluster Component Security practice questions.
Frequently asked questions
Do Kubernetes namespaces isolate network traffic between Pods?
No. A namespace scopes object names, RBAC permissions and quotas, but Pod IPs stay reachable from every Pod in the cluster. Network isolation between namespaces requires NetworkPolicy objects and a CNI plugin that enforces them.
Is traffic between Pods encrypted in Kubernetes by default?
No. Kubernetes uses TLS for control plane connections such as clients to the API server, but application traffic between Pods is sent as the application wrote it. Encryption must be added by the CNI plugin (for example WireGuard or IPsec between nodes), a service mesh with mutual TLS, or TLS in the application.
Why should Pods be blocked from the cloud instance metadata API?
The metadata endpoint at 169.254.169.254 can return temporary credentials for the node's cloud identity, which a compromised Pod could reuse to act in the cloud account. The Kubernetes documentation recommends restricting it, for example with an egress NetworkPolicy that excludes that address, or with the cloud provider's metadata and workload identity settings.
What is the risk of insecure-skip-tls-verify in a kubeconfig?
It tells kubectl not to verify the API server's certificate. The connection is still encrypted, but an attacker in the network path can impersonate the API server and capture the user's credentials. The fix is to add the cluster CA certificate as certificate-authority-data.
Can you revoke a Kubernetes client certificate?
Not individually. The API server accepts any unexpired certificate signed by its client CA, and there is no revocation list or User object to delete. To cut off a leaked certificate, remove the RBAC bindings for its user name and for its groups (the certificate's Organization fields), and move the person to a new user name. If one of its groups is system:masters, which bypasses RBAC, no binding change helps; only replacing the client CA or waiting for expiry ends it.
What is the difference between kubeadm admin.conf and super-admin.conf?
admin.conf's user is in the kubeadm:cluster-admins group, which gets cluster-admin through a ClusterRoleBinding that can be removed. super-admin.conf's user is in system:masters, which bypasses RBAC entirely, so it should be kept offline as a break-glass credential.
Why is permission to create PersistentVolumes risky?
PersistentVolumes are cluster-scoped, and the right to create them includes hostPath-backed volumes that expose the node's filesystem. Pod Security admission checks only the Pod's own volumes, so a Pod mounting a claim bound to a hostPath PV gets past a Baseline namespace. Keep PV creation with administrators and give users PVCs and StorageClasses.
Does API server encryption at rest protect PersistentVolume data?
No. EncryptionConfiguration encrypts API objects such as Secrets before they are stored in etcd. Volume data goes directly from the node to the storage system, so its encryption is configured on the storage backend or CSI driver, typically through StorageClass parameters; an encrypting default StorageClass applies it to every claim that names no class.
Source
This lesson covers the "Kubernetes Cluster Component Security" domain of the official KCSA exam guide. Vendors revise their guides — check the source for the current version.
- KCSA certification page and curriculum — The Linux Foundation
Sign up free to mark lessons complete, bookmark topics and track your exam readiness.