SaveMyCert
Log in
5 of 5 free questions left today·for 10 a day
Kubernetes Cluster Component Security

KCSA Container Networking, Client Security and Storage Security

21 min readKCSA · Kubernetes Cluster Component SecurityUpdated

Container networking, client security and storage security are the parts of Kubernetes cluster component security that protect the Pod network, the credentials clients use to reach the API server, and the volumes where workloads keep data. On the network side, this KCSA lesson explains the flat Pod network, the CNI plugin as a privileged component, encrypting Pod traffic and limiting access to the cloud metadata API. On the client side, it covers kubeconfig files, TLS verification, revocation, short-lived OIDC tokens, kubeadm admin files and kubectl proxy. On the storage side, it covers hostPath volumes, PersistentVolume creation rights, reclaim policies, access modes and volume encryption.

What you’ll learn
  • Explain the flat Kubernetes network model and why namespaces, ClusterIP Services and selectors do not restrict which Pods can connect
  • Describe what the CNI plugin does, why its binaries and configuration are root-equivalent, and how it differs from kube-proxy, the scheduler and CoreDNS
  • Compare CNI node encryption, service mesh mutual TLS and application TLS for encrypting Pod traffic, and explain why node encryption does not prove workload identity
  • Restrict Pod access to the cloud instance metadata API with an egress network policy and explain why RBAC and token settings do not
  • Protect kubeconfig files and client credentials: verify the API server's CA, review untrusted files, prefer short-lived OIDC tokens and revoke access through RBAC
  • Distinguish kubeadm's admin.conf from super-admin.conf and explain the risk of exposing kubectl proxy
  • Secure storage by limiting hostPath and PersistentVolume creation, choosing reclaim policies and access modes, and encrypting volume data through StorageClasses

What container networking, client and storage security cover

This part of Kubernetes cluster component security protects the three things that sit between the control plane and the workloads: the Pod network that carries application traffic, the client credentials people and tools use to reach the API server, and the volumes where Pods keep data. Each has a default that is more open than many teams assume.

AreaWhat it isThe default to knowMain controls
Container networkingThe CNI plugin and the Pod-to-Pod network it buildsEvery Pod can reach every other Pod; traffic between Pods is not encryptedNetworkPolicy enforced by the CNI, CNI node encryption, service mesh mutual TLS, egress limits on the cloud metadata API
Client securitykubeconfig files, kubectl and the credentials they carryA kubeconfig file is a portable identity; issued client certificates cannot be revokedShort-lived OIDC tokens, individual least-privilege identities, CA verification, trusted files only, break-glass admin credentials
StorageVolumes, PersistentVolumes, claims and StorageClasseshostPath exposes the node; deleted claims can leave data behind; API server encryption does not cover volume dataPod Security Standards, PV creation kept with admins, reclaim policies, encrypting StorageClasses, narrow access modes

The control plane components themselves (API server, controller manager, scheduler, etcd) and the node components (kubelet, container runtime, kube-proxy) are covered in their own KCSA lessons. How to write NetworkPolicy rules and how the Pod Security Standards are enforced are covered in the Kubernetes Security Fundamentals domain; this lesson uses them only where networking, client access or storage depends on them.

The flat Kubernetes network model: any Pod can reach any Pod

By default every Pod in a Kubernetes cluster can open a connection to every other Pod, on any node and in any namespace, because the network model gives each Pod its own IP address that the whole cluster can reach without NAT. Nothing blocks that traffic until a network policy does.

The Kubernetes network model requires the network plugin to provide three things: every Pod gets a unique cluster-wide IP, Pods on any node can talk to Pods on every other node directly, and node agents can reach the Pods on their node. This makes service discovery and routing simple, and it also means the network is flat and allow-all out of the box.

Namespaces are not a network boundary

A namespace scopes object names, RBAC permissions, quotas and Pod Security labels. It does not separate traffic. A namespace boundary stops a user from listing or editing another team's objects, but a packet sent to a Pod IP is never checked against namespaces at all. Separating the API objects is therefore not the same as separating the traffic; only a network policy that the CNI plugin enforces does the second.

Services control exposure, not callers

Service settingWhat it actually controlsWhat it does not control
type: ClusterIPThe Service gets a virtual IP reachable only from inside the cluster (every Pod and node)Which namespaces or Pods may call it; every Pod in the cluster can
selectorWhich Pods are the backends that receive the trafficWhich Pods are allowed to be clients
DNS name (<service>.<namespace>.svc.cluster.local, where cluster.local is the default cluster domain)How the Service is found by nameAccess; the namespace in the name is not a permission check
NodePort / LoadBalancerExposure on node addresses or an external load balancerAuthentication of callers

To limit who can call a workload you need network policies on the backend Pods, enforced by a CNI plugin that supports them, plus authentication in the application itself. A NetworkPolicy object has no effect if the installed plugin does not enforce it.

The CNI plugin: a privileged component that wires every Pod

The Container Network Interface (CNI) plugin is the component that gives each new Pod its network interface and IP address and connects it to the node's network; the container runtime calls it every time it creates a Pod sandbox. Kubernetes defines the network model but does not implement it; the plugin the operator installs does.

ComponentNetworking job
CNI pluginCreates the Pod's interface, allocates its IP, sets up routes; many plugins also enforce NetworkPolicy and offer node-to-node encryption
kube-proxyPrograms iptables, IPVS or nftables rules that implement Service virtual IPs; it does not create Pod interfaces or allocate Pod IPs
kube-schedulerChooses a node for the Pod; it reserves no IP address
CoreDNSAnswers name lookups for Services and Pods; it configures no interfaces

Why the CNI files are as sensitive as root

On each node the plugin's executables live in /opt/cni/bin and its configuration in /etc/cni/net.d (the usual default paths). The runtime runs those binaries as root for every Pod it starts. Anyone who can write to those directories can therefore run their own code as root on the node and reroute or capture every Pod's traffic. Only root should be able to modify them, and changes should be detected.

The kubelet's credentials, the runtime socket and static Pod manifests are sensitive too, but they live in other locations; node hardening is covered in the node security lesson.

Choosing a CNI plugin is a security decision: check that it enforces NetworkPolicy (including egress rules), whether it can encrypt traffic between nodes, and that it comes from a trusted, maintained source.

Encryption in transit: Pod traffic is plaintext unless you add it

Kubernetes does not encrypt application traffic between Pods; it travels across the Pod network exactly as the application sent it unless the CNI plugin, a service mesh or the application itself adds encryption. The TLS that Kubernetes configures protects control plane connections: clients to the API server, the API server to etcd, and the API server to kubelets (encrypted, but the kubelet's serving certificate is verified only when the API server is given the kubelet CA with --kubelet-certificate-authority). None of those connections carries the traffic Pods send to each other.

OptionHow it worksChanges Pods or code?Proves which workload is calling?
CNI node-to-node encryption (for example WireGuard or IPsec)The plugin encrypts Pod traffic as it leaves one node and decrypts it on the otherNo; transparent to PodsNo; the keys belong to nodes, so the receiver learns only which node sent the packet
Service mesh mutual TLSProxies (traditionally sidecar containers in each Pod) set up mTLS using certificates issued per workload, often tied to the service accountSidecar meshes add containers to Pods; no application code changeYes; each side presents a workload certificate that policy can check
Application TLSEach service serves and calls over HTTPS/TLS itselfYes; every application must changeOnly if the application also checks client certificates
API server EncryptionConfigurationEncrypts chosen API resources (such as Secrets) at rest in etcd-Not a network control at all

Encryption is not workload identity

Two different requirements are often confused. Encrypting traffic between nodes is a confidentiality requirement, and CNI encryption meets it. Letting a service accept calls only from one named workload, proved cryptographically, is an identity requirement: it needs per-workload credentials, such as the certificates a mesh issues for mutual TLS, plus authorization rules that check them. Node encryption cannot meet it, because its keys identify nodes, and many unrelated Pods share a node. When a scenario adds constraints, compare them with the "Changes Pods or code?" column above. Some meshes also offer modes without sidecars, using per-node proxies.

Protecting the cloud instance metadata API from Pods

On cloud-hosted clusters a Pod can usually reach the provider's instance metadata API at the link-local address 169.254.169.254, and that endpoint can hand out temporary credentials for the node's cloud identity, so the Kubernetes documentation recommends restricting Pod access to it. A compromised Pod that reads those credentials can act in the cloud account with the node's permissions.

Pod traffic leaves through the node, so by default it reaches the same metadata endpoint the node uses. The endpoint is a cloud service, not a Kubernetes one: it answers HTTP requests about the instance and returns the instance role's credentials.

Which controls fit

Because the threat is a network connection the Pod opens to a cloud address, the fitting controls act on that connection or on what the endpoint will return:

  • Egress NetworkPolicy. A policy that selects the workload Pods and allows egress to 0.0.0.0/0 with an except entry of 169.254.169.254/32 stops them reaching the endpoint, provided the CNI plugin enforces policy. Direction matters: egress rules filter connections a selected Pod opens, ingress rules filter connections that arrive at it.
  • Cloud provider settings. Requiring session tokens, limiting how many network hops a metadata response may travel, and using the provider's workload identity feature so Pods never need the node's credentials.

Kubernetes API controls do not reach this path. RBAC decides which Kubernetes API requests a subject may make, and service account token settings govern the Kubernetes credential mounted in a Pod; neither filters packets to a cloud address.

Practical notes on ipBlock

ipBlock is intended for addresses outside the cluster. How a plugin treats Pod IPs that fall inside a broad CIDR such as 0.0.0.0/0 varies, so allow in-cluster destinations with Pod or namespace selectors where you need them. Once a Pod is selected by any egress policy, everything not allowed is denied, so make sure the policies also allow what the workload really needs, such as DNS to the cluster DNS Pods.

System components that genuinely need the metadata API (for example a cloud controller or node agent) can be allowed by a separate policy that selects only them.

kubeconfig files: a portable identity that must be trusted and verified

A kubeconfig file tells kubectl and other clients which clusters exist, which credentials to use and which pairing of the two (a context) to use by default, so whoever holds the file holds those credentials and must be treated like a password.

SectionContainsSecurity relevance
clustersserver URL, certificate-authority or certificate-authority-data (the CA's public certificate), optionally insecure-skip-tls-verifyDecides whether the client verifies the API server; holds no private keys
usersA client certificate and private key, a bearer token, or an exec credential pluginThe actual secret: embedded keys and tokens are a stolen identity; exec runs a local program
contextsA cluster + user pair and a default namespaceA convenience only; the default namespace restricts nothing

The secret in a client kubeconfig is the users entry: whoever has a copy can authenticate as that user. Cluster-side secrets and permissions stay on the cluster, and every request is authorized by the API server, never by kubectl. Keep the file readable by its owner only and avoid copying it between machines.

Verify the API server: never skip TLS verification

TLS gives encryption and server authentication. insecure-skip-tls-verify: true keeps the encryption but drops the authentication, so kubectl will send its credentials to anyone in the network path who answers as the API server. When kubectl reports x509: certificate signed by unknown authority, the correct fix is to add the cluster's CA certificate as certificate-authority-data (or a certificate-authority file path). The error says the client cannot verify the server, so the fix belongs on that side of the trust relationship: give the client the CA, rather than removing the check. (Current API server releases no longer offer an insecure HTTP port.)

Use kubeconfig files only from trusted sources

The Kubernetes documentation warns to inspect an unfamiliar kubeconfig carefully before using it. A crafted file can make kubectl run local programs through an exec credential plugin under users, and can point certificate and key fields at local files. The risk lands on the machine that runs kubectl, with the privileges of the person running it. Review first the entries that execute something or read local paths; descriptive fields are lower risk, though the server URL still decides where your credentials go.

Client credentials: short-lived tokens, individual identities and revocation

The safest client credentials are short-lived and tied to one person with least-privilege RBAC, because Kubernetes cannot revoke an individual client certificate and a stolen long-lived credential keeps working until it expires.

CredentialHow the API server validates itHow access is cut off
X.509 client certificateAccepts any unexpired certificate signed by its client CA; user name from the subject's Common Name, groups from OrganizationNo revocation list: the old certificate stays valid until it expires. Remove the RBAC bindings for its user name and for its groups, and give the person a new user name
OIDC ID token (usually fetched by an exec plugin)Checks the token's signature against the provider's published signing keys and its expiry, locally; it does not ask the identity provider about each requestDisabling the identity provider account stops new tokens; a token already cached stays valid until it expires. Removing the user's RBAC bindings closes that window at once
Static bearer tokenCompared with a token list configured on the API serverRequires changing API server configuration; avoid for people
Service account tokenSigned by the cluster; current tokens are short-lived and bound to a PodDelete the service account or its bindings; for workloads, not people

There are no User objects

Kubernetes has no User resource to delete: a "user" is just the name and groups an authenticator reports. A leaked certificate keeps authenticating until it expires, so cutting it off means leaving its identity with no permissions. Removing the user-name bindings is enough only if the certificate's groups (its Organization fields) have no bindings of their own and none of them is system:masters, which bypasses RBAC. Because the old certificate still carries the old name, any binding later created for that name would empower it again, which is why the person moves to a new user name.

Limiting damage from a stolen kubeconfig

  • Short-lived credentials: an exec plugin that obtains tokens from an OIDC identity provider means a stolen file holds no long-lived secret, and the provider can refuse future tokens.
  • Individual, least-privilege identities: one identity per person with narrow RBAC limits the blast radius and allows revoking one person without affecting others. Shared admin credentials do the opposite.
  • Know what does not help: controls that protect the connection or change client defaults do nothing against an attacker who connects normally with a valid stolen credential.

Admin credentials and kubectl proxy

Full-control credentials should be revocable and handed out rarely, and local helpers such as kubectl proxy must stay bound to localhost, because both turn one credential into broad access for whoever holds or reaches it.

kubeadm: admin.conf versus super-admin.conf

FileIdentity's groupHow it gets full controlCan access be withdrawn?
/etc/kubernetes/admin.confkubeadm:cluster-adminsA ClusterRoleBinding to the cluster-admin ClusterRoleYes; remove the binding
/etc/kubernetes/super-admin.conf (current kubeadm releases)system:mastersThe API server authorizes this group before RBAC is consultedNo binding change can stop it; it lasts until the certificate expires

The difference that matters is how access can be withdrawn, not how the files connect. Keep super-admin.conf offline as a break-glass credential, hand out admin.conf only when necessary, and give day-to-day administrators their own identities bound through RBAC. The RBAC good practices likewise advise against putting users in system:masters.

kubectl proxy

kubectl proxy runs a local HTTP listener that forwards requests to the API server using the credentials of the person who started it; the listener itself asks callers for nothing. By default it binds to 127.0.0.1 and accepts only localhost host headers, which keeps it personal. Widening --address to all interfaces and --accept-hosts to any host turns it into a network service: anyone who can reach the port acts with the starter's identity, and the API server sees only that identity. Prefer giving each tool its own credential, or reach a remote dashboard through an authenticated route instead.

Volumes and hostPath: keep Pods away from the node's filesystem

A hostPath volume mounts a file or directory from the node into a container, which can expose kubelet credentials, the container runtime socket and other Pods' data, so the Baseline and Restricted Pod Security Standards disallow it.

Volume typeWhat the Pod getsAllowed by Baseline?
hostPathA path on the node's filesystemNo
emptyDirA fresh scratch directory created for the Pod and removed with itYes
configMap / secret / projectedKeys from API objects projected as filesYes
persistentVolumeClaimStorage bound through a claim; the Pod spec names no node pathYes (see the next section for the catch)

When a hostPath is genuinely needed

Some system agents, for example a node monitoring agent that reads one host directory, really do need node files. The documentation advises scoping a hostPath to the specific file or directory required, never a parent of it, and mounting it readOnly: true when the agent only reads, in a namespace with a policy that permits it. The hostPath type field only checks (or creates) what kind of path exists; it adds no protection. Settings that let mounts propagate between container and host need a privileged container and widen exposure further. An agent that only reads data never needs write access to it.

Persistent storage: PV creation rights, reclaim policies, access modes and encryption

Securing persistent storage means keeping PersistentVolume creation with administrators, choosing reclaim policies with data left behind in mind, using the narrowest access mode, and getting encryption at rest from the storage backend through the StorageClass.

Who may create PersistentVolumes

PersistentVolumes (PVs) are cluster-scoped; PersistentVolumeClaims (PVCs) are namespaced requests for storage. The Kubernetes RBAC good practices warn that permission to create arbitrary PVs includes creating hostPath-backed PVs, which give any Pod that mounts them access to the node's filesystem. This also defeats Pod Security admission: it checks the Pod's own spec.volumes, sees an allowed persistentVolumeClaim volume and does not look through the claim to the PV behind it. So a namespace label enforcing a Pod Security level does not, on its own, keep node files away from anyone who can also create PVs. Users should request storage through PVCs and administrator-controlled StorageClasses.

Reclaim policies and data remanence

PolicyWhen the claim is deletedSecurity consequence
RetainThe PV moves to Released and keeps its data; it still carries the old claim reference and is not rebound automaticallyProtects against accidental deletion, but sensitive data persists and must be wiped deliberately before the storage is reused
DeleteThe PV and its backing storage asset are deletedData is gone; this is the default for dynamically provisioned volumes when a StorageClass sets no reclaimPolicy
Recycle (deprecated)A basic scrub, then the volume is made available againDeprecated in favour of dynamic provisioning

Dynamically provisioned PVs inherit their StorageClass's reclaimPolicy. The reclaim policy is the only StorageClass setting that decides what happens to the data when a claim is deleted; the others govern binding and resizing.

Access modes

ModeWho can mount it
ReadWriteOnceRead-write by a single node; several Pods on that node can still use it
ReadWriteOncePodRead-write by a single Pod in the whole cluster (stable since Kubernetes 1.29; CSI volumes only)
ReadOnlyManyRead-only by many nodes
ReadWriteManyRead-write by many nodes

The "once" in ReadWriteOnce is per node, which is weaker than it sounds: two Pods scheduled to the same node can both write. Access modes are the field that limits how many consumers can mount a volume; security context settings govern file ownership and permissions inside the container instead.

Encryption at rest for volume data

The API server's EncryptionConfiguration encrypts chosen API resources before they are written to etcd. Data a Pod writes to a volume goes from the node to the storage system and never passes through the API server, so it is not covered, however the configuration is set. Volume encryption comes from the storage backend or CSI driver, usually configured with StorageClass parameters (for example an encryption flag and a customer-managed key ID). To make encryption automatic, make an encrypting StorageClass the default: claims that name no class get it, so the guarantee is set once by administrators rather than in each manifest. Remember that EncryptionConfiguration only ever encrypts API objects, so naming storage-related resources there protects object metadata in etcd, not the bytes on disk. Applications may add their own encryption too, but it is not the only route.

Tip. KCSA questions on this topic are mostly short scenarios about default behaviour and which control fits. Networking items test the flat Pod network (whether a cross-namespace or cross-node connection succeeds, what ClusterIP and selectors really restrict), which component assigns Pod IPs, why the CNI directories are sensitive, and how to encrypt Pod traffic under constraints such as no code changes or no sidecars, including the gap between node encryption and workload identity. Expect a question on why and how to block Pods from the cloud metadata API, with near-miss controls in the wrong direction or layer. Client items cover what a kubeconfig holds, insecure-skip-tls-verify and the unknown-authority error, reviewing an untrusted kubeconfig, limiting damage from a stolen file, what remains after a client certificate leaks or an OIDC account is disabled, admin.conf versus super-admin.conf, and exposed kubectl proxy listeners. Storage items cover hostPath and Pod Security Standards, scoping a necessary hostPath, PV creation rights, reclaim policies, access modes and where volume encryption is configured. One item may ask you to select two answers.

Key takeaways
  • The Pod network is flat: any Pod can reach any Pod across nodes and namespaces until a NetworkPolicy, enforced by the CNI plugin, restricts it.
  • ClusterIP keeps a Service off external networks but is reachable from every Pod; the selector chooses backends, not allowed clients.
  • The container runtime runs the CNI plugin from /opt/cni/bin with config from /etc/cni/net.d as root, so write access to those paths is root on the node.
  • Kubernetes does not encrypt Pod-to-Pod traffic; use CNI node encryption, mesh mutual TLS or application TLS. Node encryption proves the sending node, not the workload.
  • Block Pods from 169.254.169.254 with an egress NetworkPolicy ipBlock except entry; the metadata API can hand out the node's cloud credentials.
  • A kubeconfig is a portable identity; never use insecure-skip-tls-verify, supply certificate-authority-data, and inspect untrusted files for exec plugins.
  • Client certificates cannot be revoked and there are no User objects; remove the bindings for the certificate's user name and groups, and prefer short-lived OIDC tokens with individual least-privilege identities.
  • super-admin.conf is in system:masters and bypasses RBAC, so keep it offline; admin.conf's access comes from a removable ClusterRoleBinding.
  • Baseline blocks hostPath in a Pod, but the right to create PersistentVolumes can reopen it through a hostPath PV and a claim.
  • Retain leaves data on Released volumes; ReadWriteOncePod limits a volume to one Pod; volume encryption comes from the StorageClass and CSI driver, not EncryptionConfiguration.

Check yourself: 3 questions on this topic

Exam-style KCSA questions on Container networking, client security and storage, answered right here with an explanation after each. Want more? KCSA Kubernetes Cluster Component Security practice questions.

Frequently asked questions

Do Kubernetes namespaces isolate network traffic between Pods?

No. A namespace scopes object names, RBAC permissions and quotas, but Pod IPs stay reachable from every Pod in the cluster. Network isolation between namespaces requires NetworkPolicy objects and a CNI plugin that enforces them.

Is traffic between Pods encrypted in Kubernetes by default?

No. Kubernetes uses TLS for control plane connections such as clients to the API server, but application traffic between Pods is sent as the application wrote it. Encryption must be added by the CNI plugin (for example WireGuard or IPsec between nodes), a service mesh with mutual TLS, or TLS in the application.

Why should Pods be blocked from the cloud instance metadata API?

The metadata endpoint at 169.254.169.254 can return temporary credentials for the node's cloud identity, which a compromised Pod could reuse to act in the cloud account. The Kubernetes documentation recommends restricting it, for example with an egress NetworkPolicy that excludes that address, or with the cloud provider's metadata and workload identity settings.

What is the risk of insecure-skip-tls-verify in a kubeconfig?

It tells kubectl not to verify the API server's certificate. The connection is still encrypted, but an attacker in the network path can impersonate the API server and capture the user's credentials. The fix is to add the cluster CA certificate as certificate-authority-data.

Can you revoke a Kubernetes client certificate?

Not individually. The API server accepts any unexpired certificate signed by its client CA, and there is no revocation list or User object to delete. To cut off a leaked certificate, remove the RBAC bindings for its user name and for its groups (the certificate's Organization fields), and move the person to a new user name. If one of its groups is system:masters, which bypasses RBAC, no binding change helps; only replacing the client CA or waiting for expiry ends it.

What is the difference between kubeadm admin.conf and super-admin.conf?

admin.conf's user is in the kubeadm:cluster-admins group, which gets cluster-admin through a ClusterRoleBinding that can be removed. super-admin.conf's user is in system:masters, which bypasses RBAC entirely, so it should be kept offline as a break-glass credential.

Why is permission to create PersistentVolumes risky?

PersistentVolumes are cluster-scoped, and the right to create them includes hostPath-backed volumes that expose the node's filesystem. Pod Security admission checks only the Pod's own volumes, so a Pod mounting a claim bound to a hostPath PV gets past a Baseline namespace. Keep PV creation with administrators and give users PVCs and StorageClasses.

Does API server encryption at rest protect PersistentVolume data?

No. EncryptionConfiguration encrypts API objects such as Secrets before they are stored in etcd. Volume data goes directly from the node to the storage system, so its encryption is configured on the storage backend or CSI driver, typically through StorageClass parameters; an encrypting default StorageClass applies it to every claim that names no class.

Source

This lesson covers the "Kubernetes Cluster Component Security" domain of the official KCSA exam guide. Vendors revise their guides — check the source for the current version.

Sign up free to mark lessons complete, bookmark topics and track your exam readiness.

Spotted a mistake in this lesson?