Hardening Kubernetes: The Stuff That Actually Matters

| 7 min read |
kubernetes security hardening devops

Kubernetes defaults are built for getting things running, not for keeping attackers out. A layered hardening walkthrough covering pods, RBAC, network policies, secrets, and the control plane.

Quick take

Kubernetes out of the box is optimized for convenience, not security. Lock down pods, network, RBAC, secrets, and the control plane as separate layers. Assume every layer will be tested. My NATO background taught me defense in depth isn’t a buzzword – it’s the only model that survives contact with reality.

Kubernetes ships with defaults designed to get your workloads running. Not to resist a motivated attacker. Not to survive a leaked credential. Not to contain a compromised container. If you’re running a production cluster with default settings, you’re running a cluster that trusts everything by default. That should scare you.

I’ve hardened clusters across multiple organizations and I keep seeing the same gaps. This post walks through the layers that matter, with config you can actually apply. Some of this comes from my time working with NATO cyber defense systems, where “assume breach” wasn’t a thought experiment – it was the operating model.

Start with a threat model (seriously)

I know. Nobody wants to do this part. But every security control you skip because “we’re not a bank” is a control an attacker won’t encounter. You don’t need a 40-page document. You need answers to three questions:

  1. What data would hurt you if it leaked?
  2. Who has access to your cluster, and how?
  3. If a container gets compromised, what can the attacker reach?

If you can’t answer these quickly, start there before touching any YAML.

Pod security: the non-negotiable baseline

Every container should run with the least privilege possible. Full stop. Here is what that looks like in practice:

apiVersion: v1
kind: Pod
metadata:
  name: app
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 1000
  containers:
    - name: app
      image: myregistry/app:v1.2.3@sha256:abc123...
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop: ["ALL"]
      resources:
        requests:
          cpu: "100m"
          memory: "128Mi"
        limits:
          cpu: "500m"
          memory: "256Mi"

Key points:

  • runAsNonRoot: true – if your app needs root, fix the app. I’ve seen exactly one legitimate case for running as root in a non-system container in the last three years.
  • Drop ALL capabilities – then add back only what’s needed. Most applications need zero Linux capabilities.
  • Read-only root filesystem – forces you to use volumes for writable paths, which is where you want writes anyway.
  • Resource limits – not just for cost. An unbound container can starve the node and affect every other workload on it.
  • Pin image digests – latest isn’t a version. Pin to a SHA256 digest for anything that matters.

PodSecurityPolicy is deprecated as of 1.21 and removed in 1.25. Replace it. Pod Security Admission (beta in 1.23) implements three levels – privileged, baseline, restricted. At minimum, enforce baseline cluster-wide and restricted on sensitive namespaces:

apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

For more granular control, OPA Gatekeeper or Kyverno let you write custom policies. I prefer Kyverno for readability, but both work.

Network policies: default deny, then allow

Kubernetes networking is flat by default. Every pod can talk to every other pod. That’s fantastic for an attacker who compromises one service and wants to pivot laterally.

Start with a default deny policy on every namespace:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: production
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

Then explicitly allow what’s needed:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-app-to-db
  namespace: production
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes:
    - Egress
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: postgres
      ports:
        - protocol: TCP
          port: 5432
    - to:  # DNS
        - namespaceSelector: {}
      ports:
        - protocol: UDP
          port: 53

A critical gotcha: NetworkPolicy only works if your CNI supports it. Flannel doesn’t enforce network policies. Calico, Cilium, and Weave do. Verify enforcement in every environment, including staging. I’ve seen teams spend weeks writing policies that were silently ignored because the CNI was wrong.

Don’t forget egress. Ingress gets all the attention, but an attacker who compromises a pod and can make outbound HTTPS calls to the internet has an exfiltration path. Restrict egress to known destinations.

RBAC: strict and boring

Good RBAC is boring. It should be so restrictive that nobody notices it until they try to do something they shouldn’t.

  • One service account per workload. Not one per namespace. Not the default service account.
  • Scope roles to namespaces and specific resources. Cluster-wide roles are for platform teams, not applications.
  • Disable automatic service account token mounting for pods that don’t need to talk to the API server:
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa
  namespace: production
automountServiceAccountToken: false

For human access, use OIDC with your identity provider. Short-lived tokens from kubectl via OIDC are vastly better than long-lived kubeconfig certificates. Nobody should have cluster-admin in production unless they’re actively performing cluster operations, and even then, consider just-in-time access.

Secrets: they aren’t encrypted

This surprises people. Kubernetes Secrets are base64-encoded in etcd. That’s encoding, not encryption. Anyone with etcd access can read every secret in your cluster.

Fix this in layers:

  1. Encrypt etcd at rest. Configure EncryptionConfiguration on the API server with aescbc or secretbox providers.
  2. Use an external secrets manager. HashiCorp Vault, AWS Secrets Manager, or GCP Secret Manager with an operator like External Secrets. The secret value never lives in etcd.
  3. Scope access. Only the service accounts that need a secret should be able to read it. RBAC applies to secrets too.
  4. Rotate. If you can’t rotate a secret without a deployment, fix your rotation process before you fix the secret.

Never bake secrets into container images. Never put them in ConfigMaps. Never log them. These sound obvious, but I find violations in almost every cluster audit.

Control plane hygiene

The API server is the front door to everything. Lock it down:

  • Disable anonymous authentication (--anonymous-auth=false).
  • Enable audit logging. You need to know who did what and when.
  • Restrict API server access to trusted networks. If your API server is reachable from the public internet, you have a bigger problem than this blog post covers.
  • Keep etcd on a private network, TLS-encrypted, and only reachable by the API server.
  • Treat kubeconfig files like production database credentials. Because that’s what they’re.

If you’re on a managed service (EKS, GKE, AKS), the provider handles some of this. But “managed” doesn’t mean “secure.” Review what they expose and what they lock down. I’ve seen managed clusters with surprisingly permissive defaults.

Image supply chain

Stop using latest. Stop using mutable tags. Every production image should be:

  • Built from a minimal base (distroless, Alpine, or scratch for Go binaries).
  • Scanned for vulnerabilities in CI before it reaches a registry.
  • Signed, ideally with cosign or Notary.
  • Pulled from a private registry with access controls.

If you’re pulling public images directly into production, you’re trusting that Docker Hub (or whoever) hasn’t been compromised. That trust should make you uncomfortable.

Node-level protections

Nodes are part of your trust boundary. A compromised node means compromised pods.

  • Use a minimal OS (Bottlerocket, Flatcar, Talos). Less software means fewer attack vectors.
  • Patch regularly. Automate node rotation if you can.
  • Lock down the kubelet API. Disable anonymous kubelet auth.
  • Apply seccomp profiles. The RuntimeDefault profile is a good starting point and blocks a significant number of syscalls:
securityContext:
  seccompProfile:
    type: RuntimeDefault

Runtime detection tools like Falco are worth considering. They won’t prevent an attack, but they can alert you when something unexpected happens – a shell spawning inside a container, an unusual network connection, a binary that shouldn’t exist being executed.

Monitoring and incident response

Hardening without visibility is just hoping. You need:

  • Centralized audit logs from the API server.
  • Alerts on failed authentication attempts and privilege escalations.
  • Monitoring for unexpected image pulls or outbound connections.
  • A written incident response playbook. Not a novel. A one-page runbook that answers: who gets paged, what gets isolated, how do we preserve evidence.

Write the playbook before you need it. During an actual incident isn’t the time to figure out who has cluster access.

The operating discipline

None of this works as a one-time project. Security is operational:

  • All cluster config changes go through code review. GitOps is your friend here.
  • Stay on supported Kubernetes versions. Running an EOL version means running with known unpatched vulnerabilities.
  • Test your controls in staging. I’ve seen policies that worked perfectly in staging and failed silently in production because the namespace labels were wrong.
  • Run backup and restore drills. Not “we’ve Velero installed.” Actually restore from backup and verify the cluster works.

Kubernetes hardening isn’t a checklist you complete once. It’s a set of defaults you enforce continuously, across pods, network, identities, secrets, and the control plane. When those defaults are solid, incidents become containable instead of catastrophic.