10 Kubernetes Multi-Tenant Security Practices
10 Kubernetes multi-tenant security practices: namespaces, RBAC, quotas, default-deny NetworkPolicies, Pod Security Admission, sandboxing, vCluster.
Kubernetes multi-tenant security means letting several teams or customers share one cluster without one tenant reading, slowing or breaking another's workloads. You build it in layers: a namespace per tenant, least-privilege RBAC, quotas, default-deny network policies, Pod Security Admission, admission policies, node isolation, sandboxed runtimes, and a virtual control plane when tenants cannot trust each other.
This guide treats multi-tenancy as a set of ten controls, each mapped to the threat it stops and the YAML or tool that implements it. For general cluster hardening that applies even with a single tenant, read our 12 Kubernetes security best practices first. This page focuses on what changes when the cluster is shared.
Key takeaways
- Kubernetes describes isolation as a spectrum from "soft" (trusted internal teams) to "hard" (tenants who do not trust each other). Decide where you are before picking controls.
- Namespaces are the unit of tenancy. RBAC Roles, ResourceQuotas, LimitRanges and NetworkPolicies are all namespace scoped.
- Without a default-deny NetworkPolicy, any Pod can reach any other Pod in the cluster, whatever namespace it is in.
- Pod Security Admission with the
restrictedlevel stops tenants from running privileged containers that could escape to the node. - For hostile tenants, namespace isolation is not enough. Add dedicated nodes, sandboxed runtimes such as gVisor or Kata Containers, or a virtual control plane per tenant.
- API Priority and Fairness and quotas protect against the "noisy neighbour" who floods the API server or eats node capacity.
Soft and hard multi-tenancy
The Kubernetes multi-tenancy documentation describes two common use cases. In the first, several teams in one organisation share a cluster and often have direct kubectl or GitOps access. There is some trust, but you still need guard rails. In the second, a vendor runs a copy of a workload per customer. Customers have no cluster access, and isolation must assume they are hostile to each other.
| Aspect | Soft multi-tenancy (teams) | Hard multi-tenancy (customers) |
|---|---|---|
| Trust between tenants | Some trust, same company | None assumed |
| Tenant cluster access | Namespace-scoped kubectl or GitOps | Usually none |
| Typical model | Namespace per team | Namespace per customer, or virtual control plane, or dedicated nodes |
| Main risks | Accidental changes, resource starvation | Data access across tenants, container escape |
| Controls needed | Practices 1 to 6 | All ten |
The ten practices at a glance
| # | Practice | Threat it reduces | Layer |
|---|---|---|---|
| 1 | Namespace per tenant | Name clashes, unclear ownership | Control plane |
| 2 | Least-privilege RBAC | Tenants changing each other's objects | Control plane |
| 3 | ResourceQuota and LimitRange | Noisy neighbours, capacity exhaustion | Control plane |
| 4 | Default-deny NetworkPolicies | Lateral movement between tenants | Data plane |
| 5 | Pod Security Admission | Privileged containers and node escape | Data plane |
| 6 | Admission policy as code | Rules tenants could bypass | Control plane |
| 7 | Node isolation | Shared kernel between hostile tenants | Data plane |
| 8 | Sandboxed container runtimes | Kernel exploits from a container | Data plane |
| 9 | Storage and secret isolation | Reading another tenant's data | Data plane |
| 10 | Virtual control planes, API fairness and audit | API flooding, cluster-scoped conflicts, undetected abuse | Control plane |
Control plane isolation
1. Give each tenant its own namespace
Namespaces let object names overlap and are the scope for most security policies. The Kubernetes docs recommend isolating each workload in its own namespace, even when one tenant runs several workloads. Label every tenant namespace so policies can select them.
apiVersion: v1
kind: Namespace
metadata:
name: team-payments
labels:
tenant: payments
pod-security.kubernetes.io/enforce: restricted
Namespaces do not isolate cluster-scoped objects such as CustomResourceDefinitions, ClusterRoles, StorageClasses or nodes. Tenants should not be able to create those. Our namespace organisation tips cover naming and labelling conventions in more detail.
2. Bind tenants to namespace-scoped RBAC
Give each tenant a Role and RoleBinding in their own namespace and nothing cluster-wide. Reuse the built-in edit or view ClusterRoles through a RoleBinding, which grants those permissions only inside the namespace.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: payments-devs-edit
namespace: team-payments
subjects:
- kind: Group
name: payments-devs
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: edit
apiGroup: rbac.authorization.k8s.io
Watch for permissions that cross tenant lines indirectly: creating Pods with any service account, reading Secrets, or escalate, bind and impersonate verbs. Check access with kubectl auth can-i --list --as=jane -n team-billing. The 10 Kubernetes RBAC rules post explains these verbs.
3. Set ResourceQuota and LimitRange per namespace
A ResourceQuota caps what a namespace can consume in total. A LimitRange sets default requests and limits for containers that do not declare them, so the quota can be enforced.
apiVersion: v1
kind: ResourceQuota
metadata:
name: tenant-quota
namespace: team-payments
spec:
hard:
requests.cpu: "20"
requests.memory: 40Gi
limits.memory: 60Gi
pods: "100"
services.loadbalancers: "2"
persistentvolumeclaims: "20"
---
apiVersion: v1
kind: LimitRange
metadata:
name: tenant-defaults
namespace: team-payments
spec:
limits:
- type: Container
defaultRequest:
cpu: 100m
memory: 128Mi
default:
memory: 512Mi
Object count quotas (Pods, Services, ConfigMaps) also protect the API server and etcd from a tenant that creates thousands of small objects.
Data plane isolation
4. Start every tenant namespace with default-deny NetworkPolicies
By default Kubernetes allows all Pod-to-Pod traffic across namespaces. Apply a default-deny policy in each tenant namespace, then allow traffic within the namespace and to DNS explicitly.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: team-payments
spec:
podSelector: {}
policyTypes: ["Ingress", "Egress"]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-same-namespace-and-dns
namespace: team-payments
spec:
podSelector: {}
policyTypes: ["Ingress", "Egress"]
ingress:
- from:
- podSelector: {}
egress:
- to:
- podSelector: {}
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
- protocol: TCP
port: 53
NetworkPolicies only work if your CNI plugin enforces them (Calico, Cilium and others do). Because tenants can usually edit NetworkPolicies in their own namespace, cluster admins often add cluster-wide rules with CNI-specific policies so tenants cannot open their namespace to everyone. See 10 Kubernetes network policies for more patterns.
5. Enforce Pod Security Admission
Pod Security Admission is built into the API server and has been stable since Kubernetes v1.25. It applies the Pod Security Standards (privileged, baseline, restricted) per namespace through labels. Tenant namespaces should enforce restricted, which blocks privileged containers, host namespaces, hostPath volumes and running as root.
kubectl label namespace team-payments \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/warn=restricted \
pod-security.kubernetes.io/audit=restricted --overwrite
Tenants must not be allowed to change these labels, so do not grant update or patch on Namespace objects.
6. Add admission policies for rules Kubernetes does not cover
Some tenant rules need custom policy: only pull images from the tenant's registry path, require a tenant label, block LoadBalancer Services, or stop tenants from editing the labels above. Options include the built-in ValidatingAdmissionPolicy (stable since v1.30, written in CEL), Kyverno and OPA Gatekeeper.
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: tenant-images-only
spec:
failurePolicy: Fail
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["pods"]
validations:
- expression: "object.spec.containers.all(c, c.image.startsWith('registry.example.com/' + namespaceObject.metadata.labels['tenant'] + '/'))"
message: "Images must come from the tenant's registry path."
A ValidatingAdmissionPolicyBinding then applies the policy to namespaces with a tenant label.
7. Isolate nodes for sensitive or untrusted tenants
Containers on the same node share a kernel. For tenants with stricter needs, give them dedicated node pools using taints, tolerations and node affinity. The taint keeps other tenants off; the affinity keeps this tenant on.
# On the node: kubectl taint nodes node-7 tenant=payments:NoSchedule
spec:
tolerations:
- key: tenant
operator: Equal
value: payments
effect: NoSchedule
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: tenant
operator: In
values: ["payments"]
Tenants could add the toleration themselves, so enforce node placement with an admission policy from practice 6.
8. Run untrusted workloads in a sandboxed runtime
When dedicated nodes cost too much, a sandboxed runtime adds a boundary between the container and the host kernel. gVisor intercepts system calls in a user-space kernel, and Kata Containers runs each Pod in a lightweight VM. You expose them through a RuntimeClass.
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
---
apiVersion: v1
kind: Pod
metadata:
name: tenant-job
namespace: team-payments
spec:
runtimeClassName: gvisor
containers:
- name: job
image: registry.example.com/payments/job:2.1
Sandboxes add some overhead and not every workload is compatible, so test before making them mandatory.
9. Separate storage and secrets
PersistentVolumeClaims are namespaced, but PersistentVolumes and StorageClasses are cluster scoped. Use dynamic provisioning so each claim gets a fresh volume, set the reclaim policy to Delete for tenant data that must not be reused, and never let tenants create hostPath volumes (Pod Security Admission restricted already blocks them). Keep each tenant's Secrets in their own namespace, enable encryption at rest, and consider an external secret manager per tenant. Our guide to tools to manage Kubernetes Secrets compares the options.
Shared control plane protection
10. Use virtual control planes, API fairness and audit logs
Some conflicts cannot be solved with namespaces. Two tenants may need different versions of the same CRD, or a tenant may need cluster-admin rights for an operator. The Kubernetes docs describe a "virtual control plane per tenant" model for this. Each tenant gets its own API server running on shared nodes. Open-source implementations include vCluster and Kamaji, while Capsule groups several namespaces into one tenant on a shared control plane.
On a shared API server, API Priority and Fairness (stable since v1.29) queues requests by priority level and flow, so one tenant's controller cannot starve everyone else. Pair it with audit logging, filtered by namespace and user, so you can answer "who did what in which tenant" after an incident.
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
resources:
- group: ""
resources: ["secrets", "configmaps"]
- level: RequestResponse
verbs: ["create", "update", "patch", "delete"]
resources:
- group: "rbac.authorization.k8s.io"
Choosing a tenancy model
| Model | Isolation | Cost and effort | Good fit |
|---|---|---|---|
| Namespace per tenant | Soft to medium | Lowest | Internal teams |
| Namespace plus dedicated nodes | Medium to hard | Medium | Regulated teams, paying customers |
| Virtual control plane per tenant | Hard for the API layer | Medium | Tenants needing CRDs or admin rights |
| Cluster per tenant | Hardest | Highest | Strict compliance, very large tenants |
Many platforms mix models: shared namespaces for internal teams and dedicated clusters for the few tenants with compliance needs. Managing several clusters brings its own tooling, covered in 12 multi-cluster management solutions.
Monitoring tenant isolation
Controls drift over time: someone adds a broad ClusterRoleBinding, a namespace loses its labels, or a new CNI setting turns off policy enforcement. Check isolation continuously, not only at setup.
- Alert on new ClusterRoleBindings and on RoleBindings that grant
cluster-admin, using audit log events. - Run a scheduled check that every tenant namespace has a default-deny NetworkPolicy, a ResourceQuota and the expected Pod Security labels.
- Track quota usage per namespace with kube-state-metrics, which exposes
kube_resourcequotametrics, so tenants hear about limits before they hit them. - Use a runtime detection tool such as Falco to flag unexpected behaviour, for example a shell spawned in a tenant container.
- Run policy engines in audit mode first when adding new rules, then switch to enforce once existing workloads comply.
Common multi-tenancy mistakes
| Mistake | Why it matters | Fix |
|---|---|---|
Granting tenants admin on their namespace | admin can create RoleBindings, so tenants can grant access to others | Prefer edit or a custom Role |
| Letting tenants edit Namespace labels | They can switch off Pod Security enforcement | No update or patch on Namespaces |
| Allowing NodePort or LoadBalancer Services freely | Tenants expose workloads outside the cluster without review | Quota them to zero or block them by policy |
| Shared service accounts across namespaces | One compromised workload gains another's access | One service account per workload, tokens not mounted unless needed |
| Ignoring DNS | Tenants can discover other tenants' Services by name | Accept it for soft tenancy, or use per-tenant control planes for hard tenancy |
The broader picture of who should own cluster maintenance in a shared platform is discussed in who should own Kubernetes cluster maintenance.
Onboarding checklist for a new tenant
- Create the namespace with tenant and Pod Security labels.
- Bind the tenant group to
editor a custom Role in that namespace only. - Apply ResourceQuota and LimitRange.
- Apply default-deny and allow-DNS NetworkPolicies.
- Confirm admission policies select the namespace.
- Set node placement or RuntimeClass if the tenant needs stronger isolation.
- Run
kubectl auth can-i --listas a tenant user and test cross-namespace traffic to confirm it is blocked.
Templating these steps with Helm, Kustomize or a GitOps tool keeps every tenant consistent.
Learn Kubernetes security
RBAC, NetworkPolicies, Pod Security Admission, RuntimeClass and audit logging are all part of the CKS training, available in our Kharadi, Pune classroom and live online.
Frequently asked questions
What is multi-tenancy in Kubernetes?
It is sharing one cluster between several teams or customers, called tenants, while isolating their workloads, data and resource use from each other.
Are namespaces enough for multi-tenant isolation?
For trusted internal teams, namespaces plus RBAC, quotas, network policies and Pod Security Admission are usually enough. For tenants who do not trust each other, add node isolation, sandboxed runtimes or a virtual control plane.
How do I stop Pods in one namespace from reaching another?
Apply a default-deny NetworkPolicy in each namespace and then allow only the traffic you need. Your CNI plugin must support NetworkPolicy enforcement.
What is the difference between soft and hard multi-tenancy?
Soft multi-tenancy assumes some trust between tenants, such as teams in one company. Hard multi-tenancy assumes tenants may be hostile and needs stronger isolation.
How do I prevent noisy neighbours?
Set a ResourceQuota and LimitRange in every tenant namespace, and rely on API Priority and Fairness to stop one tenant from flooding the API server.
What is a virtual control plane?
It gives each tenant its own Kubernetes API server running on shared infrastructure, so tenants can install their own CRDs and have admin rights without affecting others. vCluster and Kamaji are examples.
Which Pod Security level should tenant namespaces use?
Use restricted where possible. It blocks privileged containers, host namespaces, hostPath volumes and running as root.
Should tenants manage their own NetworkPolicies?
They can manage policies inside their namespace, but cluster admins should enforce baseline rules that tenants cannot remove, using CNI cluster-wide policies or admission control.
References
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0