Part of our kubernetes series, lets Understand Kubernetes RBAC with concepts. Also, will try to cover some examples with top misconfigurations. Before we jump into kubernetes RBAC, let’s see what RBAC is.
What Is Role-Based Access Control?
Role-based access control is control over user groups and access to resources based on a defined role.
According to the National Institute of Science and Technology (NIST), “rudimentary forms of role-based access control were implemented in a variety of ad hoc forms on many systems beginning in the 1970s.” However, a formal model wasn’t proposed until 1992. Ferrailo and Kuhn published a paper that proposed an alternative to the traditional models of Mandatory Access Control (MAC) and Discretionary Access Control (DAC). RBAC defined three basic requirements for access control:
1. Role Assignment: subjects are assigned roles and only allowed transactions if allowed by the defined user-role.
2. Role Authorization: subjects only use roles for which they are authorized.
3. Transaction Authorization: subjects only execute transactions authorized by that subject’s role memberships.
A role is a collection of permissions. This allows organizations to grant appropriate permissions to employees or contractors, and ensure privileges and permissions keep to a role hierarchy.
What Is Kubernetes RBAC?
The Kubernetes API provides access to sensitive data, including deployment details, persistent storage settings, and secrets. Over the years, the Kubernetes community has provided several important security features to the Kubernetes API, including role-based access control (RBAC).
RBAC helps protect Kubernetes clusters by letting you control who has access to each API resource. This is much important, as several misconfigurations can lead to weak access controls that fail to implement the principle of least privilege.
Lets check about Kubernetes RBAC in detail.
Why Does RBAC Matter?
An RBAC mechanism provides fine-grained control over user privileges for software applications. It helps avoid accounts with excessive privileges by assigning functions to the individual users who need them.
Providing users with too many access privileges increases the risk of stolen or lost credentials and exposes the organization to insider threats. For example, developers should not have permission to delete a production deployment. Even if you trust every employee in your organization, malicious outsiders could gain control over privileged accounts through social engineering or phishing attacks.
RBAC in Kubernetes allows you to create policies that prevent users from performing admin-level actions, such as deleting pods.
There are several ways to use RBAC in a Kubernetes project:
- Integrate with an existing enterprise solution — the RBAC solution manages access to protected resources via a central role directory. Users can authenticate their identity and access resources in a Kubernetes pod, typically using a single sign-on (SSO) solution.
- Maintain fine-grained control over user access within a cluster — you can use RBAC to control which users have access to the Kubernetes cluster and the level of access to various parts of the project. This fine-grained control allows team members to access only the specific resources they need.
Kubernetes RBAC Basic Concepts
Here are some of the main Kubernetes elements for RBAC.
Role and Cluster Roles
A Role or ClusterRole in Kubernetes contains the rules and permissions for a RBAC given role. These permissions are always additive, so there are no negative rules like “deny.”
RBAC roles set the permissions for a specific namespace. When you create a new role, you must specify its namespace. In contrast, a cluster role is a resource without a namespace. These two resources (Role, ClusterRole) have different names because all Kubernetes objects must always be namespaced or non-namespaced — the same object cannot be both.
Cluster roles serve a variety of purposes, including:
1. Defining permissions for namespace resources that provide access to a specific namespace.
2. Defining permissions for namespace resources that provide access to all namespaces.
3. Defining permissions for cluster-wide resources.
You can use roles to define the permission in a specific namespace, while cluster roles can define permissions across the entire cluster.
Role and Cluster Role Bindings
Role bindings –RoleBinding grant the permissions defined by roles to the relevant user or user group. They contain a list of service accounts, users, or groups – subjects - and references to the relevant roles. While a role binding grants permissions in a specified namespace, a cluster role binding –ClusterRoleBinding grants access across the cluster.
Role bindings can reference any role in a given namespace. Alternatively, a role binding can reference a cluster role and bind it to the role binding’s namespace. You can use cluster role bindings to bind cluster roles to cluster-wide namespaces.
The name of either type of binding must include a valid path segment.
Resource References
The Kubernetes API represents and enables access to most resources using the string representing the object’s name – for example, “pods” to represent a pod. The RBAC system can refer to a resource using the name displayed in the URL of the associated API endpoint. A Kubernetes API might also refer to sub-resources such as pod logs.
Aggregated Cluster Roles
You can combine multiple cluster roles into an aggregated cluster role. A Kubernetes controller that runs in your cluster control plane will detect a ClusterRole object with aggregation rule settings. An aggregationRule defines the label selector the controller must use to match all the cluster role objects to be included in the aggregated cluster role’s rules field.
When you create a cluster role matching an existing aggregated cluster role’s label selector, the controller will add the new rule to the aggregated cluster role. For example, you can add a new rule to a specific cluster role (i.e., “monitoring”) by creating another cluster role marked rbac.role.com/aggregate-to-monitoring: true.
All default user roles will use this cluster role aggregation, allowing cluster administrators to include custom resource rules. You can use the rules served by aggregated API servers or custom resource definitions to extend your default roles. For example, a cluster role might allow the default “admin” role to manage specific custom resources, while the “view” role can only read, but not edit, the resources.
User Accounts and Service Accounts
Kubernetes allows you to define access permissions for a human user (or group users) using RBAC policies. Kubernetes recognizes each human user as a user account. Alternatively, you can use RBAC policies to control the behavior of a software resource that Kubernetes recognizes as a service account.
Kubernetes conceptually distinguishes between user and service accounts, but this distinction is irrelevant to RBAC policies.
Verbs for Permissions Management
However, Kubernetes RBAC defines specific actions, providing a set of verbs to describe the actions an account can run on a given resource.
For instance, you might allow users to “create” and “list” specific resources, specifying the relevant verbs in your RBAC policy.
Run the following to see the list of all available verbs in a cluster:
# kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1# kubectl get --show-kind --ignore-not-found -l = -n
Kubernetes RBAC Example
Now that we know the fundamentals of RBAC in Kubernetes, let’s look at how to use it.
Check for RBAC Support
First, make sure that your cluster supports RBAC. RBAC was introduced with Kubernetes 1.6, and most clusters enable it by default, but it never hurts to check.
Look for a RBAC configuration file in /etc/kubernetes/manifests on your master node(s) or the Kubernetes API server pod, and make sure it contains the flag:
--authorization-mode=Node,RBACDefine Users and Service Accounts
Next, you need to define users and/or service accounts, to which you’ll later assign permissions. As a simple example, here are the steps for creating a user account for a user whom we’ll name John.
Start by creating a new private key:
# openssl genrsa -out motoskia.key 2048Then, you need to create a certificate signing request containing the public key and other
subject information:
# openssl req -new -key motoskia.key -out motoskia.csr -subj “/CN=motoskia/O=examplegroup”
Note that Kubernetes will use the Organization (O=examplegroup) field to determine user
group membership for RBAC.
You will sign this CSR using the root Kubernetes CA, found in /etc/kubernetes/pki for this example (the file location in your deployment may vary):
# openssl x509 -req -in motoskia.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key - CAcreateserial -out motoskia.crt
Signature ok
subject=/CN=motoskia/O=examplegroupGetting CA Private Key
You can inspect the new certificate:
# openssl x509 -in john.crt -text
Certificate:
Data:
Version: 1 (0x0)
Serial Number: 11349652818125543147 (0x9cf3f4qwe0b372bb)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=kubernetes
Validity
Not Before: Jun 2 10:12:24 2023 GMT
Not After : Jun 2 10:12:24 2023 GMT
Subject: CN=motoskia, O=examplegroup
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)Finally, register the new credentials and config context:
# kubectl config set-credentials motoskia — client-certificate=/home/newusers/motoskia.crt — client-key=/home/newusers/motoskia.key# kubectl config set-context motoskia@kubernetes — cluster=kubernetes — user=motoskiaCreate a Role or ClusterRole
Next, we need to create a Role or ClusterRole. This is where we define the actions that can be performed on a resource.
For example, here is a role that grants get, watch, and list permissions for pods.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: pod-reader
rules:
- apiGroups: [“”] # “” indicates the core API group
resources: [“pods”]
verbs: [“get”, “watch”, “list”]
Create a RoleBinding or ClusterRoleBinding
Finally, we have to “bind” our Role or ClusterRole to the user or account we created. The binding is what actually allows the specified user or account to perform the given role.
# kubectl create rolebinding podreader-binding — user=motoskiaTest Access to Resource
The command auth can-i can be used to query the RBAC authorization mode. The -n flag specifies the namespace, and the – as flag identifies the service account. For service accounts, you need to prefix the role binding with system:serviceaccount.
To test if a service account can access resources:
1. Use the following command for user-impersonation:# kubectl <verb-name> <resource-name> — as=argocd
2. Use the following command to verify API access:# kubectl auth can-i <verb-name> <resource-name>
3. Run the following command to see if requests can be made from the service account and list pod in the namespace:
# kubectl auth can-i get pods -n demo — as=system:serviceaccount:test:demoAccount
Top 4 RBAC Misconfigurations
Here are some common configuration issues with role-based access control.
1. Upgrading from ABAC
If your cluster ran an older Kubernetes version, it might have weak ABAC policies, such as full API access privileges to service accounts.
A default RBAC policy grants scoped permissions to a set of nodes, controllers, and components on the control plane. Still, it doesn’t grant access to any service account beyond the kube-system namespace (except discovery permissions granted to every authenticated user).
While this approach is secure, it can disrupt the existing workloads that expect to receive API permissions automatically. You can manage the transition to RBAC by running both authorizers (ABAC and RBAC) and specifying a policy file with your existing ABAC policy:
--authorization-mode=…,RBAC,ABAC
--authorization-policy-file=mypolicy.json
Here is a detailed explanation of the first command line option:
- If an early authorizer like Node denies a request, the RBAC authorizer will attempt to authorize the relevant API request.
- If the RBAC authorizer denies the API request, the ABAC authorizer will run.
- The system will allow any request authorized by one of the policies (ABAC or RBAC), even if the other component does not.
When you run the kube-apiserver at a log level of five or above for the RBAC policy, you can view the RBAC-denied requests in the API server log with the RBAC prefix. This information is useful for determining the appropriate roles for each service account, user, and group.
After you grant roles to each service account and the workloads run without RBAC denials in the server logs, it is safe to remove your ABAC authorizer.
2. Improperly Using Role Aggregation
Role aggregation is available in Kubernetes versions 1.9 and up, allowing you to simplify granting privileges by combining them in defined roles. However, if you don’t carefully review every role aggregation, they can modify the role’s intended use. For example, a system:view role might improperly combine rules that violate the role’s purpose, such as using verbs that allow users to modify clusters.
3. Granting Duplicated Roles
Role definitions can overlap and give users the same level of access in multiple ways. Sometimes, roles may overlap intentionally, with administrators knowingly granting duplicate privileges. However, this configuration is complex and makes it harder to understand which users have each access permission.
Another issue is that revoking access is more difficult when the administrator doesn’t realize that different role bindings provide the same access.
4. Keeping Unused Roles
If you create a role but don’t grant it to any user, it adds to the complexity of managing RBAC policies without improving security. Likewise, roles that only relate to deleted users (such as a service account in a deleted namespace or a former employee) create excess noise, obscuring the relevant configurations. Removing inactive or unused roles is usually safe, allowing you to focus on active ones.
Kubernetes RBAC Best Practices
Kubernetes RBAC is a powerful tool. The following are best practices for using it as responsibly as possible.
Minimize API Server Flags
The Kubernetes API has a number of optional flags that, when enabled, may simplify some aspects of Kubernetes management. But they also heighten security risks. Avoid these flags whenever possible:
· –insecure-port: Opens access to unauthorized, unauthenticated requests. If this parameter is equal to 0, it means no insecure port.
· –insecure-bind-address: Ideally, you should avoid insecure connections altogether, but in case you really need them, you can use this parameter to just bind to localhost. Make sure this parameter is not set, or at least not set to a network-reachable IP address.
· –anonymous-auth: Enables anonymous requests to the secure port of the API server.
Follow Least Privilege
As with any RBAC system, Kubernetes RBAC is most effective when admins follow the principle of least privilege, which means that each user or account receives only the minimum privileges necessary to perform their job.
In practice, this means doing things like using Roles instead of ClusterRoles wherever possible. Even though setting up Roles for each namespace is a bit more cumbersome than being able to define a ClusterRole to the entire cluster, Roles are more secure because they apply to fewer resources.
Likewise, avoid wildcards [“*”] when you define verbs or access to resources. Wildcards are the “chmod 777” of Kubernetes: they’re convenient to apply, but they open up all sorts of unauthorized access holes.
Avoid Default Service Accounts
Kubernetes creates default service accounts to identify the processes running in a pod.
You may be tempted to use these default accounts to assign permissions rather than going to the trouble of setting up your own accounts. That’s not a best practice. Instead, create service-specific service accounts, which provide the foundation for more granular access controls.
By the way, if you’re wondering about default users, Kubernetes doesn’t have default user accounts, so you don’t need to worry about those. You have to create users explicitly if you want to assign Roles or ClusterRoles to them.
Update RBAC Policies Continuously
Kubernetes RBAC is only as effective as the RBAC policies you create. And if your policies become outdated, you may quickly end up with users or service accounts that are over-permissioned.
For that reason, it’s a best practice to ensure that whenever you create, remove, or update the permissions of any user or service account, or when you create a namespace or pods, you also modify or delete all RBAC policies associated with that entity. This can be somewhat tedious, given that Kubernetes has multiple types of files associated with RBAC (Roles, RoleBindings, ClusterRoles, and ClusterRoleBindings). Still, it’s important to make RBAC policy updates a systematic and continuous part of Kubernetes management.
Minimize Distribution of Privileged Tokens
Ideally, you should not assign service accounts with strong privileges to your pods. If your workload requires strong privileges, consider the following:
- Limit the number of nodes with pods running sensitive workloads – make sure all DaemonSets are really required and run with least privileges to limit the blast radius for container escapes.
- Do not run sensitive pods next to untrusted or public pods – you can achieve these using taints, tolerations,
NodeAffinity, orPodAntiAffinity. Pay particular attention to situations where untrusted pods do not meet the security standards of sensitive pods.
Cluster Administrator Roles
The built-in cluster-admin role grants virtually unlimited access to your cluster. During the migration from legacy ABAC controllers to RBAC, some administrators and users ignored warnings in the documentation and duplicated the broad cluster-admin permission from the old ABAC configuration. Identify if this was done in your cluster and remove this permission.
If a user or group is regularly granted cluster administrator rights, an account compromise or mistake can have dangerously far-reaching effects. Service accounts generally do not need this access. In either case, you should create a more granular role or cluster role and grant it only to specific users who need it.