Description

Deckhouse Virtualization Platform (DVP) provides a standard set of roles for managing access to project and cluster resources. There are two types of roles:

  • Use roles: Assigned to project users, allowing them to manage resources within a specified project.
  • Manage roles: Intended for DVP administrators, granting them permissions to manage resources at the platform-wide level.

DVP permissions are configured using the standard Kubernetes RBAC approach. This involves creating RoleBinding or ClusterRoleBinding resources to assign the appropriate role.

Use roles

The use role can only be used in the RoleBinding resource.

Use roles are intended to assign rights to a user in a specific namespace. Users refer to, for example, developers who use a cluster configured by an administrator to deploy their applications. Such users don’t need to manage DVP modules or a cluster, but they need to be able to, for example, create their Ingress resources, configure application authentication, and collect logs from applications.

The use role defines permissions for accessing namespaced resources of modules and standard namespaced resources of Kubernetes (Pod, Deployment, Secret, ConfigMap, etc.).

The module creates the following use roles:

  • d8:use:role:viewer: Allows viewing standard Kubernetes resources in a specific namespace, except for Secrets and RBAC resources, as well as authenticating in the cluster.
  • d8:use:role:user: In addition to the role d8:use:role:viewer it allows viewing secrets and RBAC resources in a specific namespace, connecting to pods, deleting pods (but not creating or modifying them), executing kubectl port-forward and kubectl proxy, as well as changing the number of replicas of controllers.
  • d8:use:role:manager: In addition to the role d8:use:role:user it allows managing module resources (for example, Certificate, PodLoggingConfig, etc.) and standard namespaced Kubernetes resources (Pod, ConfigMap, CronJob, etc.) in a specific namespace.
  • d8:use:role:admin: In addition to the role d8:use:role:manager it allows managing the resources ResourceQuota, ServiceAccount, Role, RoleBinding, NetworkPolicy in a specific namespace.

Example of administrator permissions granted to the joe user in the vms project:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: project-admin-joe
  namespace: vms
subjects:
- kind: User
  name: joe@example.com # The .spec.email parameter for user joe from users.deckhouse.io
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: d8:use:role:admin
  apiGroup: rbac.authorization.k8s.io

Example of administrator permissions granted to the vms-admins user group in the vms project:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: project-admin-joe
  namespace: vms
subjects:
- kind: Group
  name: vms-admins # the .spec.name parameter for the vms-admins group from groups.deckhouse.io
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: d8:use:role:admin
  apiGroup: rbac.authorization.k8s.io

Manage roles

The manage role does not grant access to the namespace of user applications.

The manage role grants access only to system namespaces (starting with d8- or kube-), and only to those system namespaces where the modules of the corresponding role subsystem are running.

Manage roles are intended for assigning rights to manage the entire DVP or a part of it (the subsystem), but not the users applications themselves. The manage role, for example, can allow a security administrator to manage security modules (responsible for the security functions of the cluster). Thus, the security administrator will be able to configure authentication, authorization, security policies, etc., but will not be able to manage other cluster functions (such as network and monitoring settings) or change settings in the namespaces of users applications.

The manage role defines access rights:

  • To cluster-wide Kubernetes resources.
  • To manage DVP modules (moduleConfig resource) within the subsystem of the role, or to all DVP modules for the role d8:manage:all:*.
  • To manage cluster-wide resources of DVP modules within the subsystem of the role, or to all resources of DVP modules for the role d8:manage:all:*.
  • To system namespaces (starting with d8- or kube-) in which the modules of the subsystem of the role operate, or to all system namespaces for the role d8:manage:all:*.

The manage role name format is d8:manage:<SUBSYSTEM>:<ACCESS_LEVEL>, where:

  • SUBSYSTEM is the role’s subsystem. It can be one of the subsystem, or all, for access across all subsystems.

  • ACCESS_LEVEL is the access level.

    Examples of manage roles:

    • d8:manage:all:viewer: Access to view the configuration of all DVP modules (moduleConfig resource), their cluster-wide resources, their namespaced resources, and standard Kubernetes objects (except Secrets and RBAC resources) in all system namespaces (starting with d8- or kube-).
    • d8:manage:all:manager: Similar to the role d8:manage:all:viewer, but with admin-level access, i.e., view/create/modify/delete the configuration of all DVP modules (moduleConfig resource), their cluster-wide resources, their namespaced resources, and standard Kubernetes objects in all system namespaces (starting with d8- or kube-).
    • d8:manage:observability:viewer: Access to view the configuration of DVP modules (moduleConfig resource) from the observability area, their cluster-wide resources, their namespaced resources, and standard Kubernetes objects (except secrets and RBAC resources) in the system namespaces d8-log-shipper, d8-monitoring, d8-okmeter, d8-operator-prometheus, d8-upmeter, kube-prometheus-pushgateway.

The module provides two access levels for administrators:

  • viewer: Allows viewing standard Kubernetes resources, the configuration of modules (resources moduleConfig), cluster-wide resources of modules, and namespaced resources of modules in the module namespace.
  • manager: In addition to the role viewer it allows managing standard Kubernetes resources, the configuration of modules (resources moduleConfig), cluster-wide resources of modules, and namespaced resources of modules in the module namespace.

Example of cluster manager permissions granted to the joe user:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-admin-joe
subjects:
- kind: User
  name: joe@example.com # The .spec.email parameter for user joe from users.deckhouse.io
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: d8:manage:all:manager # Manage role name
  apiGroup: rbac.authorization.k8s.io

DVP can grant restricted permissions to administrators for managing resources and modules associated with specific subsystems.

To assign network subsystem manager permissions to the joe user, use the following configuration:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: network-admin-joe
subjects:
- kind: User
  name: joe@example.com # The .spec.email parameter for user joe from users.deckhouse.io
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: d8:manage:networking:manager # Manage role name
  apiGroup: rbac.authorization.k8s.io

Subsystems of the role-based model

Each DVP module belongs to a specific subsystem. For each subsystem, there is a set of roles with different levels of access. Roles are updated automatically when the module is enabled or disabled.

For example, for the networking subsystem, there are the following manage roles that can be used in ClusterRoleBinding:

  • d8:manage:networking:viewer
  • d8:manage:networking:manager

The subsystem of the role restricts its action to all system namespaces of the cluster (subsystem all) or to those namespaces in which the area modules operate (see the table of area compositions).

Role-based model subsystems composition table.

Namespaces in which modules of a scope operate
all All modules All namespaces
deckhouse
  • deckhouse
  • multitenancy-manager
  • d8-multitenancy-manager
  • d8-system
infrastructure
  • chrony
  • cloud-provider-aws
  • cloud-provider-azure
  • cloud-provider-dynamix
  • cloud-provider-gcp
  • cloud-provider-huaweicloud
  • cloud-provider-openstack
  • cloud-provider-vcd
  • cloud-provider-vsphere
  • cloud-provider-yandex
  • cloud-provider-zvirt
  • descheduler
  • istio
  • keepalived
  • local-path-provisioner
  • loki
  • node-manager
  • terraform-manager
  • vertical-pod-autoscaler
  • d8-chrony
  • d8-cloud-instance-manager
  • d8-cloud-provider-aws
  • d8-cloud-provider-azure
  • d8-cloud-provider-dynamix
  • d8-cloud-provider-gcp
  • d8-cloud-provider-huaweicloud
  • d8-cloud-provider-openstack
  • d8-cloud-provider-vcd
  • d8-cloud-provider-vsphere
  • d8-cloud-provider-yandex
  • d8-cloud-provider-zvirt
  • d8-descheduler
  • d8-istio
  • d8-keepalived
  • d8-local-path-provisioner
  • d8-monitoring
  • d8-system
  • kube-system
kubernetes
  • control-plane-manager
  • flow-schema
  • kube-dns
  • kube-proxy
  • priority-class
  • d8-system
  • kube-system
networking
  • cilium-hubble
  • cni-cilium
  • cni-flannel
  • cni-simple-bridge
  • flow-schema
  • ingress-nginx
  • istio
  • kube-dns
  • kube-proxy
  • metallb
  • network-gateway
  • network-policy-engine
  • node-local-dns
  • openvpn
  • static-routing-manager
  • d8-cni-cilium
  • d8-cni-flannel
  • d8-cni-simple-bridge
  • d8-ingress-nginx
  • d8-istio
  • d8-metallb
  • d8-network-gateway
  • d8-openvpn
  • d8-static-routing-manager
  • d8-system
  • kube-system
observability
  • dashboard
  • extended-monitoring
  • log-shipper
  • monitoring-applications
  • monitoring-custom
  • monitoring-deckhouse
  • monitoring-kubernetes
  • monitoring-kubernetes-control-plane
  • monitoring-ping
  • okmeter
  • operator-prometheus
  • prometheus
  • prometheus-metrics-adapter
  • prometheus-pushgateway
  • upmeter
  • d8-dashboard
  • d8-log-shipper
  • d8-monitoring
  • d8-okmeter
  • d8-operator-prometheus
  • d8-upmeter
  • kube-prometheus-pushgateway
security
  • admission-policy-engine
  • cert-manager
  • operator-trivy
  • runtime-audit-engine
  • secret-copier
  • user-authn
  • user-authz
  • d8-admission-policy-engine
  • d8-cert-manager
  • d8-operator-trivy
  • d8-runtime-audit-engine
  • d8-user-authn

Current role-based model

Features:

In addition to the RBAC, you can use a set of high-level roles in the module:

  • User: Has access to information about all objects (including viewing pod logs) but cannot exec into containers, read secrets, and perform port-forwarding.
  • PrivilegedUser: The same as User + can exec into containers, read secrets, and delete pods (and thus, restart them).
  • Editor: Is the same as PrivilegedUser + can create and edit all objects that are usually required for application tasks.
  • Admin: The same as Editor + can delete service objects (auxiliary resources such as ReplicaSet, certmanager.k8s.io/challenges and certmanager.k8s.io/orders).
  • ClusterEditor: The same as Editor + can manage a limited set of cluster-wide objects that can be used in application tasks (ClusterXXXMetric, KeepalivedInstance, DaemonSet, etc.). This role is best suited for cluster operators.
  • ClusterAdmin: The same as both ClusterEditor and Admin + can manage cluster-wide service objects (e.g., MachineSets, Machines, OpenstackInstanceClasses…, as well as ClusterAuthorizationRule, ClusterRoleBindings and ClusterRole). This role is best suited for cluster administrators. Note that since ClusterAdmin can edit ClusterRoleBindings, they can broaden their privileges within the cluster.
  • SuperAdmin: Can perform any actions with any objects (note that namespaceSelector and limitNamespaces restrictions remain valid).

Currently, the multi-tenancy mode (namespace-based authorization) is implemented according to a temporary scheme and isn’t guaranteed to be entirely safe and secure!

If a ClusterAuthorizationRule Custom Resource contains the namespaceSelector field, neither limitNamespaces nor allowAccessToSystemNamespacesare taken into consideration.

The allowAccessToSystemNamespaces, namespaceSelector and limitNamespaces options in the custom resource will no longer be applied if the authorization system’s webhook is unavailable for some reason. As a result, users will have access to all namespaces. After the webhook availability is restored, the options will become relevant again.

Experimental role-based model

Unlike the current DVP role-based model, the new role-based one does not use ClusterAuthorizationRule and AuthorizationRule resources. All access rights are configured in the standard Kubernetes RBAC way, i.e., by creating RoleBinding or ClusterRoleBinding resources and specifying one of the roles prepared by the user-authz module in them.

The module creates special aggregated cluster roles (ClusterRole). By using these roles in RoleBinding or ClusterRoleBinding, you can do the following:

  • Manage access to modules of a specific subsystem.

    For example, you can use the d8:manage:networking:manager role in ClusterRoleBinding to allow a network administrator to configure network modules (such as cni-cilium, ingress-nginx, istio, etc.).

  • Manage access to user resources of modules within the namespace.

    For example, the d8:use:role:manager role in RoleBinding enables deleting/creating/editing the PodLoggingConfig resource in the namespace. At the same time, it does not grant access to the cluster-wide ClusterLoggingConfig and ClusterLogDestination resources of the log-shipper module, nor does it allow configuration of the log-shipper module itself.

Default access list for each role

verbs aliases:

  • read - get, list, watch
  • read-write - get, list, watch, create, delete, deletecollection, patch, update
  • write - create, delete, deletecollection, patch, update

Role User:

read:
    - apiextensions.k8s.io/customresourcedefinitions
    - apps/daemonsets
    - apps/deployments
    - apps/replicasets
    - apps/statefulsets
    - autoscaling.k8s.io/verticalpodautoscalers
    - autoscaling/horizontalpodautoscalers
    - batch/cronjobs
    - batch/jobs
    - configmaps
    - discovery.k8s.io/endpointslices
    - endpoints
    - events
    - events.k8s.io/events
    - extensions/daemonsets
    - extensions/deployments
    - extensions/ingresses
    - extensions/replicasets
    - extensions/replicationcontrollers
    - limitranges
    - metrics.k8s.io/nodes
    - metrics.k8s.io/pods
    - namespaces
    - networking.k8s.io/ingresses
    - networking.k8s.io/networkpolicies
    - nodes
    - persistentvolumeclaims
    - persistentvolumes
    - pods
    - pods/log
    - policy/poddisruptionbudgets
    - rbac.authorization.k8s.io/rolebindings
    - rbac.authorization.k8s.io/roles
    - replicationcontrollers
    - resourcequotas
    - serviceaccounts
    - services
    - storage.k8s.io/storageclasses

Role PrivilegedUser (includes all rules from the role User):

create:
    - pods/eviction
create,get:
    - pods/attach
    - pods/exec
delete,deletecollection:
    - pods
read:
    - secrets

Role Editor (includes all rules from the role User, PrivilegedUser):

read-write:
    - apps/deployments
    - apps/statefulsets
    - autoscaling.k8s.io/verticalpodautoscalers
    - autoscaling/horizontalpodautoscalers
    - batch/cronjobs
    - batch/jobs
    - configmaps
    - discovery.k8s.io/endpointslices
    - endpoints
    - extensions/deployments
    - extensions/ingresses
    - networking.k8s.io/ingresses
    - persistentvolumeclaims
    - policy/poddisruptionbudgets
    - serviceaccounts
    - services
write:
    - secrets

Role Admin (includes all rules from the role User, PrivilegedUser, Editor):

create,patch,update:
    - pods
delete,deletecollection:
    - apps/replicasets
    - extensions/replicasets

Role ClusterEditor (includes all rules from the role User, PrivilegedUser, Editor):

read:
    - rbac.authorization.k8s.io/clusterrolebindings
    - rbac.authorization.k8s.io/clusterroles
write:
    - apiextensions.k8s.io/customresourcedefinitions
    - apps/daemonsets
    - extensions/daemonsets
    - storage.k8s.io/storageclasses

Role ClusterAdmin (includes all rules from the role User, PrivilegedUser, Editor, Admin, ClusterEditor):

read-write:
    - deckhouse.io/clusterauthorizationrules
write:
    - limitranges
    - namespaces
    - networking.k8s.io/networkpolicies
    - rbac.authorization.k8s.io/clusterrolebindings
    - rbac.authorization.k8s.io/clusterroles
    - rbac.authorization.k8s.io/rolebindings
    - rbac.authorization.k8s.io/roles
    - resourcequotas

You can get an additional list of access rules for a role from the cluster (existing user-defined rules and non-default rules from other DVP modules):

D8_ROLE_NAME=Editor
d8 k get clusterrole -A -o jsonpath="{range .items[?(@.metadata.annotations.user-authz\.deckhouse\.io/access-level=='$D8_ROLE_NAME')]}{.rules}{'\n'}{end}" | jq -s add