The multitenancy-manager module: usage examples

Default project templates

The following project templates are included in the Deckhouse Kubernetes Platform:

• default — a template that covers basic project use cases:
  • resource limitation
  • network isolation
  • automatic alerts and log collection
  • choice of security profile
  • project administrators setup

  Template description on GitHub.

• secure — includes all the capabilities of the default template and additional features:
  • setting up permissible UID/GID for the project
  • audit rules for project users’ access to the Linux kernel
  • scanning of launched container images for CVE presence

  Template description on GitHub.

• secure-with-dedicated-nodes — includes all the capabilities of the secure template and additional features:
  • defining the node selector for all the pods in the project: if a pod is created, the node selector pod will be substituted with the project’s node selector automatically.
  • defining the default toleration for all the pods in the project: if a pod is created, the default toleration will be added to the pod automatically.

  Template description on GitHub.

To list all available parameters for a project template, execute the command:

d8 k get projecttemplates <PROJECT_TEMPLATE_NAME> -o jsonpath='{.spec.parametersSchema.openAPIV3Schema}' | jq

Creating a project

1. To create a project, create the Project resource by specifying the name of the project template in .spec.projectTemplateName field.

2. In the .spec.parameters field of the Project resource, specify the parameter values suitable for the ProjectTemplate .spec.parametersSchema.openAPIV3Schema.

   Example of creating a project using the Project resource from the default ProjectTemplate:

   apiVersion: deckhouse.io/v1alpha2
   kind: Project
   metadata:
     name: my-project
   spec:
     description: This is an example from the Deckhouse documentation.
     projectTemplateName: default
     parameters:
       resourceQuota:
         requests:
           cpu: 5
           memory: 5Gi
           storage: 1Gi
         limits:
           cpu: 5
           memory: 5Gi
       networkPolicy: Isolated
       podSecurityProfile: Restricted
       extendedMonitoringEnabled: true
       administrators:
       - subject: Group
         name: k8s-admins
   

3. To check the status of the project, execute the command:

   d8 k get projects my-project
   

   A successfully created project should be in the Deployed state. If the state equals Error, add the -o yaml argument to the command (e.g., d8 k get projects my-project -o yaml) to get more detailed information about the error.

Creating a project automatically for a namespace

You can create a new project for a namespace. To do this, add the projects.deckhouse.io/adopt annotation to the namespace. For example:

1. Create a new namespace:

   d8 k create ns test
   

2. Add the annotation:

   d8 k annotate ns test projects.deckhouse.io/adopt=""
   

3. Make sure that the project was created:

   d8 k get projects
   

   A new project corresponding to the namespace will appear in the project list:

   NAME        STATE      PROJECT TEMPLATE   DESCRIPTION                                            AGE
   deckhouse   Deployed   virtual            This is a virtual project                              181d
   default     Deployed   virtual            This is a virtual project                              181d
   test        Deployed   empty                                                                     1m
   

You can change the template of the created project to the existing one.

Creating your own project template

Default templates cover basic project use cases and serve as a good example of template capabilities.

To create your own template:

1. Take one of the default templates as a basis, for example, default.

2. Copy it to a separate file, for example, my-project-template.yaml using the command:

   d8 k get projecttemplates default -o yaml > my-project-template.yaml
   

3. Edit the my-project-template.yaml file, make the necessary changes.

   It is necessary to change not only the template, but also the scheme of input parameters for it.

   Project templates support all Helm templating functions.

4. Change the template name in the .metadata.name field.

5. Apply your new template with the command:

   d8 k apply -f my-project-template.yaml
   

6. Check the availability of the new template with the command:

   d8 k get projecttemplates <NEW_TEMPLATE_NAME>
   

Using labels to manage resources

When creating resources in ProjectTemplate, you can use special labels to control how the multitenancy-manager processes these resources.

Skipping creation of the heritage: multitenancy-manager label

By default, all resources created from ProjectTemplate receive the label heritage: multitenancy-manager.
This label prohibits changes to resources by users or any other controller except multitenancy-manager.
If you need to allow resource modification (for example, for compatibility with other systems, or if implementing your own control over the created objects), add the label projects.deckhouse.io/skip-heritage-label to the resource.

Example:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
  namespace: {{ .projectName }}
  labels:
    projects.deckhouse.io/skip-heritage-label: "true"
    app: my-app
data:
  key: value

In this case, the resource will receive the labels projects.deckhouse.io/project and projects.deckhouse.io/project-template, but will not receive the label heritage: multitenancy-manager.

Excluding resources from management by multitenancy-manager

If you need to exclude a resource from management by multitenancy-manager (for example, if the resource should be managed manually or by another controller), add the label projects.deckhouse.io/unmanaged to the resource.

Example:

---
apiVersion: v1
kind: Secret
metadata:
  name: external-secret
  namespace: {{ .projectName }}
  labels:
    projects.deckhouse.io/unmanaged: "true"
type: Opaque
data:
  token: <base64-encoded-value>

Resources with the label projects.deckhouse.io/unmanaged:

• Will be created only once when the project is created;
• Will not be updated with subsequent template changes or updates;
• Will not be monitored in the project’s status;
• Will receive the labels projects.deckhouse.io/project and projects.deckhouse.io/project-template but will not receive the label heritage: multitenancy-manager.

Implementing validation of object changes with a custom label

The multitenancy-manager module uses ValidatingAdmissionPolicy to protect resources labeled heritage: multitenancy-manager from manual changes.
You can implement similar validation for resources with any label.

How validation works in multitenancy-manager

Validation occurs for objects labeled heritage: multitenancy-manager.
The following components are used for this:

1. ValidatingAdmissionPolicy: Defines validation rules:
   • Operations: UPDATE and DELETE.
   • Check: only operations on behalf of the controller’s service account are allowed.
   • Applies to all resources and API groups.
2. ValidatingAdmissionPolicyBinding: Defines which objects the validation applies to:
   • Uses namespaceSelector and objectSelector to select resources by the label heritage: multitenancy-manager.

Creating your own validation

To implement validation for resources with a different label (for example, heritage: my-custom-label):

1. Create a file with the ValidatingAdmissionPolicy and ValidatingAdmissionPolicyBinding resource manifests:

   apiVersion: admissionregistration.k8s.io/v1
   kind: ValidatingAdmissionPolicy
   metadata:
     name: my-custom-label-validation
   spec:
     failurePolicy: Fail
     matchConstraints:
       resourceRules:
         - apiGroups:   ["*"]
           apiVersions: ["*"]
           operations:  ["UPDATE", "DELETE"]
           resources:   ["*"]
           scope: "*"
     validations:
       - expression: 'request.userInfo.username == "system:serviceaccount:my-namespace:my-service-account"' # Replace with your service account
         reason: Forbidden
         messageExpression: 'object.kind == ''Namespace'' ? ''This resource is managed by '' + object.metadata.name + '' system. Manual modification is forbidden.''
           : ''This resource is managed by '' + object.metadata.namespace + '' system. Manual modification is forbidden.'''
   ---
   apiVersion: admissionregistration.k8s.io/v1
   kind: ValidatingAdmissionPolicyBinding
   metadata:
     name: my-custom-label-validation
   spec:
     policyName: my-custom-label-validation
     validationActions: [Deny, Audit]
     matchResources:
       namespaceSelector:
         matchLabels:
           heritage: my-custom-label
       objectSelector:
         matchLabels:
           heritage: my-custom-label
   

2. Configure the validation parameters:

   • policyName: Unique policy name (must match in Policy and Binding).
   • request.userInfo.username: The name of the service account allowed to change resources (replace with your service account).
   • heritage: my-custom-label: The value of the heritage label for your resources (replace with your value). The use of the values multitenancy-manager, deckhouse is prohibited.
   • failurePolicy: Fail: Policy on validation failure.
     • Fail: Reject the request on validation failure.
     • Ignore: Ignore validation errors.
   • validationActions: Validation actions:
     • Deny: Deny unauthorized operations.
     • Audit: Record operations in the audit log.

3. Apply the policy:

   d8 k apply -f my-validation-policy.yaml
   

4. Ensure your resources have the corresponding heritage label:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: my-resource
     labels:
       heritage: my-custom-label