Deckhouse Code supplies two runner images: registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner and registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner-helper. Deploying and operating the runners (hosts, manifests, Helm charts, upgrades, monitoring) is the administrator’s responsibility.

A runner is a GitLab Runner process that takes CI/CD jobs from Deckhouse Code and runs them in containers, Kubernetes pods or virtual machines, depending on its executor.

Runner images

The image tag is the GitLab Runner version, for example v19.3.1. Both images are built for linux/amd64 only.

ImageContents
runnerGitLab Runner, the fleeting-plugin-dvp plugin in /usr/local/bin. The process runs as the gitlab-runner user (UID and GID 1001) and reads its configuration from /etc/gitlab-runner/config.toml
runner-helperThe helper that the Docker and Kubernetes executors start next to each job to clone the repository and to upload artifacts and cache; it contains git, git-lfs and bash

The Docker and Kubernetes executors take the helper image from the helper_image parameter. The examples below set it to the runner-helper image of the same version as the runner.

Creating a runner

A runner connects to Deckhouse Code with an authentication token. To create an instance runner and get its token:

  1. Open “Admin” → “CI/CD” → “Runners” and click “Create instance runner”.
  2. In “Tags”, list the tags of the jobs the runner takes, or select “Run untagged jobs”.
  3. Click “Create runner”.
  4. Copy the runner authentication token (it starts with glrt-) from the page that opens.

The examples below use <TOKEN> for this token, <VERSION> for the image tag and https://code.example.com for the address of Deckhouse Code.

Autoscaling on Deckhouse Virtualization Platform

The fleeting-plugin-dvp plugin lets the instance executor run jobs on virtual machines of a VirtualMachinePool in Deckhouse Virtualization Platform. The runner sets the number of virtual machines in the pool (spec.replicas): one virtual machine per job plus the idle reserve. It connects to a virtual machine over SSH and runs the job in its shell. When a virtual machine is no longer needed, the plugin deletes it and lowers spec.replicas.

The plugin creates neither the pool nor the user on the virtual machines: the pool and the SSH access are prepared in advance. In the example below, the runner runs as a Deployment in the cluster of Deckhouse Virtualization Platform, in the code-runner-dvp namespace together with the pool.

Namespace and SSH key

Create the namespace, a key pair and a Secret with the private key:

d8 k create namespace code-runner-dvp
ssh-keygen -t ed25519 -N "" -f code-runner-ssh
d8 k -n code-runner-dvp create secret generic code-runner-ssh --from-file=id_ed25519=code-runner-ssh

The public key code-runner-ssh.pub goes into the pool manifest.

Virtual machine pool

The pool starts empty and removes virtual machines by name (scaleDownPolicy: Explicit), which is what the plugin does on scale-down. The disk image needs an SSH server, bash and git; cloud-init creates the user that the runner connects as. Save the manifest as code-runner-pool.yaml:

apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualMachinePool
metadata:
  name: code-runner
  namespace: code-runner-dvp
spec:
  replicas: 0
  scaleDownPolicy: Explicit
  virtualMachineTemplate:
    spec:
      virtualMachineClassName: generic
      runPolicy: AlwaysOn
      cpu:
        cores: 2
      memory:
        size: 2Gi
      blockDeviceRefs:
        - kind: VirtualDisk
          name: disk
      provisioning:
        type: UserData
        userData: |
          #cloud-config
          package_update: true
          packages:
            - git
          users:
            - name: gitlab-runner
              shell: /bin/bash
              sudo: ALL=(ALL) NOPASSWD:ALL
              lock_passwd: false
              ssh_authorized_keys:
                - <PUBLIC_KEY>
  virtualDiskTemplates:
    - name: disk
      spec:
        dataSource:
          type: ObjectRef
          objectRef:
            kind: VirtualImage
            name: ubuntu-2204
        persistentVolumeClaim:
          size: 20Gi
          storageClassName: <STORAGE_CLASS_NAME>

<PUBLIC_KEY> is the content of code-runner-ssh.pub; ubuntu-2204 is a VirtualImage in the same namespace. The jobs write their working copies and caches to the disk, so the disk size limits the size of a job.

Create the pool:

d8 k apply -f code-runner-pool.yaml

Runner deployment

  1. Save the following manifest as code-runner-dvp.yaml. It holds the runner configuration, the service account with the rights to the pool and the Deployment of the runner with the private key mounted:

    apiVersion: v1
kind: Secret
metadata:
  name: code-runner-config
  namespace: code-runner-dvp
stringData:
  config.toml: |
    concurrent = 4

    [[runners]]
      name = "dvp"
      url = "https://code.example.com"
      token = "<TOKEN>"
      executor = "instance"
      shell = "bash"

      [runners.autoscaler]
        plugin = "fleeting-plugin-dvp"
        max_instances = 5

        [runners.autoscaler.plugin_config]
          namespace = "code-runner-dvp"
          name = "code-runner"

        [runners.autoscaler.connector_config]
          username = "gitlab-runner"
          key_path = "/ssh/id_ed25519"
          use_static_credentials = true
          use_external_addr = false

        [[runners.autoscaler.policy]]
          idle_count = 1
          idle_time = "20m"
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: code-runner
  namespace: code-runner-dvp
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: code-runner
  namespace: code-runner-dvp
rules:
  - apiGroups: ["virtualization.deckhouse.io"]
    resources: ["virtualmachinepools"]
    verbs: ["get", "patch"]
  - apiGroups: ["virtualization.deckhouse.io"]
    resources: ["virtualmachines"]
    verbs: ["get", "list", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: code-runner
  namespace: code-runner-dvp
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: code-runner
subjects:
  - kind: ServiceAccount
    name: code-runner
    namespace: code-runner-dvp
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: code-runner
  namespace: code-runner-dvp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: code-runner
  template:
    metadata:
      labels:
        app: code-runner
    spec:
      serviceAccountName: code-runner
      securityContext:
        runAsUser: 1001
        runAsGroup: 1001
        fsGroup: 1001
      initContainers:
        - name: config
          image: registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner:<VERSION>
          command: ["cp", "/config-secret/config.toml", "/etc/gitlab-runner/config.toml"]
          volumeMounts:
            - name: config-secret
              mountPath: /config-secret
            - name: config
              mountPath: /etc/gitlab-runner
      containers:
        - name: runner
          image: registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner:<VERSION>
          volumeMounts:
            - name: config
              mountPath: /etc/gitlab-runner
            - name: ssh
              mountPath: /ssh
      volumes:
        - name: config-secret
          secret:
            secretName: code-runner-config
        - name: config
          emptyDir: {}
        - name: ssh
          secret:
            secretName: code-runner-ssh
            defaultMode: 0440

    The parameters that bind the runner to the pool:

    • plugin_config.namespace and plugin_config.name: the namespace and the name of the VirtualMachinePool; with no kubeconfig_path, the plugin uses the service account of the pod;
    • connector_config.username: the user that cloud-init creates;
    • connector_config.key_path: the private key whose public part is in ssh_authorized_keys;
    • use_external_addr = false: the runner connects to the internal address of the virtual machine (status.ipAddress);
    • shell = "bash": on Debian and Ubuntu, /bin/sh is dash, and the job preparation script fails under it;
    • max_instances: the upper limit of virtual machines in the pool; idle_count and idle_time keep that many free virtual machines for that long.

    The Role lets the plugin read and patch the pool and read and delete its virtual machines. The init container copies config.toml from the Secret into a writable directory, where the runner also keeps its system ID. defaultMode: 0440 lets the gitlab-runner group, which fsGroup assigns to the key file, read the key.

  2. Apply the manifest:

    d8 k apply -f code-runner-dvp.yaml

  3. Check the connection to Deckhouse Code:

    d8 k -n code-runner-dvp exec deploy/code-runner -- gitlab-runner verify

A runner outside the cluster reaches the pool with a kubeconfig file named in kubeconfig_path; its user needs the rights of the Role above.

A restarted runner replaces the virtual machines of the pool with new ones. To have it take over the running virtual machines, enable the state storage of the autoscaler in a directory on a persistent volume mounted into the runner container:

  [runners.autoscaler.state_storage]
    enabled = true
    dir = "/home/gitlab-runner/state"
    keep_instance_with_acquisitions = true

Docker executor

The Docker executor runs every job in a container of the Docker daemon on the runner host. The runner itself runs in a container that reaches the daemon through its socket.

  1. Create a volume for the runner configuration:

    docker volume create code-runner-config

  2. Register the runner, which writes config.toml to the volume:

    docker run --rm -v code-runner-config:/etc/gitlab-runner \
  registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner:<VERSION> \
  register --non-interactive \
    --url https://code.example.com \
    --token <TOKEN> \
    --executor docker \
    --docker-image alpine:3 \
    --docker-helper-image registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner-helper:<VERSION>

    --docker-image is the image of the jobs whose .gitlab-ci.yml names no image.

  3. Start the runner:

    docker run -d --name code-runner --restart always \
  --group-add "$(stat -c %g /var/run/docker.sock)" \
  -v code-runner-config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner:<VERSION>

    The runner runs as the gitlab-runner user; --group-add adds it to the group that owns the Docker socket.

  4. Check the connection to Deckhouse Code:

    docker exec code-runner gitlab-runner verify

Kubernetes executor

The Kubernetes executor runs every job in a pod of its own. The runner itself runs as a Deployment in the same cluster and creates the job pods in the namespace named in config.toml.

  1. Create the namespace:

    d8 k create namespace code-runner

  2. Save the following manifest as code-runner.yaml. It holds the runner configuration, the service account with the rights to manage the job pods and the Deployment of the runner:

    apiVersion: v1
kind: Secret
metadata:
  name: code-runner-config
  namespace: code-runner
stringData:
  config.toml: |
    concurrent = 4

    [[runners]]
      name = "kubernetes"
      url = "https://code.example.com"
      token = "<TOKEN>"
      executor = "kubernetes"
      [runners.kubernetes]
        namespace = "code-runner"
        image = "alpine:3"
        helper_image = "registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner-helper:<VERSION>"
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: code-runner
  namespace: code-runner
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: code-runner
  namespace: code-runner
rules:
  - apiGroups: [""]
    resources: ["pods", "pods/exec", "pods/attach", "pods/log", "secrets", "configmaps", "services", "events"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: code-runner
  namespace: code-runner
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: code-runner
subjects:
  - kind: ServiceAccount
    name: code-runner
    namespace: code-runner
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: code-runner
  namespace: code-runner
spec:
  replicas: 1
  selector:
    matchLabels:
      app: code-runner
  template:
    metadata:
      labels:
        app: code-runner
    spec:
      serviceAccountName: code-runner
      securityContext:
        runAsUser: 1001
        runAsGroup: 1001
        fsGroup: 1001
      initContainers:
        - name: config
          image: registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner:<VERSION>
          command: ["cp", "/config-secret/config.toml", "/etc/gitlab-runner/config.toml"]
          volumeMounts:
            - name: config-secret
              mountPath: /config-secret
            - name: config
              mountPath: /etc/gitlab-runner
      containers:
        - name: runner
          image: registry.deckhouse.ru/deckhouse/ee/packages/code/images/runner:<VERSION>
          volumeMounts:
            - name: config
              mountPath: /etc/gitlab-runner
      volumes:
        - name: config-secret
          secret:
            secretName: code-runner-config
        - name: config
          emptyDir: {}

    The init container copies config.toml from the Secret into a writable directory, where the runner also keeps its system ID.

  3. Apply the manifest:

    d8 k apply -f code-runner.yaml

  4. Check the connection to Deckhouse Code:

    d8 k -n code-runner exec deploy/code-runner -- gitlab-runner verify
←
Administration / Upgrade
Usage / Overview
→