The module lifecycle stage: Generally available version

The module has requirements for installation

Working with virtual machines

Installing and configuring the operating system

How to install an operating system in a virtual machine from an ISO image?

Below is a typical Windows guest OS installation scenario from an ISO image. Before you begin, host the ISO on an HTTP endpoint reachable from the cluster.

  1. Create an empty VirtualDisk for OS installation:

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualDisk
metadata:
  name: win-disk
  namespace: default
spec:
  persistentVolumeClaim:
    size: 100Gi
    storageClassName: local-path

  2. Create ClusterVirtualImage resources for the Windows OS ISO and the VirtIO driver ISO:

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: ClusterVirtualImage
metadata:
  name: win-11-iso
spec:
  dataSource:
    type: HTTP
    http:
      url: "http://example.com/win11.iso"
    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: ClusterVirtualImage
metadata:
  name: win-virtio-iso
spec:
  dataSource:
    type: HTTP
    http:
      url: "https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso"

  3. Create a virtual machine (VM):

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualMachine
metadata:
  name: win-vm
  namespace: default
  labels:
    vm: win
spec:
  virtualMachineClassName: generic
  runPolicy: Manual
  osType: Windows
  bootloader: EFI
  cpu:
    cores: 6
    coreFraction: 50%
  memory:
    size: 8Gi
  enableParavirtualization: true
  blockDeviceRefs:
    - kind: VirtualDisk
      name: win-disk
    - kind: ClusterVirtualImage
      name: win-11-iso
    - kind: ClusterVirtualImage
      name: win-virtio-iso

  4. Start the virtual machine:

    d8 v start win-vm

  5. Connect to the VM console and complete the OS installation and VirtIO drivers using the graphical installer.

    VNC connection:

    d8 v vnc -n default win-vm

  6. After the installation is complete, restart the virtual machine.

  7. For further work, connect via VNC again:

    d8 v vnc -n default win-vm

How to provide a Windows answer file (Sysprep)?

Unattended Windows installation uses an answer file (unattend.xml or autounattend.xml).

The example answer file below:

  • Sets the English UI language and keyboard layout.
  • Connects the VirtIO drivers for the setup stage (the order of devices in blockDeviceRefs on the VirtualMachine resource must match the paths in the file).
  • Creates disk layout for installation with EFI.
  • Creates the cloud administrator and the regular user account.
Example of the contents of the autounattend.xml file… 
<?xml version="1.0" encoding="utf-8"?>
<unattend xmlns="urn:schemas-microsoft-com:unattend" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State">
  <settings pass="offlineServicing"></settings>
  <settings pass="windowsPE">
    <component name="Microsoft-Windows-International-Core-WinPE" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
      <SetupUILanguage>
        <UILanguage>en-US</UILanguage>
      </SetupUILanguage>
      <InputLocale>0409:00000409</InputLocale>
      <SystemLocale>en-US</SystemLocale>
      <UILanguage>en-US</UILanguage>
      <UserLocale>en-US</UserLocale>
    </component>
    <component name="Microsoft-Windows-PnpCustomizationsWinPE" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
      <DriverPaths>
        <PathAndCredentials wcm:keyValue="4b29ba63" wcm:action="add">
          <Path>E:\amd64\w11</Path>
        </PathAndCredentials>
        <PathAndCredentials wcm:keyValue="25fe51ea" wcm:action="add">
          <Path>E:\NetKVM\w11\amd64</Path>
        </PathAndCredentials>
      </DriverPaths>
    </component>
    <component name="Microsoft-Windows-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
      <DiskConfiguration>
        <Disk wcm:action="add">
          <DiskID>0</DiskID>
          <WillWipeDisk>true</WillWipeDisk>
          <CreatePartitions>
            <!-- Recovery partition -->
            <CreatePartition wcm:action="add">
              <Order>1</Order>
              <Type>Primary</Type>
              <Size>250</Size>
            </CreatePartition>
            <!-- EFI system partition (ESP) -->
            <CreatePartition wcm:action="add">
              <Order>2</Order>
              <Type>EFI</Type>
              <Size>100</Size>
            </CreatePartition>
            <!-- Microsoft reserved partition (MSR) -->
            <CreatePartition wcm:action="add">
              <Order>3</Order>
              <Type>MSR</Type>
              <Size>128</Size>
            </CreatePartition>
            <!-- Windows partition -->
            <CreatePartition wcm:action="add">
              <Order>4</Order>
              <Type>Primary</Type>
              <Extend>true</Extend>
            </CreatePartition>
          </CreatePartitions>
          <ModifyPartitions>
            <!-- Recovery partition -->
            <ModifyPartition wcm:action="add">
              <Order>1</Order>
              <PartitionID>1</PartitionID>
              <Label>Recovery</Label>
              <Format>NTFS</Format>
              <TypeID>de94bba4-06d1-4d40-a16a-bfd50179d6ac</TypeID>
            </ModifyPartition>
            <!-- EFI system partition (ESP) -->
            <ModifyPartition wcm:action="add">
              <Order>2</Order>
              <PartitionID>2</PartitionID>
              <Label>System</Label>
              <Format>FAT32</Format>
            </ModifyPartition>
            <!-- MSR partition does not need to be modified -->
            <!-- Windows partition -->
            <ModifyPartition wcm:action="add">
              <Order>3</Order>
              <PartitionID>4</PartitionID>
              <Label>Windows</Label>
              <Letter>C</Letter>
              <Format>NTFS</Format>
            </ModifyPartition>
          </ModifyPartitions>
        </Disk>
        <WillShowUI>OnError</WillShowUI>
      </DiskConfiguration>
      <ImageInstall>
        <OSImage>
          <InstallTo>
            <DiskID>0</DiskID>
            <PartitionID>4</PartitionID>
          </InstallTo>
        </OSImage>
      </ImageInstall>
      <UserData>
        <ProductKey>
          <Key><PRODUCT_KEY></Key>
          <WillShowUI>OnError</WillShowUI>
        </ProductKey>
        <AcceptEula>true</AcceptEula>
      </UserData>
      <UseConfigurationSet>false</UseConfigurationSet>
    </component>
  </settings>
  <settings pass="generalize"></settings>
  <settings pass="specialize">
    <component name="Microsoft-Windows-Deployment" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
      <RunSynchronous>
        <RunSynchronousCommand wcm:action="add">
          <Order>1</Order>
          <Path>powershell.exe -NoProfile -Command "$xml = [xml]::new(); $xml.Load('C:\Windows\Panther\unattend.xml'); $sb = [scriptblock]::Create( $xml.unattend.Extensions.ExtractScript ); Invoke-Command -ScriptBlock $sb -ArgumentList $xml;"</Path>
        </RunSynchronousCommand>
        <RunSynchronousCommand wcm:action="add">
          <Order>2</Order>
          <Path>powershell.exe -NoProfile -Command "Get-Content -LiteralPath 'C:\Windows\Setup\Scripts\Specialize.ps1' -Raw | Invoke-Expression;"</Path>
        </RunSynchronousCommand>
        <RunSynchronousCommand wcm:action="add">
          <Order>3</Order>
          <Path>reg.exe load "HKU\DefaultUser" "C:\Users\Default\NTUSER.DAT"</Path>
        </RunSynchronousCommand>
        <RunSynchronousCommand wcm:action="add">
          <Order>4</Order>
          <Path>powershell.exe -NoProfile -Command "Get-Content -LiteralPath 'C:\Windows\Setup\Scripts\DefaultUser.ps1' -Raw | Invoke-Expression;"</Path>
        </RunSynchronousCommand>
        <RunSynchronousCommand wcm:action="add">
          <Order>5</Order>
          <Path>reg.exe unload "HKU\DefaultUser"</Path>
        </RunSynchronousCommand>
      </RunSynchronous>
    </component>
  </settings>
  <settings pass="auditSystem"></settings>
  <settings pass="auditUser"></settings>
  <settings pass="oobeSystem">
    <component name="Microsoft-Windows-International-Core" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
      <InputLocale>0409:00000409</InputLocale>
      <SystemLocale>en-US</SystemLocale>
      <UILanguage>en-US</UILanguage>
      <UserLocale>en-US</UserLocale>
    </component>
    <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS">
      <UserAccounts>
        <LocalAccounts>
          <LocalAccount wcm:action="add">
            <Name>cloud</Name>
            <DisplayName>cloud</DisplayName>
            <Group>Administrators</Group>
            <Password>
              <Value><ADMIN_PASSWORD></Value>
              <PlainText>true</PlainText>
            </Password>
          </LocalAccount>
          <LocalAccount wcm:action="add">
            <Name>User</Name>
            <DisplayName>user</DisplayName>
            <Group>Users</Group>
            <Password>
              <Value><USER_PASSWORD></Value>
              <PlainText>true</PlainText>
            </Password>
          </LocalAccount>
        </LocalAccounts>
      </UserAccounts>
      <AutoLogon>
        <Username>cloud</Username>
        <Enabled>true</Enabled>
        <LogonCount>1</LogonCount>
        <Password>
          <Value><ADMIN_PASSWORD></Value>
          <PlainText>true</PlainText>
        </Password>
      </AutoLogon>
      <OOBE>
        <ProtectYourPC>3</ProtectYourPC>
        <HideEULAPage>true</HideEULAPage>
        <HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE>
        <HideOnlineAccountScreens>false</HideOnlineAccountScreens>
      </OOBE>
      <FirstLogonCommands>
        <SynchronousCommand wcm:action="add">
          <Order>1</Order>
          <CommandLine>powershell.exe -NoProfile -Command "Get-Content -LiteralPath 'C:\Windows\Setup\Scripts\FirstLogon.ps1' -Raw | Invoke-Expression;"</CommandLine>
        </SynchronousCommand>
      </FirstLogonCommands>
    </component>
  </settings>
</unattend>

Replace <PRODUCT_KEY> with your Windows product key, and <ADMIN_PASSWORD> and <USER_PASSWORD> with the passwords of the accounts being created. Windows reads these passwords from the file in plain text, so don’t leave the finished answer file on a shared resource.

  1. Save the answer file as autounattend.xml (use the example above or adjust it to your needs).

  2. Create a secret with the type provisioning.virtualization.deckhouse.io/sysprep:

    d8 k create secret generic sysprep-config --type="provisioning.virtualization.deckhouse.io/sysprep" --from-file=./autounattend.xml

  3. Create a virtual machine that will use the answer file during installation. Specify provisioning with type SysprepRef in the specification. If necessary, add other Base64-encoded files to the specification required for the answer file scripts to run successfully.

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualMachine
metadata:
  name: win-vm
  namespace: default
  labels:
    vm: win
spec:
  virtualMachineClassName: generic
  provisioning:
    type: SysprepRef
    sysprepRef:
      kind: Secret
      name: sysprep-config
  runPolicy: AlwaysOn
  osType: Windows
  bootloader: EFI
  cpu:
    cores: 6
    coreFraction: 50%
  memory:
    size: 8Gi
  enableParavirtualization: true
  blockDeviceRefs:
    - kind: VirtualDisk
      name: win-disk
    - kind: ClusterVirtualImage
      name: win-11-iso
    - kind: ClusterVirtualImage
      name: win-virtio-iso

How to create a golden image for Linux?

A golden image is a pre-configured virtual machine (VM) image that can be used to quickly create new VMs with pre-installed software and settings.

  1. Create a virtual machine, install the required software on it, and perform all necessary configurations.

  2. Install and configure qemu-guest-agent (recommended):

    • For RHEL/CentOS:

      yum install -y qemu-guest-agent

    • For Debian/Ubuntu:

      apt-get update
apt-get install -y qemu-guest-agent

  3. Enable and start the service:

    systemctl enable qemu-guest-agent
systemctl start qemu-guest-agent

  4. Set the machine run policy to AlwaysOnUnlessStoppedManually, otherwise you will not be able to shut it down.

  5. Prepare the image. Clean unused filesystem blocks:

    fstrim -v /
fstrim -v /boot

  6. Clean network settings:

    • For RHEL:

      nmcli con delete $(nmcli -t -f NAME,DEVICE con show | grep -v ^lo: | cut -d: -f1)
rm -f /etc/sysconfig/network-scripts/ifcfg-eth*

    • For Debian/Ubuntu:

      rm -f /etc/network/interfaces.d/*

  7. Clean system identifiers:

    echo -n > /etc/machine-id
rm -f /var/lib/dbus/machine-id
ln -s /etc/machine-id /var/lib/dbus/machine-id

  8. Remove the SSH host keys:

    rm -f /etc/ssh/ssh_host_*

  9. Clean the systemd journal:

    journalctl --vacuum-size=100M --vacuum-time=7d

  10. Clean package manager cache:

    • For RHEL:

      yum clean all

    • For Debian/Ubuntu:

      apt-get clean

  11. Clean temporary files:

    rm -rf /tmp/*
rm -rf /var/tmp/*

  12. Clean logs:

    find /var/log -name "*.log" -type f -exec truncate -s 0 {} \;

  13. Clean command history:

    history -c

  14. On RHEL, reset and restore the SELinux contexts in one of two ways.

    Restore the contexts right away:

    restorecon -R /

    Or schedule a relabel for the next boot:

    touch /.autorelabel

  15. Verify that /etc/fstab references UUID or LABEL rather than names like /dev/sdX:

    blkid
cat /etc/fstab

  16. Reset cloud-init state (logs and seed):

    cloud-init clean --logs --seed

  17. Perform final synchronization and buffer cleanup:

    sync
echo 3 > /proc/sys/vm/drop_caches

  18. Shut down the virtual machine:

    poweroff

  19. Create a VirtualImage resource that references the prepared VM’s VirtualDisk:

    d8 k apply -f -<<EOF
apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualImage
metadata:
  name: <IMAGE_NAME>
  namespace: <NAMESPACE>
spec:
  dataSource:
    type: ObjectRef
    objectRef:
      kind: VirtualDisk
      name: <SOURCE_DISK_NAME>
EOF

    Or create a ClusterVirtualImage resource so the image is available cluster-wide for all projects:

    d8 k apply -f -<<EOF
apiVersion: virtualization.deckhouse.io/v1alpha2
kind: ClusterVirtualImage
metadata:
  name: <IMAGE_NAME>
spec:
  dataSource:
    type: ObjectRef
    objectRef:
      kind: VirtualDisk
      name: <SOURCE_DISK_NAME>
      namespace: <NAMESPACE>
EOF

    Here, <IMAGE_NAME> is the name of the image being created, <NAMESPACE> is the namespace of the prepared machine, and <SOURCE_DISK_NAME> is the name of its disk.

  20. Create a new VirtualDisk from the resulting image:

    d8 k apply -f -<<EOF
apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualDisk
metadata:
  name: <VM_DISK_NAME>
  namespace: <NAMESPACE>
spec:
  dataSource:
    type: ObjectRef
    objectRef:
      kind: VirtualImage
      name: <IMAGE_NAME>
EOF

    Here, <VM_DISK_NAME> is the name of the disk of the new machine.

After completing these steps, you will have a golden image that can be used to quickly create new virtual machines with pre-installed software and configurations.

Connecting to a virtual machine

You can connect to a virtual machine (VM) via the serial console (d8 v console) or VNC (d8 v vnc). These methods use different communication channels with the guest OS and depend on its configuration. Both methods are covered in Connecting to a virtual machine.

The sections below describe common situations where only one connection method works.

Why does VNC not work when the serial console is available?

VNC displays the guest OS screen and requires virtual terminal support in the kernel. The serial console works independently of the graphics subsystem.

Check in the guest OS whether virtual terminal support is enabled in the kernel configuration:

cat /boot/config-$(uname -r) | grep CONFIG_VT

The output should show CONFIG_VT=y:

CONFIG_VT=y

If the output shows CONFIG_VT is not set, rebuild the kernel with the option enabled or use an OS image with a suitable kernel configuration.

Why does the serial console not work when VNC is available?

The serial console connects to the ttyS0 port in the guest OS. If the getty service for this port is not running, d8 v console will not show a login prompt even though VNC continues to work.

In the guest OS, enable and start the serial-getty service for ttyS0:

sudo systemctl enable --now serial-getty@ttyS0.service

Then connect to the serial console again.

Configuring virtual machines

How to use cloud-init to configure virtual machines?

Cloud-init is used for initial guest OS configuration on first boot. The configuration is written in YAML and starts with the #cloud-config directive.

When using cloud images (for example, official distribution images), you must provide a cloud-init configuration. Without it, some distributions do not configure network connectivity, and the virtual machine becomes unreachable on the network, even if the main network (Main) is attached.

In addition, cloud images do not allow login by default — you must either add SSH keys for the default user or create a new user with SSH access. Otherwise, you will not be able to access the virtual machine.

Updating and installing packages

Example cloud-config for updating the system and installing packages from a list:

#cloud-config
# Update package lists.
package_update: true
# Upgrade installed packages to latest versions.
package_upgrade: true
# List of packages to install.
packages:
  - nginx
  - curl
  - htop
# Commands to run after package installation.
runcmd:
  - systemctl enable --now nginx.service

Creating a user

Example cloud-config for creating a local user with a password and SSH key:

#cloud-config
# List of users to create.
users:
    # Username.
  - name: cloud
    # Password hash.
    passwd: "<PASSWORD_HASH>"
    # Do not lock the account.
    lock_passwd: false
    # Sudo privileges without a password prompt.
    sudo: ALL=(ALL) NOPASSWD:ALL
    # Default shell.
    shell: /bin/bash
    # SSH keys for access.
    ssh-authorized-keys:
      - <SSH_PUBLIC_KEY>
# Allow password authentication via SSH.
ssh_pwauth: true

To generate a password hash for the passwd field, run:

mkpasswd --method=SHA-512 --rounds=4096

Creating a file with required permissions

Example cloud-config for creating a file with specified access permissions:

#cloud-config
# List of files to create.
write_files:
    # File path.
  - path: /opt/scripts/start.sh
    # File content.
    content: |
      #!/bin/bash
      echo "Starting application"
    # File owner, user and group.
    owner: cloud:cloud
    # Access permissions in octal format.
    permissions: '0755'

Configuring disk and filesystem

Example cloud-config for disk partitioning, filesystem creation, and mounting:

#cloud-config
# Disk partitioning setup.
disk_setup:
  # Disk device.
  /dev/sdb:
    # Partition table type, gpt or mbr.
    table_type: gpt
    # Automatically create partitions.
    layout: true
    # Do not overwrite existing partitions.
    overwrite: false

# Filesystem setup.
fs_setup:
    # Filesystem label.
  - label: data
    # Filesystem type.
    filesystem: ext4
    # Partition device.
    device: /dev/sdb1
    # Automatically detect partition.
    partition: auto

# Filesystem mounting.
mounts:
  # [device, mount_point, fs_type, options, dump, pass]
  - ["/dev/sdb1", "/mnt/data", "ext4", "defaults", "0", "2"]

Configuring network interfaces for additional networks

The settings described in this section apply only to additional networks. The main network (Main) is configured automatically via cloud-init and does not require manual configuration.

Additional networks are configured manually via cloud-init. The write_files block creates the configuration files, and the runcmd block applies the settings.

Connecting additional networks to a virtual machine is covered in Additional network interfaces.

The following examples cover common ways to configure networking in the guest OS:

  • systemd-networkd
  • Netplan (Ubuntu)
  • ifcfg (RHEL/CentOS)
  • Alpine Linux

Example cloud-config for distributions that use systemd-networkd (Debian, CoreOS, and others):

#cloud-config
write_files:
  - path: /etc/systemd/network/10-eth1.network
    content: |
      [Match]
      Name=eth1

      [Network]
      Address=192.168.1.10/24
      Gateway=192.168.1.1
      DNS=8.8.8.8

runcmd:
  - systemctl restart systemd-networkd

Example cloud-config for Ubuntu and other systems that use Netplan:

#cloud-config
write_files:
  - path: /etc/netplan/99-custom.yaml
    content: |
      network:
        version: 2
        ethernets:
          eth1:
            addresses:
              - 10.0.0.5/24
            gateway4: 10.0.0.1
            nameservers:
              addresses: [8.8.8.8]
          eth2:
            dhcp4: true

runcmd:
  - netplan apply

Example cloud-config for RHEL-compatible distributions that use the ifcfg scheme and NetworkManager:

#cloud-config
write_files:
  - path: /etc/sysconfig/network-scripts/ifcfg-eth1
    content: |
      DEVICE=eth1
      BOOTPROTO=none
      ONBOOT=yes
      IPADDR=192.168.1.10
      PREFIX=24
      GATEWAY=192.168.1.1
      DNS1=8.8.8.8

runcmd:
  - nmcli connection reload
  - nmcli connection up eth1

Example cloud-config for distributions that use the traditional /etc/network/interfaces format (Alpine and similar):

#cloud-config
write_files:
  - path: /etc/network/interfaces
    append: true
    content: |
      auto eth1
      iface eth1 inet static
          address 192.168.1.10
          netmask 255.255.255.0
          gateway 192.168.1.1

runcmd:
  - /etc/init.d/networking restart

How to use Ansible to provision virtual machines?

Ansible is an automation tool for running tasks on remote servers over SSH. This example shows how to use Ansible with virtual machines (VMs) in the demo-app project.

The example assumes that:

  • demo-app namespace contains a VM named frontend.
  • VM has a cloud user with SSH access.
  • Private SSH key on the machine where Ansible runs is stored in /home/user/.ssh/id_rsa.

  1. Create an inventory.yaml file:

    ---
all:
  vars:
    ansible_ssh_common_args: '-o ProxyCommand="d8 v port-forward --stdio=true %h %p"'
    # Default user for SSH access.
    ansible_user: cloud
    # Path to private key.
    ansible_ssh_private_key_file: /home/user/.ssh/id_rsa
  hosts:
    # Host name in the format <VM_NAME>.<NAMESPACE>.
    frontend.demo-app:

  2. Check the virtual machine uptime:

    ansible -m shell -a "uptime" -i inventory.yaml all

# frontend.demo-app | CHANGED | rc=0 >>
# 12:01:20 up 2 days,  4:59,  0 users,  load average: 0.00, 0.00, 0.00

If you do not want to use an inventory file, pass all parameters on the command line:

ansible -m shell -a "uptime" \
  -i "frontend.demo-app," \
  -e "ansible_ssh_common_args='-o ProxyCommand=\"d8 v port-forward --stdio=true %h %p\"'" \
  -e "ansible_user=cloud" \
  -e "ansible_ssh_private_key_file=/home/user/.ssh/id_rsa" \
  all

How to automatically generate inventory for Ansible?

The d8 v ansible-inventory command requires d8 v0.27.0 or higher.

The command works only for virtual machines that have the main cluster network (Main) connected.

Instead of manually creating an inventory file, you can use the d8 v ansible-inventory command, which automatically generates an Ansible inventory from virtual machines in the specified namespace. The command is compatible with the ansible inventory script interface.

Only machines in the Running phase that have an assigned IP address get into the inventory. Host names are formatted as <VM_NAME>.<NAMESPACE> (for example, frontend.demo-app).

  1. Optionally set host variables via annotations (for example, the SSH user):

    d8 k -n demo-app annotate vm frontend vars.ansible.deckhouse.io/ansible_user="cloud"

  2. Run Ansible with a dynamically generated inventory:

    ANSIBLE_INVENTORY_ENABLED=yaml ansible -m shell -a "uptime" all -i <(d8 v ansible-inventory -n demo-app -o yaml)

The <(...) construct is necessary because Ansible expects a file or script as the source of the host list. Simply specifying the command in quotes won’t work, because Ansible tries to execute the string as a script. The <(...) construct passes the command output as a file that Ansible can read.

  1. Or save the inventory to a file and run the check:

    d8 v ansible-inventory --list -o yaml -n demo-app > inventory.yaml
ansible -m shell -a "uptime" -i inventory.yaml all

How to redirect traffic to a virtual machine?

A virtual machine runs in a Kubernetes cluster, so traffic reaches it the same way it reaches any other workload. Routing is handled by the standard Kubernetes Service resource, which selects targets by labels.

  1. Create a service with the required settings.

    For example, consider a virtual machine with the label vm: frontend-0, an HTTP service exposed on ports 80 and 443, and SSH access on port 22:

    apiVersion: virtualization.deckhouse.io/v1alpha2
kind: VirtualMachine
metadata:
  name: frontend-0
  namespace: dev
  labels:
    vm: frontend-0
spec: ...

  2. To route network traffic to the virtual machine’s ports, create a service:

    This service listens on ports 80 and 443 and forwards traffic to the corresponding ports of the target virtual machine. SSH access from outside is provided on port 2211:

    apiVersion: v1
kind: Service
metadata:
  name: frontend-0-svc
  namespace: dev
spec:
  type: LoadBalancer
  ports:
  - name: ssh
    port: 2211
    protocol: TCP
    targetPort: 22
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    vm: frontend-0

Platform management

How to increase the DVCR size?

The DVCR volume size is set in the virtualization module ModuleConfig (spec.settings.dvcr.storage.persistentVolumeClaim.size). The new value must be greater than the current one.

  1. Check the current DVCR size:

    d8 k get mc virtualization -o jsonpath='{.spec.settings.dvcr.storage.persistentVolumeClaim}'

    Example output:

    {"size":"58G","storageClass":"linstor-thick-data-r1"}

  2. Increase size using patch (set the value you need):

    d8 k patch mc virtualization \
  --type merge -p '{"spec": {"settings": {"dvcr": {"storage": {"persistentVolumeClaim": {"size":"59G"}}}}}}'

    Example output:

    moduleconfig.deckhouse.io/virtualization patched

  3. Verify that ModuleConfig shows the new size:

    d8 k get mc virtualization -o jsonpath='{.spec.settings.dvcr.storage.persistentVolumeClaim}'

    Example output:

    {"size":"59G","storageClass":"linstor-thick-data-r1"}

  4. Check the current DVCR status:

    d8 k get pvc dvcr -n d8-virtualization

    Example output:

    NAME STATUS VOLUME                                    CAPACITY    ACCESS MODES   STORAGECLASS           AGE
dvcr Bound  pvc-6a6cedb8-1292-4440-b789-5cc9d15bbc6b  57617188Ki  RWO            linstor-thick-data-r1  7d

How to change the DVCR StorageClass when a PVC already exists?

You can change the DVCR storage StorageClass only by recreating the PVC. All images previously loaded into DVCR are lost, along with data for existing ClusterVirtualImage and VirtualImage resources.

The spec.settings.dvcr.storage.persistentVolumeClaim.storageClassName field in the virtualization module ModuleConfig sets the StorageClass for the virtual machine image storage volume (DVCR). While a PVC for that volume exists in the d8-virtualization namespace, you cannot change the field via the API.

You cannot change storageClassName on an existing PVC in place, and DVCR data is not migrated between storage classes.

To change the DVCR StorageClass, perform the following steps:

  1. Stop DVCR:

    d8 k -n d8-virtualization scale deployment dvcr --replicas=0

  2. List PVCs in the d8-virtualization namespace and find the PVC for the DVCR volume:

    d8 k get pvc -n d8-virtualization

  3. Delete the PVC you found. Replace <PVC_NAME> with the resource name. If the command fails because of insufficient permissions, run it as system:sudouser:

    d8 k --as system:sudouser -n d8-virtualization delete pvc/<PVC_NAME>

  4. Set the new StorageClass in ModuleConfig, substituting the class you need for <STORAGE_CLASS_NAME>:

    d8 k patch mc virtualization --type merge -p '{"spec":{"settings":{"dvcr":{"storage":{"persistentVolumeClaim":{"storageClassName":"<STORAGE_CLASS_NAME>"}}}}}}'

    Example output:

    moduleconfig.deckhouse.io/virtualization patched

  5. Start DVCR:

    d8 k -n d8-virtualization scale deployment dvcr --replicas=1

  6. Verify the PVC:

    d8 k get pvc -n d8-virtualization

    Example output:

    NAME   STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          VOLUMEATTRIBUTESCLASS   AGE
dvcr   Bound    pvc-b43f2e33-32cc-435a-aa1d-b53df35b030a   100Gi      RWO            linstor-thin-r1-hdd   <unset>                 34s

The storage for the chosen StorageClass must be reachable from the nodes where DVCR runs: system nodes, or worker nodes if the cluster has no system nodes.

How to restore the cluster if images from registry.deckhouse.io cannot be pulled after a license change?

After a license change on a cluster with containerd v1 and removal of the outdated license, images from registry.deckhouse.io may stop being pulled. Nodes then retain the outdated configuration file /etc/containerd/conf.d/dvcr.toml, which is not removed automatically. Because of it, the registry module does not start, and without it DVCR does not work.

Applying a NodeGroupConfiguration (NGC) manifest removes the file on the nodes. After the registry module starts, delete the manifest, since this is a one-time fix.

  1. Save the manifest to a file (for example, containerd-dvcr-remove-old-config.yaml):

    apiVersion: deckhouse.io/v1alpha1
kind: NodeGroupConfiguration
metadata:
  name: containerd-dvcr-remove-old-config.sh
spec:
  weight: 32 # Must be in range 32–90.
  nodeGroups: ["*"]
  bundles: ["*"]
  content: |
    # Copyright 2023 Flant JSC
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #      http://www.apache.org/licenses/LICENSE-2.0
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.

    rm -f /etc/containerd/conf.d/dvcr.toml

  2. Apply the saved manifest:

    d8 k apply -f containerd-dvcr-remove-old-config.yaml

  3. Verify that the registry module is running:

    d8 k -n d8-system -o yaml get secret registry-state | yq -C -P '.data | del .state | map_values(@base64d) | .conditions = (.conditions | from_yaml)'

    Example output when the registry module has started successfully:

    conditions:
# ...
  - lastTransitionTime: "..."
    message: ""
    reason: ""
    status: "True"
    type: Ready

  4. Delete the one-time NodeGroupConfiguration manifest:

    d8 k delete -f containerd-dvcr-remove-old-config.yaml

The migration procedure is covered in Migrating container runtime to containerd v2.