Module csi-mind-ustor

The csi-mind-ustor module lets cluster workloads store data on uStor volumes. The volumes are attached to the nodes over NBD (Network Block Device).

The connection to the uStor cluster is described by a MindUstorStorageConnection custom resource, and each StorageClass of the driver by a MindUstorStorageClass custom resource. The module installs the uStor client on the nodes and keeps its configuration in line with the connection.

Main features

The module provides the following features:

• StorageClasses for uStor pools, described by MindUstorStorageClass resources
• Persistent volumes in uStor, attached to the nodes over NBD
• Online volume expansion, without restarting the pod
• Volumes formatted with ext4 or xfs, and raw Block volumes
• Volume snapshots and restoring volumes from them
• ReadWriteMany Block volumes for the live migration of virtual machines of the virtualization module
• Automatic installation and configuration of the uStor client (ustor-client, ustor-nbd) on the nodes
• Volume usage metrics for the alerts and dashboards of PersistentVolumeClaims
• A regular check of the login to the uStor management API, and alerts when a connection or a StorageClass is not ready

How the module works

The module works as follows:

1. When the module is enabled, it installs the uStor client on all nodes of the cluster with a supported OS and labels these nodes storage.deckhouse.io/csi-mind-ustor-client-ready=true. The CSI node plugin runs only on the labeled nodes.
2. The administrator creates Secrets with the uStor user credentials and the client certificates, and a MindUstorStorageConnection that refers to them.
3. The module writes the uStor client configuration and the certificates to /etc/ustor on the nodes selected by dataPlane.nodeSelector of the connection, keeps the CSI node plugin only on them and starts the CSI controller. The module logs in to the uStor management API with the credentials of the connection every hour.
4. The administrator creates a MindUstorStorageClass. The module creates a StorageClass with the same name for the ustor-nbd.csi.mindsw.io driver.
5. When a PersistentVolumeClaim with this StorageClass is created, the driver creates an image in the uStor pool. A pod that uses the volume is scheduled only on a node where the CSI node plugin runs. When the pod starts, the volume is attached to its node over NBD and mounted.

StorageClasses of the ustor-nbd.csi.mindsw.io driver can be created, changed and deleted only through MindUstorStorageClass. Only their annotations can be changed directly.

System requirements

The module has the following requirements:

• Deckhouse Platform (DP) 1.72 or later.
• A uStor 1.6 cluster whose management API is reachable from the Kubernetes cluster.
• The nodes that mount uStor volumes reach the uStor etcd and OSDs over the uStor data network.
• The nbd kernel module can be loaded on these nodes.
• The nodes run Astra Linux 1.8, RED OS 8, Ubuntu 22.04 or 24.04, or Debian 12. uStor volumes cannot be mounted on nodes with other operating systems.
• A dedicated uStor user for the CSI controller, prepared as described in Preparing the uStor user.
• For volume snapshots, a module that provides the snapshot.storage.k8s.io API, for example snapshot-controller.

Limitations

The module has the following limitations:

• The module serves one uStor cluster: only the oldest MindUstorStorageConnection is used.
• Filesystem volumes support only the ReadWriteOnce and ReadWriteOncePod access modes.
• Block volumes support only the ReadWriteOncePod access mode. ReadWriteMany is allowed only for the virtual machine disks of the virtualization module.
• Cloning a volume directly from another PersistentVolumeClaim is not supported: take a snapshot and restore it.
• CSI ephemeral inline volumes are not supported.
• The parameters of a MindUstorStorageClass cannot be changed after it is created.