The csi-yadro-tatlin-unified module

The module installs and configures the CSI driver for SAN TATLIN.UNIFIED. The module allows you to create a StorageClass in Kubernetes by creating Kubernetes custom resources YadroTatlinUnifiedStorageClass.

Caution! The user is not allowed to create a StorageClass for the csi-tatlinunified.yadro.com CSI driver.

System requirements and recommendations

Requirements

• Presence of a deployed and configured TATLIN.UNIFIED SAN.
• Unique iqn in /etc/iscsi/initiatorname.iscsi on each of Kubernetes Nodes

Quickstart guide

Note that all commands must be run on a machine that has administrator access to the Kubernetes API.

Enabling module

• Enable the csi-yadro-tatlin-unified module. This will result in the following actions across all cluster nodes:
  • registration of the CSI driver;
  • launch of service pods for the csi-yadro-tatlin-unified components.

kubectl apply -f - <<EOF
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: csi-yadro-tatlin-unified
spec:
  enabled: true
  version: 1
EOF

• Wait for the module to become Ready.

kubectl get module csi-yadro-tatlin-unified -w

Creating a StorageClass

To create a StorageClass, you need to use the YadroTatlinUnifiedStorageClass and YadroTatlinUnifiedStorageConnection resource. Here is an example command to create such a resource:

kubectl apply -f -<<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: YadroTatlinUnifiedStorageConnection
metadata:
  name: yad1
spec:
  controlPlane:
    address: "172.19.28.184"
    username: "admin"
    password: "cGFzc3dvcmQ=" # MUST BE BASE64 ENCODED
    ca: "base64encoded"
    skipCertificateValidation: true
  dataPlane:
    protocol: "iscsi"
    iscsi:
      volumeExportPort: "p50,p51,p60,p61"
EOF

To use NVMe-over-TCP (requires Tatlin Unified Gen2 firmware 3.2.0 or newer and a pre-created access group of type nvme on the array, e.g. tatlin-cli accessgroup create --name nvme_group --type nvme), set dataPlane.protocol to nvme-tcp and supply the access group name:

kubectl apply -f -<<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: YadroTatlinUnifiedStorageConnection
metadata:
  name: yad-nvme
spec:
  controlPlane:
    address: "172.19.28.184"
    username: "admin"
    password: "cGFzc3dvcmQ=" # MUST BE BASE64 ENCODED
    ca: "base64encoded"
    skipCertificateValidation: true
  dataPlane:
    protocol: "nvme-tcp"
    nvmeTcp:
      volumeExportPort: "p40,p41"
      accessGroupName: "nvme_group"
EOF

When at least one YadroTatlinUnifiedStorageConnection uses nvme-tcp, the module additionally installs nvme-cli, loads the nvme-tcp kernel module, enables options nvme_core multipath=Y and writes a TATLIN-specific iopolicy udev rule on every CSI data node.

The nvme-tcp kernel module itself must come with the node’s kernel: it is built for one exact kernel, so the module cannot ship it. Most distributions include it in the kernel package, but Ubuntu’s generic kernels carry it in linux-modules-extra-$(uname -r), which a minimal Ubuntu image may lack. On a node without it bashible logs The kernel ... has no nvme-tcp module and NVMe-TCP volumes fail to attach there with NVMe discovery failed; install that package on the node (or in the node image), and the next bashible run loads the module.

Every host of an nvme access group sees every namespace of the group: the volumes of the other nodes and, if the group is shared, of other clusters too. So that sds-node-configurator does not offer them as free disks, the module keeps the BlockDeviceFilter csi-yadro-tatlin-unified-hide-tatlin, which hides every block device of model TATLIN.UNIFIED from it, whatever the protocol. The filter is created once the BlockDeviceFilter resource is available in the cluster; a changed selector is put back within a minute.

Hosts in access groups will be created automatically, but you must add nodes nqn to them. On each node execute

nvme show-hostnqn

And add it to corresponding host. Or you can create hosts with nqn manually according docs

kubectl apply -f -<<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: YadroTatlinUnifiedStorageClass
metadata:
  name: yad1
spec:
  fsType: "xfs"
  pool: "pool-hdd"
  storageConnectionName: "yad1"
  reclaimPolicy: Delete
EOF

To cap the IOPS or throughput of the class’s volumes, add spec.qos — see the FAQ.

• You can check objects creation (READY must be True; PHASE is a coarse summary of the same verdict, kept for compatibility):

kubectl get yadrotatlinunifiedstorageconnections.storage.deckhouse.io <yadrotatlinunifiedstorageconnection name>

kubectl get yadrotatlinunifiedstorageclasses.storage.deckhouse.io <yadrotatlinunifiedstorageclass name>

NAME         PHASE     READY   AGE
yadro-fast   Created   True    5m

An empty READY means the controller has not produced a verdict yet, which is not the same as a failure. Add -o wide for the REASON column, or read the condition message when READY is False:

kubectl get yadrotatlinunifiedstorageclasses.storage.deckhouse.io <yadrotatlinunifiedstorageclass name> \
  -o jsonpath='{.status.conditions[?(@.type=="Ready")].message}'

• To confirm the controller has acted on your latest edit rather than on the previous one, compare status.observedGeneration with metadata.generation — while the first trails the second, the status still describes the previous spec:

kubectl get yadrotatlinunifiedstorageclasses.storage.deckhouse.io <yadrotatlinunifiedstorageclass name> \
  -o jsonpath='{.metadata.generation}{" -> "}{.status.observedGeneration}{"\n"}'

Checking module health

You can verify the functionality of the module using the instructions here