The csi-hpe module: FAQ

What are the requirements for the csi-hpe module?

The csi-hpe module requires:

• Presence of a deployed and configured HPE SAN.
• Unique iqn in /etc/iscsi/initiatorname.iscsi on each of Kubernetes Nodes
• The snapshot-controller module must be enabled in the cluster (required for volume snapshot functionality).

How to check module health?

To do this, you need to check the status of the pods in the d8-csi-hpe namespace. All pods should be in the Running or Completed state and should be running on all nodes.

kubectl -n d8-csi-hpe get pod -owide -w

How much of a volume is reserved for the superuser?

Nothing, by default: the module tells the driver to format ext4 with -m0, so the whole volume is available to the workload.

Where the classic 5% ext4 reserve is wanted — for example to keep a filesystem writable for a privileged process after a workload has filled it — annotate the HPEStorageClass with the percentage to reserve:

d8 k annotate hpestorageclass <name> storage.deckhouse.io/ext4-reserved-percent=5

The value is a whole number of percent between 0 and 50; an invalid one leaves the HPEStorageClass with Ready=False and the reason in its status, instead of breaking volume creation later.

The reserve applies only to volumes created after the annotation was set — filesystems that already exist keep the reserve they were created with. A class with fsType: xfs is unaffected: XFS keeps no blocks for the superuser.

Changing the annotation makes the controller recreate the StorageClass, because parameters of an existing StorageClass are immutable in Kubernetes. Existing volumes and PVCs are not affected.

Where do open-iscsi and multipath-tools on a node come from?

From the module itself. The NodeGroupConfiguration never asks the node’s package manager for them, so a node in a closed environment with no route to the distribution’s repositories is set up the same way as any other, and no bashible run waits on those repositories.

On every node it serves, bashible pulls the module’s package image iscsi-tools from the module’s registry, unpacks it and runs its install script. The stack is two pairs — iscsiadm with iscsid, and multipath with multipathd — and the script decides for each pair separately:

• a pair the host already has of its own stays the host’s, and the module only enables and configures the distribution’s unit for its daemon;
• a pair the host lacks lands under /var/lib/deckhouse/sds/csi-hpe, every binary is started through its own dynamic loader with its own library path, and a unit of the module’s comes up for its daemon.

A client and its daemon always come from the same source, because iscsiadm of one version does not speak to iscsid of another. The two pairs do not talk to each other, so a node may well run the host’s iscsid next to the module’s multipathd.

Two things the install script arranges for the module’s pairs are worth knowing about, because both are invisible until something does not mount.

The driver reaches the host’s iscsiadm through the container’s /bin/iscsiadm, which is the in-tree host-iscsiadm binary: it chroots into /host and looks the client up there, against a PATH of its own. When iscsiadm is the module’s, the script leaves a wrapper at /usr/local/sbin/iscsiadm, which that PATH searches ahead of the distribution’s directories, and the wrapper defers to a distribution iscsiadm the moment one appears.

And multipathd from the package reads its configuration under the prefix it was built with, not under /etc. When multipathd is the module’s, the script links multipath.conf and multipath from that prefix onto the node’s own. That matters more here than anywhere: the driver locates a staged LUN only through its device-mapper map, so the find_multipaths no this module writes into /etc/multipath/conf.d — on every node, whichever multipathd runs there — is the difference between a working attach and device not found with serial <wwn>.

To tell which pair a node took from where, look at what is running on it:

# the distribution's daemons
systemctl is-active iscsid multipathd
# the module's daemons and binaries
systemctl is-active d8-csi-hpe-iscsid.service d8-csi-hpe-multipathd.service
ls /var/lib/deckhouse/sds/csi-hpe/bin
multipathd show config | grep find_multipaths

The module’s stack changes little on the host itself. Its libraries are never merged into the host’s /lib64 or /usr/lib; the only files it puts outside its own directory are the iscsiadm wrapper, the configuration links and the multipath path checkers in /usr/lib/multipath, where multipathd looks for them — each only for a pair that is the module’s, only where the host has no file of its own, and taken back once that pair becomes the host’s. And it does not touch /etc/iscsi/initiatorname.iscsi when the node already has one: the IQN is the node’s identity on the array, registered there in a host object, and a node that comes back under a different name is a node the array has never heard of.

sg3-utils is not installed at all, and nothing is lost by that: the driver’s only call into it was rescan-scsi-bus.sh, which this module’s patches replaced with direct sysfs writes.