Custom resources

ElasticCluster

Short names: ec

Scope: Namespaced
Version: v1alpha1

• spec

  object

  Desired state of the Ceph cluster managed by the sds-elastic module.
  • spec.network

    object

    Optional network configuration. If unset, Rook listens on every host IP of storage nodes (host networking). When set, both public and cluster must be provided together (all-or-nothing pair).

    Ceph runs with network.provider: host, so both fields must list host subnets — every daemon (mon/mgr/osd) binds directly to a host interface and must own an IP inside one of the given ranges. The Pod network is not a valid value for either field. Each field accepts one or more CIDRs, so a range set can be given when storage nodes span several host subnets.

    Both fields are immutable after creation.

    • spec.network.cluster

      array of strings

      Required value

      One or more CIDRs of the cluster (host) network used only for OSD-to-OSD traffic (replication, backfill, recovery, heartbeat). Set it to a dedicated storage-NIC subnet, or to the same value as public when there is no separate one (for example, ["10.0.1.0/24"]).
      • spec.network.cluster.Element of the array

        string

        Length: 9..18

        Pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}\/[0-9]{1,2}$

    • spec.network.public

      array of strings

      Required value

      One or more CIDRs of the public (host) network carrying client traffic (Pods reaching mon/osd) plus all inter-daemon traffic except OSD replication (for example, ["10.0.0.0/24"]).
      • spec.network.public.Element of the array

        string

        Length: 9..18

        Pattern: ^([0-9]{1,3}\.){3}[0-9]{1,3}\/[0-9]{1,2}$

  • spec.storage

    object

    Required value

    OSD backing storage configuration. The controller selects matching BlockDevices on matching nodes and provisions one LVMVolumeGroup per device, one LVMLogicalVolume per group, and one local PersistentVolume per LVMLogicalVolume that backs a Rook OSD.
    • spec.storage.blockDeviceSelector

      object

      Required value

      Selector that matches BlockDevice CRs (managed by sds-node-configurator) eligible for OSD provisioning. Selected devices must reside on nodes that match storage.nodeSelector and must be unused (consumable).

      The field is required, so the key is always present: selecting every device is written as an explicit empty selector ({}), which stands for every consumable BlockDevice on the nodes matched by storage.nodeSelector. Either selector is re-evaluated on every reconcile, so with an empty one a disk that appears on a matching node later joins the cluster with no edit to this resource.

      • spec.storage.blockDeviceSelector.matchExpressions

        array of objects

        List of label selector requirements.
        • spec.storage.blockDeviceSelector.matchExpressions.key

          string
        • spec.storage.blockDeviceSelector.matchExpressions.operator

          string

          Allowed values: In, NotIn, Exists, DoesNotExist

        • spec.storage.blockDeviceSelector.matchExpressions.values

          array of strings
      • spec.storage.blockDeviceSelector.matchLabels

        object

        Map of label selectors.
    • spec.storage.encrypted

      boolean

      Turns on dm-crypt (LUKS2) underneath every OSD of the cluster: Rook hands each OSD block device to cryptsetup before ceph-volume touches it, so both the data and the Ceph metadata next to it are unreadable once the disk leaves the node. It protects a pulled or replaced disk, not a running cluster — anything able to read through a live OSD sees plaintext.

      Key encryption keys are stored in Kubernetes Secrets of the module namespace unless encryptionKeyStore points them at an external key manager. Keys in Secrets live in the same cluster as the data they unlock, and losing them makes the OSDs unrecoverable.

      Immutable. Rook encrypts an OSD only while creating it, so the flag cannot be flipped on an existing cluster — every OSD would have to be drained and re-created. Expect a measurable cost on the data path: every read and write of every OSD goes through dm-crypt.

      Default: false

    • spec.storage.encryptionKeyStore

      object

      Where the key encryption key of every OSD is kept. Unset means Kubernetes Secrets of the cluster’s namespace, which is Rook’s own default.

      Immutable, for the same reason encrypted is: Rook writes the key to the store it was told about while creating the OSD and never migrates keys between stores, so a swap on a live cluster would leave the existing OSDs unable to unlock.

      • spec.storage.encryptionKeyStore.stronghold

        object

        Coordinates of the Vault-API store. Required when type is Stronghold, rejected otherwise. The module never provisions anything inside Stronghold: the KV mount, its policy and the role below are created by whoever runs it.
        • spec.storage.encryptionKeyStore.stronghold.address

          string

          Required value

          Base URL of the Stronghold API, reachable from the storage nodes: Ceph daemons run with host networking, so the name has to resolve and route from the node, not only from the Pod network.

          Length: 8..253

          Pattern: ^https?://[a-zA-Z0-9._-]+(:[0-9]{1,5})?/?$

        • spec.storage.encryptionKeyStore.stronghold.authMountPath

          string

          Where the Kubernetes auth method is mounted. Deckhouse Stronghold mounts the one for its own cluster at kubernetes_local, which is why that, and not Vault’s kubernetes, is the default.

          Default: kubernetes_local

          Length: 1..128

          Pattern: ^[a-zA-Z0-9._-]+$

        • spec.storage.encryptionKeyStore.stronghold.caCertSecretName

          string

          Name of a Secret in the cluster’s namespace holding the CA bundle that signs the Stronghold certificate, under the key cert. Rook takes a Secret name here rather than a path or an inline PEM.

          Maximum length: 253

        • spec.storage.encryptionKeyStore.stronghold.insecureSkipTLSVerify

          boolean

          Turns off certificate verification towards Stronghold. It exists for test clusters with a self-signed certificate and no CA bundle to hand over; anything holding real data should ship the CA instead.
        • spec.storage.encryptionKeyStore.stronghold.kvPath

          string

          Required value

          Mount path of the KV v2 engine holding the keys.

          Length: 1..128

          Pattern: ^[a-zA-Z0-9._-]+$

        • spec.storage.encryptionKeyStore.stronghold.role

          string

          Required value

          Kubernetes-auth role Rook logs in as. It has to bind the rook-ceph-osd and rook-ceph-system service accounts of the cluster’s namespace and carry a policy over kvPath.

          Length: 1..128

      • spec.storage.encryptionKeyStore.type

        string

        Required value

        Picks the store.

        KubernetesSecret keeps a key per OSD in a Secret of the module namespace: no external dependency, but the keys share the cluster with the data they unlock and disappear with it.

        Stronghold keeps them in Deckhouse Stronghold (or any store serving the same Vault API) over its KV v2 engine. The key is read on every OSD start, not once at creation, so the store has to be reachable and unsealed whenever OSDs start — and it must not be backed by the very ElasticCluster it protects, or a cold start deadlocks.

        Default: KubernetesSecret

        Allowed values: KubernetesSecret, Stronghold

    • spec.storage.keyRotation

      object

      Asks Rook to replace the key encryption key of every OSD on a schedule. What changes is the key that unlocks the volume, not the key the data itself is encrypted with, so nothing is re-encrypted and the OSDs keep serving.

      Unlike encrypted and encryptionKeyStore this is mutable: rotation is an operational policy, and turning it on or off — or moving the schedule — has to be possible on a running cluster.

      • spec.storage.keyRotation.enabled

        boolean

        Required value

        Turns the rotation on. Requires encrypted: true — without encryption there are no OSD keys to rotate.

        Default: false

      • spec.storage.keyRotation.schedule

        string

        How often the keys are replaced: a cron expression (0 3 * * 0) or one of the usual descriptors (@weekly, @monthly). Unset leaves Rook’s own default, which is weekly.

        Length: 1..64

    • spec.storage.nodeSelector

      object

      Required value

      Selector that matches Kubernetes Nodes considered as storage nodes. Each matching node hosts mon/mgr/mds (as needed) and OSD daemons if it has eligible BlockDevices.

      The field is required, so the key is always present: selecting every Node is written as an explicit empty selector ({}), which matches the whole cluster, control-plane Nodes included.

      • spec.storage.nodeSelector.matchExpressions

        array of objects

        List of label selector requirements.
        • spec.storage.nodeSelector.matchExpressions.key

          string
        • spec.storage.nodeSelector.matchExpressions.operator

          string

          Allowed values: In, NotIn, Exists, DoesNotExist

        • spec.storage.nodeSelector.matchExpressions.values

          array of strings
      • spec.storage.nodeSelector.matchLabels

        object

        Map of label selectors.
• status

  object

  Current state of resources managed by ElasticCluster.
  • status.capacity

    object

    Cluster-wide raw storage usage as observed by Rook (CephCluster.status.ceph.capacity). Empty until Rook publishes the first capacity probe.
    • status.capacity.available

      integer or string

      Free raw capacity, as a Kubernetes Quantity (BinarySI). Usable client capacity depends on pool-level replication (~ available / pool.size for replicated pools).

      Pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$

    • status.capacity.lastUpdated

      string

      Timestamp of the latest capacity probe Rook successfully observed against the cluster.
    • status.capacity.total

      integer or string

      Total raw capacity of the cluster (sum over all OSDs), as a Kubernetes Quantity (BinarySI), e.g. 500Gi or 1.2Ti.

      Pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$

    • status.capacity.used

      integer or string

      Consumed raw capacity (data + metadata + Ceph overhead), as a Kubernetes Quantity (BinarySI), e.g. 200Gi.

      Pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$

    • status.capacity.usedPercent

      string

      used / total * 100, formatted with two decimals.
  • status.cephFSID

    string

    FSID/UUID of the deployed Ceph cluster, populated from the rook-ceph-mon Secret once Rook starts the cluster.
  • status.cephTopology

    object

    Effective Ceph daemon counts the controller has applied to the underlying CephCluster. Promotion is sticky (monotonic): once an ElasticStorageClass with replication: HighRedundancy raises the counts to the high-availability profile (monCount=5, mgrCount=3), the controller never lowers them back, even when the trigger ESC is removed. Operators can force a recompute by clearing this field via the status subresource.
    • status.cephTopology.lastPromotedAt

      string

      Timestamp of the most recent count increase. Empty until the cluster first leaves the standard profile.
    • status.cephTopology.mgrCount

      integer

      Required value

      Effective CephCluster.spec.mgr.count.

      Allowed values: 1 <= X

    • status.cephTopology.monCount

      integer

      Required value

      Effective CephCluster.spec.mon.count.

      Allowed values: 1 <= X

    • status.cephTopology.reason

      string

      Machine-readable explanation of the latest topology decision. Known values: Standard (defaults are in effect), HighRedundancyESCPresent (a HighRedundancy ESC currently demands the high-availability profile), StickyHighWaterMark (the recorded counts exceed what the live ESC inventory currently demands; the prior promotion is being preserved).
  • status.cephVersion

    object

    Ceph version state. The desired version is hardcoded in the module image; bumping the module triggers a rolling upgrade in Rook.
    • status.cephVersion.requested

      string

      Ceph version requested by the controller (matches the module image).
    • status.cephVersion.running

      string

      Ceph version currently running across the cluster, derived from CephCluster.status.ceph.versions.
    • status.cephVersion.upgradeProgress

      object

      Per-stage progress while a rolling upgrade is in flight.
      • status.cephVersion.upgradeProgress.completed

        integer

        Number of daemons in the current stage already running the requested version.

        Allowed values: 0 <= X

      • status.cephVersion.upgradeProgress.lastTransitionTime

        string

        Timestamp of the last upgrade-stage transition.
      • status.cephVersion.upgradeProgress.phase

        string

        Current upgrade stage.

        Allowed values: MonUpgrading, MgrUpgrading, MdsUpgrading, OsdUpgrading, Completed

      • status.cephVersion.upgradeProgress.total

        integer

        Total number of daemons in the current stage.

        Allowed values: 0 <= X

  • status.conditions

    array of objects

    Per-component conditions: StorageReady, CephClusterReady, CredentialsReady, CsiCephReady, UpgradeReady, UpgradeInProgress, and the aggregate Ready.
    • status.conditions.lastTransitionTime

      string

      Timestamp of the last status transition for this condition.
    • status.conditions.message

      string

      Human-readable explanation of the current status.

      Maximum length: 32768

    • status.conditions.observedGeneration

      integer

      Value of metadata.generation the condition was last set against.

      Allowed values: 0 <= X

    • status.conditions.reason

      string

      Machine-readable reason for the current status, suitable for log filtering.

      Length: 1..1024

      Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

    • status.conditions.status

      string

      Current status of the condition.

      Allowed values: True, False, Unknown

    • status.conditions.type

      string

      Condition type, for example Ready, StorageReady, or CephClusterReady.

      Maximum length: 316

      Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

  • status.credentialsRef

    object

    Reference to the ElasticClusterCredentials resource that backs up cluster identity (FSID, mon-secret, admin-secret) outside the cluster’s namespace. Named after the cluster; outside the module namespace the name is prefixed with the cluster’s namespace.
    • status.credentialsRef.name

      string

      Name of the ElasticClusterCredentials resource (1:1 with this ElasticCluster’s metadata.name).
  • status.dataDirHostPath

    string

    Host directory the Ceph daemons of this cluster keep their state in on every node (CephCluster.spec.dataDirHostPath): the mon store, the Ceph configuration and keyring the daemons read, and crash reports. There is no volume behind the monitors, so this directory is the monitor data.

    Set once, when the controller first renders the CephCluster, and read from here on every reconcile after that; it never changes. A cluster created by this release or later gets /var/lib/deckhouse/sds/elastic/<namespace>/<name>. A cluster created by an earlier release keeps the path its CephCluster already has — /opt/deckhouse/sds/elastic/<name> in d8-sds-elastic, /opt/deckhouse/sds/elastic/<namespace>/<name> elsewhere — for good: the module does not move the mon store, and there is no migration.

    The value can be cleared through the status subresource but not changed; a cleared value is recorded again from the running CephCluster. The directory is not removed when the cluster is deleted, so read it before deleting a cluster you intend to re-create under the same name.

  • status.health

    object

    Latest Ceph health summary surfaced by Rook (CephCluster.status.ceph.health and the matching detail blocks). Drives the top-level OK / Warn / Err indicator on the UI dashboard.
    • status.health.checks

      array of objects

      Active health checks (warnings/errors) reported by Ceph. Empty on HEALTH_OK.
      • status.health.checks.message

        string

        Human-readable description of the check.
      • status.health.checks.name

        string

        Ceph check identifier, for example MON_DOWN, OSD_NEARFULL, POOL_NO_REDUNDANCY.
      • status.health.checks.severity

        string

        Per-check severity, mirroring Rook (HEALTH_WARN / HEALTH_ERR).
    • status.health.lastChecked

      string

      Timestamp of the latest health probe Rook successfully observed against the cluster.
    • status.health.message

      string

      Short human-readable summary Rook attaches to the health probe.
    • status.health.status

      string

      One of HEALTH_OK, HEALTH_WARN, HEALTH_ERR. Empty until Rook publishes the first health probe.

      Allowed values: HEALTH_OK, HEALTH_WARN, HEALTH_ERR

  • status.mgrs

    object

    Ceph manager daemon count and per-version histogram, derived from CephCluster.status.ceph.versions.mgr. Rook deploys a single active manager by default (with optional standbys), so knownToCeph == 1 is the typical healthy value.
    • status.mgrs.byVersion

      array of objects

      Per-version histogram for mgr daemons. Same ordering as osds.byVersion.
      • status.mgrs.byVersion.count

        integer

        Number of mgr daemons reporting this version.

        Allowed values: 0 <= X

      • status.mgrs.byVersion.version

        string

        Ceph version string Rook publishes for this bucket.
    • status.mgrs.knownToCeph

      integer

      Sum of CephCluster.status.ceph.versions.mgr values.

      Allowed values: 0 <= X

  • status.monEndpoints

    array of strings

    List of ceph-mon endpoints in the form <host>:<port>, parsed from the rook-ceph-mon-endpoints ConfigMap.
  • status.monMaxId

    string

    Highest mon id reached so far, copied verbatim from data.maxMonId of the rook-ceph-mon-endpoints ConfigMap. Always a non-negative integer encoded as string for parity with the raw ConfigMap value. Reserved for use by the disaster-recovery flow in a future release.
  • status.mons

    object

    Ceph monitor daemon count and per-version histogram, derived from CephCluster.status.ceph.versions.mon. UI uses knownToCeph to render a coarse quorum indicator (knownToCeph >= floor(spec.mon.count/2)+1).
    • status.mons.byVersion

      array of objects

      Per-version histogram for mon daemons. Same ordering as osds.byVersion.
      • status.mons.byVersion.count

        integer

        Number of mon daemons reporting this version.

        Allowed values: 0 <= X

      • status.mons.byVersion.version

        string

        Ceph version string Rook publishes for this bucket.
    • status.mons.knownToCeph

      integer

      Sum of CephCluster.status.ceph.versions.mon values.

      Allowed values: 0 <= X

  • status.observedGeneration

    integer

    The last value of metadata.generation processed by the controller.
  • status.osds

    object

    OSD daemon count and per-version histogram. Sourced from CephCluster.status.ceph.versions.osd (with desired coming from the controller’s selected BlockDevice count).
    • status.osds.byVersion

      array of objects

      Per-version histogram for OSD daemons. Sorted by count desc, then version asc. Empty until at least one OSD has reported a version.
      • status.osds.byVersion.count

        integer

        Number of OSD daemons reporting this version.

        Allowed values: 0 <= X

      • status.osds.byVersion.version

        string

        Ceph version string Rook publishes for this bucket (verbatim — e.g. ceph version 19.2.5 (...) squid (stable)).
    • status.osds.desired

      integer

      Number of OSDs the controller asked Rook for (== matched BlockDevices for this ElasticCluster).

      Allowed values: 0 <= X

    • status.osds.knownToCeph

      integer

      Sum of CephCluster.status.ceph.versions.osd values — OSD daemons that have reported a version to Ceph since the last heartbeat. Approximation of “alive in Ceph’s eyes”, not Pod readiness.

      Allowed values: 0 <= X

  • status.phase

    string

    Aggregate phase: Pending, InProgress, Ready, or Error.

    Allowed values: Pending, InProgress, Ready, Error

ElasticStorageClass

Short names: esc

Scope: Namespaced
Version: v1alpha1

• spec

  object
  • spec.clusterRef

    string

    Required value

    Name of the ElasticCluster this storage class belongs to. The referenced cluster must exist in the SAME namespace as this resource and be in Ready phase before pool provisioning starts.

    Length: 1..30

    Pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$

  • spec.replication

    string

    High-level replication strategy. The controller translates this into pool-level settings:

    • AvailabilityWithoutConsistency — 2 replicas, min_size=1, requireSafeReplicaSize=false. Maximum availability at the cost of split-brain risk; suitable for non-critical workloads.
    • ConsistencyAndAvailability (default) — 3 replicas, min_size=2. Recommended production setting.
    • HighRedundancy — 4 replicas, min_size=2, requireSafeReplicaSize=true. Tolerates two simultaneous host failures with continued I/O and one extra failure as a recovery margin (data loss only at the fourth simultaneous failure). Requires at least 5 storage nodes; the controller auto-promotes the underlying CephCluster to a 5-mon, 3-mgr topology while at least one HighRedundancy ESC exists, and keeps the promotion sticky after the trigger ESC is removed. See ElasticCluster.status.cephTopology for the audit trail.

    The ErasureCodedCompact mode is temporarily disabled and is not an accepted value.

    Default: ConsistencyAndAvailability

    Allowed values: AvailabilityWithoutConsistency, ConsistencyAndAvailability, HighRedundancy

  • spec.type

    string

    Required value

    Type of the underlying Ceph pool / Kubernetes StorageClass. RBD provisions block volumes via Rados Block Device, CephFS provisions shared-filesystem volumes via CephFS subvolumes.

    Allowed values: RBD, CephFS

• status

  object

  Current state of resources managed by ElasticStorageClass.
  • status.conditions

    array of objects

    Per-component conditions: PoolReady, CsiStorageClassReady, and the aggregate Ready.
    • status.conditions.lastTransitionTime

      string

      Timestamp of the last status transition for this condition.
    • status.conditions.message

      string

      Human-readable explanation of the current status.

      Maximum length: 32768

    • status.conditions.observedGeneration

      integer

      Value of metadata.generation the condition was last set against.

      Allowed values: 0 <= X

    • status.conditions.reason

      string

      Machine-readable reason for the current status, suitable for log filtering.

      Length: 1..1024

      Pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$

    • status.conditions.status

      string

      Current status of the condition.

      Allowed values: True, False, Unknown

    • status.conditions.type

      string

      Condition type, for example Ready, PoolReady, or CsiStorageClassReady.

      Maximum length: 316

      Pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$

  • status.observedGeneration

    integer

    ObservedGeneration is the most recent .metadata.generation reconciled by the controller.
  • status.phase

    string

    Phase is a coarse-grained summary derived from Conditions.

    Allowed values: Pending, InProgress, Ready, Error