Usage

Enabling the module

d8 k apply -f - <<EOF
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: sds-object
spec:
  enabled: true
  version: 1
EOF

Creating a cluster

A Lightweight cluster backed by PVCs on an existing StorageClass:

apiVersion: storage.deckhouse.io/v1alpha1
kind: ObjectStore
metadata:
  name: shared
spec:
  type: Lightweight
  storage:
    sizePerNode: 50Gi
    class: localpath
  redundancy: Standard

A System cluster for platform needs (Garage on control-plane nodes, hostPath; storage.class is ignored):

apiVersion: storage.deckhouse.io/v1alpha1
kind: ObjectStore
metadata:
  name: system
spec:
  type: System

Track readiness:

d8 k get objectstore
# NAME     TYPE          PHASE   ENDPOINT                                           READY   AGE
# shared   Lightweight   Ready   http://shared-garage.d8-sds-object.svc...:3900     True    3m

Running the system storage on a single replica

The built-in system store (shipped by the module, see systemBucket) runs three Garage replicas by default and rebalances them across control-plane nodes. On small or non-production installations that can be reduced to one replica:

apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: sds-object
spec:
  enabled: true
  version: 1
  settings:
    systemBucket:
      singleReplica: true

The single replica is placed on one control-plane node and never migrates between masters: its data is on that node’s local volume and there is no second copy to re-replicate from. While that node is down or removed, the system storage is unavailable — the data on its disk stays intact, but the controller will not relocate the replica.

Follow the recreate through the store’s status — it reports the teardown, then comes back Ready with one replica:

d8 k get objectstore system -o jsonpath='{.status.phase}{"\n"}{.status.conditions[?(@.type=="BackendReady")].message}{"\n"}'
d8 k get pods -n d8-sds-object -l storage.deckhouse.io/object-store=system

Declaring a Shared bucket

Bucket is cluster-scoped — an administrator declares a bucket in an object store; it carries no credentials. This is a Shared bucket, meant to be consumed from multiple namespaces via policy-gated claims:

apiVersion: storage.deckhouse.io/v1alpha1
kind: Bucket
metadata:
  name: app-data
spec:
  objectStoreRef: shared
  # bucketName defaults to metadata.name
  accessPolicy: Private
  reclaimPolicy: Retain

d8 k get bucket app-data
# NAME       OBJECTSTORE   BUCKET     PHASE   READY   AGE
# app-data   shared        app-data   Ready   True    30s

Allowing namespaces to bind the bucket

Binding a Shared bucket is deny-by-default: a namespace can claim it only when a BucketClaimPolicy for the bucket matches it. Namespaces are selected by exact names and/or RE2 patterns.

BucketClaimPolicy is a namespaced resource and is only honored in the bucket’s owner namespace: sharing is granted by whoever owns the bucket. For an administrator-declared bucket the owner is the module namespace d8-sds-object; for a bucket provisioned by a greenfield claim it is that claim’s namespace. A policy created anywhere else is rejected by the webhook — otherwise a consumer could grant access to itself.

apiVersion: storage.deckhouse.io/v1alpha1
kind: BucketClaimPolicy
metadata:
  name: app-data-teams
  # app-data is administrator-declared, so the module namespace owns it.
  namespace: d8-sds-object
spec:
  bucketRef: app-data
  allowedNamespaces:
    names:
      - my-app
    patterns:
      - "team-.*"

The owner of a greenfield bucket can share it the same way — by creating a policy in its own namespace, with bucketRef set to the bucket name from the claim’s status.boundBucketName.

Claiming the bucket

Each consuming namespace declares a BucketClaim. To bind the Shared bucket above (brownfield), set existingBucketName. To provision a new private bucket instead (greenfield), omit existingBucketName and set objectStoreRef — no policy is needed for greenfield claims.

apiVersion: storage.deckhouse.io/v1alpha1
kind: BucketClaim
metadata:
  name: app-data
  namespace: my-app
spec:
  existingBucketName: app-data   # brownfield: bind the Shared bucket (policy-gated)
  # --- or, for a new private bucket, drop existingBucketName and use: ---
  # objectStoreRef: shared
  # accessPolicy: Private
  # reclaimPolicy: Retain

d8 k -n my-app get bucketclaim app-data
# NAME       BUCKET     PHASE   READY   AGE
# app-data   app-data   Ready   True    20s

Requesting credentials

Each workload declares a BucketAccess referencing a Bound BucketClaim in its namespace. The controller mints a dedicated access key / secret key scoped to the bound bucket and writes a Secret (named <access>-s3-credentials by default) in the same namespace:

apiVersion: storage.deckhouse.io/v1alpha1
kind: BucketAccess
metadata:
  name: app-data
  namespace: my-app
spec:
  bucketClaimName: app-data
  permission: ReadWrite   # or ReadOnly

d8 k -n my-app get bucketaccess app-data
# NAME       CLAIM      PHASE   SECRET                    READY   AGE
# app-data   app-data   Ready   app-data-s3-credentials   True    20s

Consuming the credentials

The credentials Secret holds the standard S3 connection variables, ready to be mounted with envFrom:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: my-app
spec:
  template:
    spec:
      containers:
        - name: app
          image: my-app:latest
          envFrom:
            - secretRef:
                name: app-data-s3-credentials

Rotating credentials

To rotate the access key of an BucketAccess, set or change the storage.deckhouse.io/rotate annotation. The controller issues a fresh key pair, updates the Secret, and revokes the previous key:

d8 k -n my-app annotate bucketaccess app-data \
  storage.deckhouse.io/rotate="$(date +%s)" --overwrite

Reclaim policy

• Bucket reclaimPolicy: Retain (default) — deleting the Bucket keeps the bucket and its objects; Delete removes them.
• Deleting an BucketAccess always revokes its access key and removes its credentials Secret (it does not touch bucket data).
• Cluster reclaimPolicy: Retain (default) — deleting the ObjectStore preserves persisted data (for Heavy, the Ceph RGW pools are kept; for PVC-backed profiles the PVCs are left in place). Delete destroys it.

Heavy profile

The Heavy profile provisions a Ceph RADOS Gateway on top of an existing sds-elastic cluster and is selected with spec.elasticClusterRef:

apiVersion: storage.deckhouse.io/v1alpha1
kind: ObjectStore
metadata:
  name: heavy
spec:
  type: Heavy
  elasticClusterRef: main