Available in:  EE

The module lifecycle stage: Experimental

The module has requirements for installation

The module is in the Experimental stage. The API, configuration, and custom resources may change without notice; do not use it for production workloads.

The sds-object module manages S3-compatible object storage in a Deckhouse Kubernetes Platform cluster. Tenants declare what they need with a small set of COSI-aligned custom resources and the module deploys and operates the backend; an administrator picks the backend and its settings once, in a store resource of that backend’s own Kind.

Custom resources

Resource Scope Purpose
SeaweedFSStore (swfsstore) Cluster An object store on SeaweedFS: master/volume/filer StatefulSets on PVCs, S3 gateway in front. Deploys the data plane.
SDSElasticStore (sdsestore) Cluster An object store on Ceph RGW, provisioned on an existing sds-elastic cluster. Deploys the data plane.
ObjectStore (ostore) Cluster The class tenants consume — the analogue of a StorageClass. Names one store and the defaults buckets inherit; no data plane of its own.
Bucket (bkt) Namespaced What you create to get a bucket.
BucketContents (bktc) Cluster The backing object the controller provisions for a Bucket — what a PersistentVolume is to a claim. Not declared by hand.
BucketAccess (ba) Namespaced Requests scoped credentials for a Bucket; writes a standard S3 credentials Secret next to your application.

Bucket/BucketContents mirror PersistentVolumeClaim/PersistentVolume, and ObjectStore mirrors StorageClass: a name a tenant puts in a spec, with the physical storage behind it. Credentials are issued via BucketAccess against a Bucket in the same namespace; each access gets its own access key, rotatable independently via the storage.deckhouse.io/rotate annotation.

A bucket belongs to the namespace of the Bucket that asked for it, and nothing shares it with another namespace: the resource that used to do that (BucketClaimPolicy, which matched namespaces by name or by regular expression) is gone. Sharing storage between namespaces is a feature to be designed on its own terms rather than a side effect of a policy object.

BucketContents being cluster-scoped is what lets data outlive its namespace: with reclaimPolicy: Retain (the default), deleting the Bucket — or the whole namespace — leaves the contents behind in phase Released with the objects intact, and recreating a Bucket of the same name in the same namespace picks them back up.

Stores and classes

There is one Kind per backend rather than one resource with a type field, the same way sds-local-volume has a Kind for LVM rather than a generic volume group with an engine. Each Kind takes the settings its own engine actually has:

Store Kind Backend Data plane Its own settings
SeaweedFSStore SeaweedFS master/volume/filer StatefulSets, PVCs, S3 gateway component counts, the SeaweedFS replication code, the filer metadata store
SDSElasticStore Ceph RGW a Rook CephObjectStore on an existing sds-elastic cluster pool durability: replication or erasure coding

A cross-backend redundancy intent (None/Standard/High) used to sit in front of both. It is gone: erasure coding is a (k, m) pair that no replica count expresses, “how many nodes” is not a question a Ceph user answers here, and one word silently moved four other settings with it.

An ObjectStore then names a store and the defaults for buckets created through it. Tenants only ever see that name — Bucket.spec.objectStoreRef is a plain string — so which backend is behind a class, and its typed {kind, name} reference, stay administrator-facing. Several classes may point at one store with different defaults, the way several StorageClasses can share a pool.

Implementation status: both store Kinds are available. A SeaweedFSStore with metadataStore: Postgres (which is what allows more than one filer) requires the managed-postgres module; with the default LevelDB store it is self-contained and runs a single filer.

The platform’s own system storage is not here. It used to be a System profile on Garage; since Garage is AGPL-licensed and therefore never offered as a user-facing service, it moved to the separate sds-object-system module. Nothing in this module deploys Garage any more.

How it works

  • The controller reconciles each store (SeaweedFSStore, SDSElasticStore) into a backend data plane (workloads, services, configuration) and reports readiness, the S3 endpoint, and capacity in status.
  • The controller reconciles each ObjectStore by resolving its storeRef and mirroring that store’s readiness, so a class is Ready exactly when it can be provisioned through.
  • The controller reconciles each Bucket into a cluster-scoped BucketContents it owns, recording the store the class pointed at, and those contents into a bucket in that store (no credentials).
  • The controller reconciles each BucketAccess — once its Bucket is Bound — into a scoped access key, then writes a Secret (owned by the access) with the standard connection variables: S3_ENDPOINT, S3_REGION, S3_BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY.
  • When a Bucket is deleted, its contents follow the reclaim policy: Delete removes the bucket and its objects, Retain leaves the contents in phase Released.

The contents record the store and not the class, on purpose: the data lives in the store, so deleting a class takes away the name tenants provision through and nothing else.

See Usage for a walkthrough.

Requirements

  • SDSElasticStore requires the sds-elastic module with a ready ElasticCluster.
  • SeaweedFSStore is self-contained (the SeaweedFS image ships with the module); metadataStore: Postgres additionally needs the managed-postgres module for the filer metadata.