Usage

This page covers the main module workflows, from basic setup to additional storage and bucket features.

Quick start

The following steps enable the module, create object storage, and provide access to a bucket.

Enabling the module

d8 k patch moduleconfig deckhouse --type=merge --patch '{"spec":{"settings":{"allowExperimentalModules":true}}}'

Enable the module with a ModuleConfig:

d8 k apply -f - <<EOF
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
  name: sds-object
spec:
  enabled: true
  version: 1
EOF

Creating storage

Storage is the data plane. Each supported backend uses its own Kind.

An example of creating a SeaweedFS storage class on top of an existing StorageClass using the SeaweedFSStore backend:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: SeaweedFSStore
metadata:
  name: default
spec:
  masters: 3
  volumeServers: 3
  replication: "001"        # One extra copy on another volume server.
  storage:
    sizePerNode: 50Gi
    class: localpath
EOF

The component counts and the replication code are SeaweedFS’s own settings and must agree. The third digit of the replication code sets the number of extra copies on other volume servers, so volumeServers must be at least that value plus one. Otherwise, SeaweedFS cannot complete writes.

That minimum is sufficient to place the copies, but it does not make writes tolerant of a lost volume server. Keep one additional volume server as a spare if writes must recover after one server disappears:

The recovery is not instantaneous even with a spare. In failure tests with SeaweedFS 4.47, a three-volume-server 001 store paused writes for roughly 27 seconds to 4.5 minutes after abrupt node loss; bringing the node back caused intermittent errors for about 80 seconds while volumes re-registered. These figures are observations, not an SLA: master leadership, heartbeat timing, the number of volumes, and node/network recovery all affect them. Reads from a surviving replica remained available.

Use three masters and, when the S3 endpoint itself must survive a node loss, three filers backed by PostgreSQL or another shared metadata store. A single filer makes the endpoint unavailable for the whole filer outage. Configure clients to retry transient S3 5xx and connection errors with exponential backoff for longer than the expected failover window, while preserving the application’s own idempotency rules.

To use several filers, set the filers and metadataStore fields in the SeaweedFSStore’s spec: the default value metadataStore takes, LevelDB, keeps metadata on each filer’s own volume and does not support sharing it across filers.

spec:
  filers: 3
  metadataStore: Postgres   # Requires the managed-postgres module.

Creating an object storage class

ObjectStore defines a storage class available to users. It references the storage and sets default parameters for the buckets it creates.

An example of an ObjectStore referencing the SeaweedFSStore created above:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: ObjectStore
metadata:
  name: standard
spec:
  storeRef:
    kind: SeaweedFSStore
    name: default
  reclaimPolicy: Retain
  quota:
    maxSize: 100Gi          # Maximum bucket size; larger requests are rejected.
EOF

Check the storage status:

d8 k get seaweedfsstore

Example output:

NAME      VOLUMES   REPLICATION   PHASE   ENDPOINT                                    READY   AGE
default   3         001           Ready   http://default-seaweedfs.d8-sds-object...   True    3m

Once the data plane is deployed, the storage becomes Ready, and its endpoint appears in the ENDPOINT column.

Check the ObjectStore status:

d8 k get objectstore

Example output:

NAME       STORE-KIND       STORE     RECLAIM   PHASE   READY   AGE
standard   SeaweedFSStore   default   Retain    Ready   True    2m

An ObjectStore becomes Ready once the storage it references exists and is itself Ready.

The admission webhook rejects an unknown storeRef.kind value. The supported storage Kinds come from the controller’s driver registry, not from the CRD schema, so adding a new backend does not require changing the ObjectStore schema.

Requesting a bucket

Create the application namespace if it does not exist:

d8 k create namespace my-app

To create an S3 bucket, create a Bucket. The controller creates a cluster-wide BucketContents object for it, resolves the storage through the ObjectStore, and creates the bucket in the selected backend. BucketContents is linked to the originating Bucket and keeps track of where the data lives.

An example of creating a Bucket with accessPolicy: Private and reclaimPolicy: Retain:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: Bucket
metadata:
  name: app-data
  namespace: my-app
spec:
  objectStoreRef: standard
  accessPolicy: Private
  reclaimPolicy: Retain     # Do not set this if you want to inherit the class value.
EOF

Check the Bucket status:

d8 k -n my-app get bucket app-data

Example output:

NAME       CONTENTS              PHASE   READY   AGE
app-data   contents-1f2e3d-...   Ready   True    20s

In the Ready state, the Bucket shows the name of its BucketContents.

The created BucketContents is visible cluster-wide (d8 k get bktc), but it should not be created by hand. The admission webhook allows creating this resource only from the module’s own ServiceAccount. This prevents a BucketContents from appearing without a matching user-created Bucket.

If the bucket already exists

The bucket name in the backend is derived from BucketContents, so a bucket with that name may already exist for reasons unrelated to the module: it was created by hand, it survived a storage recreation, or it belongs to another cluster using the same backend.

The module tags every bucket it creates with two tags:

storage.deckhouse.io/owned-by          sds-object
storage.deckhouse.io/bucket-contents   contents-1a2b3c4d5e-team-a-data

If these tags are missing from a bucket the module has never provisioned, the module does not take it under management. BucketContents stays not ready, and the BucketReady condition gets the reason BucketNotOwnedByModule. The existing bucket’s contents are left untouched.

Example BucketReady condition state:

BucketReady   False   BucketNotOwnedByModule
  bucket "team-a-data" already exists in the backend and is not managed by this
  module (it carries no ownership tag); remove it or point this Bucket at
  another store

In this situation, either delete the conflicting bucket if it is no longer needed, or point the Bucket at different storage. The controller does not retry automatically, so it never grants access to data it does not manage.

Buckets already recorded in status.bucketName are handled separately. If an application replaces the bucket tag set or the bucket predates ownership tags, the next reconcile restores the two ownership tags and preserves the application’s other tags. A conflicting storage.deckhouse.io/owned-by or storage.deckhouse.io/bucket-contents value is never overwritten.

On SeaweedFS, the module also removes bucket policies during reconciliation. Public read access is implemented by the module-managed anonymous identity instead. This cleanup removes policies created by tenant credentials on older SeaweedFS versions before those credentials were forbidden from changing bucket policy.

Requesting credentials

To get credentials, create a BucketAccess that references a Bucket in the same namespace with the Bound condition set to True. The controller generates a separate access key and secret key for this bucket and stores them in a Secret named <ACCESS>-s3-credentials by default.

An example of creating a BucketAccess referencing the Bucket app-data with permission: ReadWrite:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: BucketAccess
metadata:
  name: app-data
  namespace: my-app
spec:
  bucketRef: app-data
  permission: ReadWrite   # Use ReadOnly for read-only access.
EOF

Check the BucketAccess status:

d8 k -n my-app get bucketaccess app-data

Example output:

NAME       BUCKET     PHASE   SECRET                    READY   AGE
app-data   app-data   Ready   app-data-s3-credentials   True    20s

Once the credentials are created, BucketAccess shows the name of the Secret they are stored in.

One access for several buckets

An application often needs more than one bucket: data, backups, logs. Instead of one BucketAccess per bucket, list the buckets in spec.buckets: the controller issues a single key pair with access to all of them and puts it in one Secret.

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: BucketAccess
metadata:
  name: app
  namespace: my-app
spec:
  buckets:
    - app-data
    - app-backups
    - app-logs
  permission: ReadWrite   # Applies to every bucket in the list.
EOF

How the list behaves:

• The list can be changed. A bucket added to it is granted to the existing key; a bucket removed from it loses its grant at once, and the key itself stays the same.
• All the buckets must be served by the same storage: one key pair works within one storage. The first bucket granted fixes it in status.storeRef. A bucket provisioned through an ObjectStore that points at a different storage is not granted, and the admission webhook warns about it.
• status.buckets shows the state of every bucket. A bucket that is not granted yet does not block the others: the Secret is written for the buckets that are granted, and the access stays not Ready until all of them are.
• A bucket that stays in the list but stops being Bound keeps its grant while other buckets of the list keep the key in use; status.buckets says so. To take the grant away, remove the bucket from the list.
• Key rotation through the storage.deckhouse.io/rotate annotation replaces the key for the whole list at once.
• A Bucket is not deleted while an access lists it. Remove the bucket from spec.buckets first.

spec.bucketRef and spec.buckets are mutually exclusive. To extend an existing access with spec.bucketRef, move it to spec.buckets and keep the bucket spec.bucketRef named in the list: the key pair does not change.

d8 k -n my-app patch bucketaccess app-data --type merge \
  -p '{"spec":{"bucketRef":null,"buckets":["app-data","app-backups"]}}'

Using the credentials

The Secret contains the standard S3 connection variables. Add it to the container in the Deployment manifest through envFrom:

Example Deployment fragment with the Secret attached:

spec:
  template:
    spec:
      containers:
        - name: app
          envFrom:
            - secretRef:
                name: app-data-s3-credentials

Storage on Ceph RGW

Storage on Ceph RGW is created through an SDSElasticStore object. It deploys a Ceph RADOS Gateway on top of an existing sds-elastic cluster and accepts Ceph’s own pool settings.

An example of creating an SDSElasticStore with size: 3 replication:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: SDSElasticStore
metadata:
  name: heavy
spec:
  elasticClusterRef: main
  # The namespace that ElasticCluster lives in; `d8-sds-elastic` when omitted.
  elasticClusterNamespace: d8-sds-elastic
  dataPool:
    replicated:
      size: 3
    # Alternatively, use erasure coding instead of replication.
    # erasureCoded: { dataChunks: 4, codingChunks: 2 }
EOF

sds-elastic runs one Ceph cluster per namespace, and everything that cluster owns — mons, OSDs, pools, and the RGW this store adds to it — lives in the namespace of its ElasticCluster. So elasticClusterNamespace is where this store’s CephObjectStore, its RGW Service and its credentials Secrets are created, not only where the cluster is looked up. Point a store at a cluster in team-a and everything it creates lands in team-a.

Both fields are immutable: the pools are in that one Ceph cluster, and re-pointing a store would abandon the data rather than move it. A store written before the field existed keeps working unchanged — an absent elasticClusterNamespace means d8-sds-elastic, which is where sds-elastic migrates a cluster created before ElasticCluster became namespaced.

An example of creating an ObjectStore referencing the SDSElasticStore:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: ObjectStore
metadata:
  name: capacity
spec:
  storeRef:
    kind: SDSElasticStore
    name: heavy
EOF

Rotating credentials

To change the access key for a BucketAccess, set or change the storage.deckhouse.io/rotate annotation. The controller issues a new key pair, updates the Secret, and revokes the previous key:

d8 k -n my-app annotate bucketaccess app-data storage.deckhouse.io/rotate="$(date +%s)" --overwrite

Using an external metadata database

metadataStore: External lets you use an external PostgreSQL instance that the module does not deploy or operate. As with metadataStore: Postgres, several filers can share one metadata database. The administrator is responsible for the external database’s availability, backups, and upgrades.

If the metadata database is unavailable, the storage cannot serve S3 requests. Data replication across volume servers does not substitute for the metadata database’s own availability.

An example of creating a Secret and a SeaweedFSStore that uses it:

d8 k apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: media-metadata-db
  namespace: d8-sds-object
stringData:
  host: pg.example.internal
  port: "5432"
  database: seaweedfs_media
  username: seaweedfs
  password: <PASSWORD>
  sslmode: verify-full
  ca.crt: |
    -----BEGIN CERTIFICATE-----
    ...
---
apiVersion: storage.deckhouse.io/v1alpha1
kind: SeaweedFSStore
metadata:
  name: media
spec:
  metadataStore: External
  externalMetadataStore:
    secretRef:
      name: media-metadata-db
  filers: 3
  storage:
    class: linstor-r2
EOF

Database requirements. Use PostgreSQL 12 or newer, a separate empty database, and a role allowed to create tables in it. The filer creates one table per bucket on first use and does not run schema migrations.

Do not use a shared application database: table names are derived from bucket names and can collide with table names used by other applications.

TLS. The default is sslmode: require: the connection is encrypted, but the server certificate is not verified. To also verify the certificate and server name, add the CA certificate to the Secret’s ca.crt key and set sslmode: verify-full. The CA certificate is mounted into the filer and used when establishing the connection.

sslmode: disable is rejected during configuration validation, since the connection carries the password and object metadata. A typo or another sslmode value libpq does not recognize can pass the database reachability check — that check only verifies the DNS/TCP connection to the given address and port. The error surfaces when the filer starts: libpq rejects the value, and the filer pod keeps restarting.

Changing connection parameters. After you change the database address, password, or CA certificate in the Secret, the filers restart and load the new parameters. Connection parameters are read only when the process starts.

With a single filer, the S3 endpoint is briefly unavailable. With several filers, they restart one at a time.

Pod placement

The spec.placement parameter sets nodeSelector and tolerations. These settings apply to all SeaweedFSStore pods the module creates.

Module upgrades roll SeaweedFS data planes through a global queue. The controller changes one StatefulSet of one store, waits until its new revision is fully ready, and only then changes the next component or store. This prevents a module image update from restarting every S3 endpoint in the cluster at once. A store that is already degraded holds the queue rather than adding another disruption; its BackendReady message names the rollout being awaited.

Master and volume-server replicas require placement on separate nodes. If no suitable separate node is available for a replica, the corresponding pod stays in the Pending state.

For filers, spreading across separate nodes is preferred. If that placement is not possible, several filers can run on the same node. Losing several filers reduces service availability until the pods are rescheduled, but does not cause metadata loss, since metadata is stored separately.

The module creates a PodDisruptionBudget for every master, volume-server, and filer set. A voluntary disruption keeps a master quorum and at least one data-plane replica running. For a single-replica component, its only pod cannot be evicted until the administrator explicitly changes or removes the budget. Readiness and liveness probes use the SeaweedFS HTTP health endpoints; opening the TCP port alone does not make a pod Ready.

The spec.placement parameter does not apply to the managed metadata database. PostgreSQL placement is configured separately in a PostgresClass, using tolerations, nodeSelector, and nodeAffinity. To use a specific PostgresClass, set it in spec.postgresClassName. If the parameter is not set, the default class is used.

Example storage placement configuration with a dedicated PostgresClass for the metadata database:

spec:
  metadataStore: Postgres
  postgresClassName: storage-dedicated
  placement:
    nodeSelector:
      node-role/storage: ""
    tolerations:
      - key: storage.deckhouse.io/dedicated
        operator: Exists
        effect: NoSchedule

Responsibility for the external database. The module does not create the database or role, does not run migrations, does not create backups, and does not monitor the PostgreSQL server’s health. Before configuring the filer, the module checks that the given address and port are reachable. A DNS resolution error or an unreachable port is reflected in the storage status.

Example status message:

BackendReady   False   Pending
  the external metadata database at pg.example.internal:5432 did not answer:
  dial tcp: lookup pg.example.internal: no such host

If the metadata database is lost, objects remain on the volume servers, but the mapping between objects and buckets is lost. Include this database in the storage’s backup plan.

Reclaim policy

What happens to stored data on deletion depends on the reclaim policy (reclaimPolicy parameter):

• For a Bucket with reclaimPolicy: Retain, corresponding bucket and its objects are kept. BucketContents moves to the Released phase and keeps a record of the original Bucket and the storage where the data lives:

  d8 k get bktc

  Example output:

  NAME                  STORE-KIND       STORE     OWNER-NS   OWNER      BUCKET            PHASE      READY   AGE
  contents-1f2e3d-...   SeaweedFSStore   default   my-app     app-data   my-app-app-data   Released   False   4h
  

  If you create a Bucket named app-data in the my-app namespace again, the controller links it back to the retained BucketContents and its data. The same happens when the namespace is deleted: the Bucket is removed with it, while Retain data survives.

  Delete instead removes the bucket with its objects, and BucketContents along with them. A Bucket that does not set the policy inherits it from its class; the class default is Retain.

• Deleting a BucketAccess always revokes its access key and deletes its Secret (the bucket’s data is not affected).

• Storage reclaimPolicy: Retain (the default) — deleting the storage keeps the data: SDSElasticStore keeps its Ceph RGW pools, SeaweedFSStore keeps its PVCs. Delete destroys the data.

• Deleting an ObjectStore does not delete the storage or any buckets already created through it. After the class is deleted, you can no longer create Buckets through it. The reference to the actual storage is kept in BucketContents.

To delete data retained after a Bucket with reclaimPolicy: Retain was deleted, delete the corresponding BucketContents. If the linked Bucket is already gone, the controller deletes the bucket in the backend.

The reclaim policy governs what happens when a Bucket is deleted. Deleting BucketContents itself is a separate operation. If the Bucket is already gone, deleting BucketContents deletes the retained bucket in the backend. If the linked Bucket still exists, the controller recreates BucketContents and links it back to the existing bucket.

Publishing storage outside the cluster

By default, storage is reachable only from inside the cluster: status.endpoint has only the internal field filled in, and that address is what goes into the credentials Secret. To make storage usable from outside, publish it through the alb module, which implements the Gateway API on top of Envoy.

Before publishing, prepare the following:

1. Create a Gateway using an ALBInstance or ClusterALBInstance from the alb module. Get the Gateway’s name and namespace from that object’s status.
2. Create a DNS record for the external endpoint’s hostname.
3. Create a kubernetes.io/tls Secret with a certificate for that hostname in the d8-sds-object namespace. The module does not issue certificates, so use a Secret created by cert-manager or one you have prepared in advance.

Then add a publish block to the storage:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: SeaweedFSStore
metadata:
  name: default
spec:
  storage:
    class: linstor-thin-r2
  publish:
    hostname: s3.example.com
    gatewayRef:
      name: public-gw      # The Gateway object created by the administrator.
      namespace: d8-alb
    tls:
      secretRef:
        name: s3-example-com-tls
EOF

The module creates a ListenerSet (host, port 443, TLS termination with the given certificate) and an HTTPRoute to the storage’s S3 port. Check the published endpoint:

d8 k get seaweedfsstore default -o jsonpath='{.status.endpoint}'

Example output:

{"external":"https://s3.example.com","internal":"http://default-seaweedfs...:8333","region":"us-east-1"}

TLS is mandatory and cannot be disabled: S3 credentials travel in the Authorization header, so publishing over plain HTTP would let anyone watching the traffic intercept them.

The module publishes the hostname, and the listener uses port 443. If the ALBInstance or ClusterALBInstance is configured with a HostPort inlet on a non-standard port, for example 8443, status.endpoint.external contains https://<HOST> without a port number. In that environment, the client must explicitly use https://<HOST>:8443.

For a LoadBalancer inlet, which accepts connections on port 443, the address from status.endpoint.external can be used as is.

Choosing the address in the credentials

By default, the credentials Secret gets the storage’s in-cluster address. To use the external address, set endpointScope: External on the BucketAccess.

An example of creating a BucketAccess with endpointScope: External:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: BucketAccess
metadata:
  name: backup-writer
  namespace: my-app
spec:
  bucketRef: app-data
  permission: ReadWrite
  endpointScope: External
EOF

If a BucketAccess requests the external address but the storage is not published, BucketAccess does not fall back to the in-cluster address automatically. The object stays not ready, and its condition reports that spec.publish needs to be configured on the storage.

When the endpoint changes (publishing was enabled, or the hostname changed), the Secrets of all affected BucketAccess objects are reissued automatically.

Addressing limits

Path-style addressing is supported: https://s3.example.com/<BUCKET>/<KEY>. It needs only one DNS name and one certificate.

Virtual-hosted addressing (https://<BUCKET>.s3.example.com/<KEY>) is represented in the API as spec.publish.addressing: VirtualHosted, but is currently rejected by the admission webhook. It requires a wildcard DNS name and a wildcard certificate, and support for it varies across the backends’ S3 gateways and has not been verified by the module yet. As a result, a published endpoint supports path-style addressing only.

Components available only inside the cluster

For SeaweedFS, only the S3 port is published externally. The module creates a separate Service with just that port for it. The storage’s main Service also carries the filer’s HTTP and gRPC APIs, so it is not used for external publishing.

In Ceph RGW, the S3 API and the Admin Ops API share a port, and administrative methods are available under /admin/… paths. As a result, publishing an SDSElasticStore also makes that path reachable from the network.

Credentials the module issues to users carry no administrative rights and do not grant access to the Admin Ops API. However, Rook creates a user named rgw-admin-ops-user with buckets=*;users=* capabilities for every object store. Its keys are stored in a Secret in the d8-sds-elastic namespace. The security of the Admin Ops API depends on protecting these credentials and on restrictions at the Gateway level.

Before publishing Ceph RGW storage, consider the following protections:

• Deny access to /admin on the Gateway, if your configuration supports it.
• If restricting /admin is not possible, consider a separate CephObjectStore for the external endpoint.
• Treat the rgw-admin-ops-user Secret as administrative credentials exposed to the internet.

SeaweedFS has no such exposure: the external Service carries only the S3 port, and the filer’s own APIs stay unreachable through the external route.

Diagnosing external publishing

The state of external publishing is reflected in a separate PublishedEndpointReady condition and does not affect the storage’s own Ready state. A broken external route does not block creating or deleting buckets, or issuing credentials for in-cluster access.

Check the PublishedEndpointReady condition:

d8 k get seaweedfsstore default -o jsonpath='{range .status.conditions[?(@.type=="PublishedEndpointReady")]}{.reason}: {.message}{end}'

Common reasons:

Public read

accessPolicy: PublicRead on a Bucket opens its objects for anonymous reading. Both backends support this mode.

An example of creating a Bucket with accessPolicy: PublicRead:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: Bucket
metadata:
  name: site-assets
  namespace: my-app
spec:
  objectStoreRef: standard
  accessPolicy: PublicRead
EOF

Verify that the object can be read without credentials:

curl -s https://s3.example.com/my-app-site-assets/logo.png -o logo.png

Public access grants only reading objects:

• Objects only, no listing. An anonymous GetObject call succeeds if the object key is known. ListObjects for the bucket stays forbidden. Public read access does not include a public listing of the bucket’s contents.
• Read only. Anonymous write, delete, and tagging are forbidden on both backends.
• This bucket only. The grant applies to this bucket’s name; it does not affect other buckets in the same storage.

Keep these two things in mind when using PublicRead:

• Public access depends on the endpoint being reachable. Inside the cluster, the S3 endpoint is reachable through the module’s Service. For access from outside, configure spec.publish on the storage. Publishing makes the endpoint reachable over the network; accessPolicy decides whether reading a given bucket requires authorization.
• A Bucket in the Released phase stays public. Deleting a Bucket with reclaimPolicy: Retain keeps the data and the BucketContents itself, along with its accessPolicy. As a result, the data stays available in the same mode after the Bucket is deleted. BucketContents cannot be edited directly: the admission webhook accepts changes to it only from the module, and the module does not apply the settings of a Released BucketContents to the backend. To turn off public read and keep the data, create the Bucket again with the same name in the same namespace and accessPolicy: Private (if the namespace was deleted, create it first). The controller links the Bucket back to the retained BucketContents and turns off public read. Set versioning and lifecycle to the values from the BucketContents spec: the controller applies these fields from the Bucket as well, so leaving them out suspends versioning and removes the expiry rules. After anonymous reading is refused (see the curl example above), the Bucket can be deleted again: BucketContents returns to the Released phase with accessPolicy: Private. If the retained data is no longer needed, delete BucketContents instead: this deletes the bucket and its objects in the backend.

After accessPolicy on a Bucket is changed to Private, public read access is turned off at the next reconcile. Access keys already issued for the bucket are not affected.

Versioning and object lock

versioning: Enabled keeps every version of an object instead of overwriting it. objectLock turns on WORM (Write Once Read Many) behavior for object versions. A version under an active retention period cannot be deleted before it ends — not by the user, not by the storage administrator, and not by the module itself.

An example of creating a Bucket with versioning and Object Lock enabled:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: Bucket
metadata:
  name: audit-log
  namespace: my-app
spec:
  objectStoreRef: standard
  versioning: Enabled
  objectLock:
    mode: Compliance     # Allowed values: Governance or Compliance.
    days: 365
  reclaimPolicy: Retain  # Required together with objectLock.
EOF

Governance mode allows bypassing the restriction with the right permission. In Compliance mode, no one can delete a protected version before its retention period ends. The retention period can be extended, but not shortened. The backend enforces these rules.

The module reflects the actual state of these settings in the backend, in status:

d8 k get bktc -o custom-columns='NAME:.metadata.name,VERSIONING:.status.versioning,LOCK:.status.objectLock.enabled,MODE:.status.objectLock.mode'

API checks

The admission webhook rejects several combinations before they ever reach the backend:

• objectLock requires versioning: Enabled. Object lock is built on versions.
• objectLock is immutable and can only be set when the Bucket is created. Ceph RGW accepts Object Lock configuration only when the bucket is created, so it can never be added to an existing bucket. This restriction is the same on both backends: to protect existing data, create a new bucket with Object Lock and migrate the data.
• objectLock is incompatible with reclaimPolicy: Delete — including when Delete comes from the default class. A bucket with protected objects cannot be deleted, so that combination would leave BucketContents in Terminating until the last retention period expires. The request is rejected outright, not silently switched to Retain.
• Versioning cannot be turned off on a locked bucket. Both backends refuse this, and so does the API.

Legal hold

Legal hold blocks deleting an object version indefinitely and stays in effect until explicitly released. It is not configured on the Bucket: the client sets and clears legal hold for a specific object version using its own credentials:

mc legalhold set myalias/my-app-audit-log/report.pdf
mc legalhold clear myalias/my-app-audit-log/report.pdf

While legal hold is set, the protected version cannot be deleted by any means. To delete it, clear the legal hold first.

Deleting protected data

Deleting a Bucket with reclaimPolicy: Retain keeps the data, moves BucketContents to the Released phase, and keeps the objects’ protection settings in effect. To delete the retained data, delete the BucketContents. If the bucket still has protected versions, the backend refuses the deletion, BucketContents keeps its finalizer, and the reason appears in status.conditions.

Check the Ready condition message:

d8 k get bktc contents-... -o jsonpath='{.status.conditions[?(@.type=="Ready")].message}'

Example output:

the backend refuses to delete the bucket while its objects are protected; ...

Once the last retention period expires, deletion proceeds automatically. Before that, protected versions can only be deleted with permission to bypass retention. Compliance mode allows no such bypass.

Storage capacity

Storage capacity data is published in status.capacity and shown in the wide output of d8 k.

Check the storage capacity:

d8 k get seaweedfsstore media -o wide

Example output:

NAME    VOLUMES   REPLICATION   PHASE   ENDPOINT                     USED%   CAPACITY   READY
media   3         001           Ready   http://...svc:8333           37.24   300Gi      True

status.capacity fragment from the SeaweedFSStore YAML representation:

status:
  capacity:
    total: 300Gi
    used: 111Gi
    available: 189Gi
    usedPercent: "37.24"
    lastUpdated: "2026-08-30T09:12:04Z"

status.capacity is calculated differently for each backend:

• SeaweedFSStore — the sum of disk capacity used by this storage’s volume servers. If a volume server is unreachable while capacity is being collected, its capacity is excluded from the total. This keeps an unreachable disk from being counted as free space.
• SDSElasticStore — the raw capacity of the entire Ceph cluster, as reported by Ceph itself. This value is not a quota for this particular SDSElasticStore and does not show how much space an application can actually use: the replication scheme and usage by the cluster’s other pools both affect it.

If capacity data could not be obtained from either source, status.capacity is not populated. A missing field means the capacity is unknown; a zero value is not used for this case.

Growing the storage

A storage grows in two ways, and they can be combined:

• spec.volumeServers — more volume servers, each with a disk of its own. The value can only be increased: removing a volume server takes its PVC, and the objects on it, with it.
• spec.storage.sizePerNode — a bigger disk under every volume server. The value can only be increased as well, and the StorageClass has to provide a working volume resizer in addition to setting allowVolumeExpansion: true. An unsupported edit is refused by the admission webhook, with the reason in the response.

Deckhouse and Rancher local-path provisioners do not resize existing PVCs. Their StorageClass may advertise allowVolumeExpansion: true, but changing the request only leaves Kubernetes waiting for an external resizer that does not exist. The webhook therefore rejects sizePerNode growth on these known provisioners; add volume servers or migrate the store to expandable CSI-backed storage instead.

Grow the disks:

d8 k patch seaweedfsstore media --type merge -p '{"spec":{"storage":{"sizePerNode":"500Gi"}}}'

The module then expands the PersistentVolumeClaim of every volume server, and the disks reach the new size at the speed of the StorageClass’s provisioner. Two things happen along the way, neither of which interrupts S3 service:

• the StatefulSet of the volume servers is rebuilt around its running pods, because Kubernetes does not allow the volume claim template of an existing StatefulSet to be changed. The pods keep serving through the rebuild, and the claims are retained;
• a volume server is restarted, one at a time, when the CSI driver can only grow a filesystem that nothing has mounted. A driver that grows a mounted one is given several minutes to do it first, because a restart costs more than the wait: a network-attached claim lets the pod come back on another node, where it waits out the attachment the old node still holds. Nothing is restarted while another volume server is not ready.

Watch the progress in status.storage, which reports the disks rather than the request:

status:
  storage:
    sizePerNode: 50Gi
    volumeSizeLimitMB: 1024
    slotsPerServer: 256
    message: growing the volume servers' disks to 500Gi (2 of 3 still smaller)

status.storage.sizePerNode is the smallest disk among the volume servers, and it reaches spec.storage.sizePerNode once every disk has grown. volumeSizeLimitMB and slotsPerServer are what SeaweedFS is run with — how large one volume file may grow and how many of them one volume server may hold — and they follow the disks, not the request, so the master is never pointed at space that is not there yet.

Shrinking sizePerNode is refused: Kubernetes does not shrink a PersistentVolumeClaim, and the objects already written would have nowhere to go. Leaving the field out means the 10Gi default, so removing it from a larger storage is refused for the same reason.

The metadata store sizes itself

SeaweedFS keeps one filer entry per object — its path, its attributes and the list of chunks it is made of — so the metadata store grows with the number of objects, not with the bytes they hold. A storage of 4 KiB objects can therefore run out of metadata room with its data disks nearly empty, and the failure is not a graceful one: when the metadata store fills up, the filer’s writes fail and the S3 gateway goes with them, reads included, because a read needs the entry too.

There is no field for it, and deliberately so: the number behind one would be the average object size, which nobody knows before the objects are written. Instead the module sizes the store itself, in two steps.

• At creation it takes one percent of what the storage can hold, floored at 2Gi and rounded up to a whole GiB. Replication is divided out — a copy is the same object and adds no entry — so a 2Ti × 3 storage with replication: "001" starts at 31Gi.
• Afterwards it follows what is actually stored. The volume servers report how many needles they hold, a needle is an entry, and the module keeps room for a kilobyte per object plus as much again for headroom. When the objects written outgrow the disk, the claim is expanded in place, exactly like the volume servers’ disks are — so the StorageClass has to allow expansion (allowVolumeExpansion: true).

On the Postgres metadata store the size goes into the managed PostgreSQL instance instead of a claim of this module’s, and the managed-postgres operator resizes the volume behind it. Both profiles get the same number, because both hold the same entries.

The size is reported in status.storage.metadataSize:

status:
  storage:
    sizePerNode: 50Gi
    metadataSize: 20Gi
    message: growing the metadata store to 20Gi for the 10485760 object(s) written so far (1 of 1 claim(s) still smaller)

It never shrinks. Deleting objects does not give the disk back — Kubernetes does not shrink a claim — and a count that comes back smaller, which is what a volume server restarting looks like, is not acted on at all.

On a StorageClass that does not allow expansion the storage keeps serving and says so on the object:

the metadata store needs 20Gi for the 10485760 object(s) written so far, and cannot grow:
StorageClass "local" does not allow volume expansion (allowVolumeExpansion is not true)

The ways out of that are a StorageClass that expands, or a new storage — neither of which the module can decide for you.

Metrics and alerts

Each storage exports the following metrics with the store_kind and store labels:

Capacity metrics are exported only after capacity data has been obtained successfully. If the data could not be obtained, the sds_object_store_capacity_bytes_* metrics are absent, so filling-up alerts do not fire based on an unknown value.

For SeaweedFS, status.capacity.nodes and the sds_object_store_node_capacity_bytes_* metrics expose every volume server separately. Filling-up alerts use these per-node values, because an aggregate percentage can hide one full server behind two empty ones. Backends without node-level capacity, such as Ceph, continue to use the store-wide values.

The sds_object_store_ready metric is exported at all times, as long as the corresponding storage exists. A value of 0 means the storage is not ready. The time series disappears once the resource or the module is deleted. This way, the not-ready state is distinguished from the resource not existing at all.

The following alerts are configured for these metrics: D8SdsObjectStoreNotReady — the storage stays not ready for more than 30 minutes; D8SdsObjectStoreFillingUp — over 85% of capacity used for an hour; D8SdsObjectStoreAlmostFull — over 90% of capacity used for 15 minutes. Storage state is also shown on the SDS Object — Stores dashboard.

The 90% threshold accounts for SeaweedFS behavior: the master stops placing new volumes on a disk at that point. Existing volumes can still accept writes until the filesystem itself fills, but a bucket that needs a new volume cannot grow.

Data integrity

Storage status carries details of the last integrity check: its source, when it ran, and any damage found.

Check the result of the last integrity check:

d8 k get seaweedfsstore,sdselasticstore -o custom-columns='NAME:.metadata.name,INTEGRITY:.status.conditions[?(@.type=="IntegrityHealthy")].status,LAST SCRUB:.status.integrity.lastScrubTime,DAMAGED:.status.integrity.damaged'

Before the first check completes, the IntegrityHealthy condition is Unknown. This distinguishes the absence of check results from a confirmed damage-free state. When a problem is found, the condition becomes False, and its message carries the backend’s own diagnostic details, including which disk to check.

The Ready state does not depend on IntegrityHealthy. Storage can keep serving the data that is still available even after a damaged volume is found.

Check mechanism and frequency

Each backend checks integrity differently:

• SeaweedFS — SeaweedFS verifies a checksum whenever an object is read. For data that applications have not accessed in a long time, the module schedules a full integrity check (scrub): SeaweedFS reads the stored data and verifies its checksums.

  Configure the periodic check in spec.integrity:

  apiVersion: storage.deckhouse.io/v1alpha1
  kind: SeaweedFSStore
  metadata:
    name: media
  spec:
    integrity:
      interval: 24h     # The default is 168h; values below 1h are raised to 1h.
      mode: Full        # Full is the default and reads every byte; Index checks only indexes.
      enabled: true
    # The rest of spec follows.

  To start an extra scrub without changing the periodic schedule, set the annotation to the current RFC3339 time or Unix timestamp:

  d8 k annotate seaweedfsstore my-store storage.deckhouse.io/scrub="$(date -u +%FT%TZ)" --overwrite

  The request remains effective across controller restarts until a scrub completes. It also works when periodic scrubbing is disabled with spec.integrity.enabled: false; autoRepair applies to this requested pass in the same way as to a scheduled pass.

  Full is the default because Index cannot detect corruption in the object content. The default interval accounts for the load: a full check reads every stored byte.

  The full check runs asynchronously, outside of reconcile. While it runs, IntegrityHealthy carries the reason ScrubInProgress. The result is written to status when the check completes; on large storage, it can take a long time. Restarting the controller stops an in-progress check; the next one starts on schedule.

  An unreachable volume server is not treated as damaged. It appears in status.integrity.unreachable. If some nodes could not be checked and no damage was found, IntegrityHealthy stays Unknown with the reason ScrubIncomplete, since the check did not cover the whole storage.

  If storage has no volumes yet, a pass over it does not count as a completed data check. The condition stays Unknown with the reason NothingToCheck, and status.integrity is not populated. Once data appears, the storage is checked on schedule.

• Ceph RGW — Ceph performs checks on its own schedule. The module gets results from ElasticCluster health checks (OSD_SCRUB_ERRORS, PG_DAMAGED, PG_NOT_DEEP_SCRUBBED, …). SDSElasticStore has no spec.integrity: running deep-scrub and automatic repair are configured at the Ceph cluster level and are not managed by the sds-object module.

status.integrity.source shows the source of the published check result.

Replication status

A missing copy is not damage, so it is reported separately in the RedundancyHealthy condition and status.redundancy.

Check the replication state:

d8 k get seaweedfsstore -o custom-columns='NAME:.metadata.name,WANTED:.status.redundancy.copiesWanted,VOLUMES:.status.redundancy.volumes,SHORT:.status.redundancy.underReplicated'

Copy counts come from the SeaweedFS master’s own topology and are updated on every reconcile, independent of the integrity-check schedule. The controller continues this lightweight accounting while BackendReady is False; it does not start a full data scrub until the backend is ready again.

RedundancyHealthy: False combined with IntegrityHealthy: True means no damage was found, but the copy count is below what is required. With replication: "000", only one copy is required, so that storage is never considered under-replicated as long as that one copy exists.

SeaweedFS 4.39 does not restore a lost replica automatically. Its maintenance framework includes balance, vacuum, erasure coding, EC balance, S3 lifecycle, and Iceberg tasks, but no dedicated replication-repair task, even though the protocol has a task type for it. So after losing a replica, restore it by hand. First, list the pods in the module namespace and choose a pod running the SeaweedFS image:

d8 k -n d8-sds-object get pods -o custom-columns='NAME:.metadata.name,IMAGES:.spec.containers[*].image'

Open a shell in the selected pod:

d8 k -n d8-sds-object exec -it <SEAWEEDFS_POD> -- sh

Then run the replication repair from that shell:

weed shell -master=<STORE>-seaweedfs-master:9333 <<'EOF'
lock
volume.fix.replication -apply
unlock
EOF

For SDSElasticStore, the condition is Unknown with the reason NotAccountedHere. Replication status is controlled by Ceph itself and available through its own health checks; the sds-object module does not duplicate that accounting.

Automatic repair

By default, the module only reports damage it finds. If you set spec.integrity.autoRepair: true, the module attempts to repair a damaged SeaweedFS copy from another copy that passed its check:

spec:
  replication: "001"      # Repair requires more than one copy.
  integrity:
    autoRepair: true

Automatic repair is off by default, because the procedure deletes the damaged copy and creates a new one. Before repairing each volume, the module checks the following conditions:

• More than one copy of the volume exists. With replication: "000", the damaged copy is the only one, so automatic repair is not possible.
• Another copy of the same volume passed the integrity check in the current cycle. An unreachable or unchecked copy is not used as a repair source.

If either condition is not met, the module does not delete the damaged copy, and status.integrity.details states why repair was skipped. If creating a new copy after removing the damaged one fails, RedundancyHealthy reports the insufficient copy count.

Successfully repaired volumes are counted in status.integrity.repaired and no longer count as damaged. Once repair succeeds, the corresponding alert stops firing.

None of this applies to Ceph RGW: Ceph itself manages placement group repair. The module does not run deep-scrub or enable osd_scrub_auto_repair.

Metrics and alerts

Integrity and replication status are also exported as metrics with the store_kind and store labels:

The following alerts are configured for these metrics:

• D8SdsObjectStoreDataDamaged — the last completed check found damage. Fires with no additional delay.
• D8SdsObjectStoreUnderReplicated — the copy count has been below what is required for 15 minutes. The delay rules out brief flaps during a planned volume-server restart.
• D8SdsObjectStoreEncryptionKeyWithheld — the encryption key value changed, but the module is not applying the new key, so previously written data does not become unreadable after the gateway restarts.
• D8SdsObjectStoreNotScrubbed — more than three spec.integrity.interval periods have passed since the last completed check. This catches the case where the periodic check has stopped running even though the last saved IntegrityHealthy result is still successful. If the periodic check is disabled, the interval metric is not published and this alert does not apply.

The Grafana dashboard SDS Object — Data integrity, in the Storage folder, shows the number of storage resources with damage, insufficient replication, and overdue checks, along with each storage’s state and time since its last integrity check.

Encryption at rest

Without spec.encryption, object content is stored on the volume servers without server-side encryption. Anyone who gets access to the disk can read that data. Encryption can be enabled without any changes to the applications writing the objects.

Create a Secret with the encryption key material:

# Generate 32 bytes of key material. Keep a backup copy of the key outside the cluster.
d8 k -n d8-sds-object create secret generic media-sse --from-literal=kek="$(head -c 32 /dev/urandom | xxd -p -c 32)"

An example of creating a SeaweedFSStore with server-side encryption enabled and the Secret referenced:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: SeaweedFSStore
metadata:
  name: media
spec:
  storage:
    class: linstor-thin-r2
  encryption:
    mode: ServerManaged
    keySecretRef:
      name: media-sse
EOF

Instead of kek, the Secret can hold a key field with a passphrase the module derives the key from. This is useful when the key material comes from a system that hands out strings rather than raw bytes.

Once encryption is enabled, the module gives the S3 gateway the wrapping key and turns on server-side encryption for its buckets. Each object is encrypted with its own data key, which is protected by the key from the Secret. The wrapping key is never stored in the filer’s metadata store, so access to the Postgres database or the filer’s PVC alone does not let anyone decrypt the objects’ contents.

Check the encryption state:

d8 k get swfsstore media -o jsonpath='{.status.encryption}' | jq

Example output:

{
  "mode": "ServerManaged",
  "keyFingerprint": "sha256:9c1f4a0b7e2d8536",
  "since": "2026-08-25T09:12:04Z"
}

Changing the encryption key

Objects already written are not re-wrapped automatically. If the gateway starts with a different wrapping key, reading objects encrypted under the previous key fails with an internal server error. To prevent losing access to data, the module compares the key fingerprint and refuses to apply a changed key without explicit confirmation.

Check the EncryptionActive condition message:

d8 k get swfsstore media -o jsonpath='{.status.conditions[?(@.type=="EncryptionActive")]}' | jq -r .message

Example output:

the wrapping key changed (sha256:9c1f4a0b7e2d8536 -> sha256:22ba07f6c4e1d980) and was NOT applied: ...

While the condition is False, the gateway keeps using the previous key, under which previously written data stays readable. To return to a normal state, restore the original key material. If the data encrypted under the previous key is genuinely no longer needed, you can confirm the new key instead:

d8 k annotate swfsstore media storage.deckhouse.io/encryption-key-change-acknowledged=sha256:22ba07f6c4e1d980

The gateway reads the key at startup. So changing the Secret by itself does not affect an already running pod, but without this safeguard, the new key would be applied at the next restart, and previously encrypted objects could become unreadable.

Once spec.encryption.mode is enabled, server-side encryption cannot be turned off for existing storage. Previously encrypted objects still need the original key. To move to storage without server-side encryption, create new storage and migrate the data to it.

Encryption limitations and SSE-C

Server-side encryption has the following limitations and can be combined with client-side encryption:

• Object names, sizes, and tags are not encrypted — only the content is.
• Objects written before encryption was enabled are not encrypted automatically. To encrypt them, upload the data again after enabling encryption.
• A client can use its own key (SSE-C). The key is sent with every request and is not stored by the storage. SSE-C does not depend on spec.encryption. Use SSE-C only through a published TLS endpoint: on the internal HTTP endpoint, the key travels in the header without transport encryption.
• SSE-KMS is not supported for SeaweedFSStore. The gateway reads KMS configuration only from a static credentials file, which is incompatible with the keys the module creates for each BucketAccess. For external KMS, use SDSElasticStore (Ceph RGW).

Encryption in Ceph RGW

For SDSElasticStore, server-side encryption requires an external key management system (KMS). Ceph RGW gets keys from the KMS and does not support handing a key directly to the RGW process. Deckhouse Stronghold with its transit engine can be used as the KMS.

Create a Secret with the Stronghold token:

# Token that may only read and use the transit key.
d8 k -n d8-sds-object create secret generic rgw-stronghold-token --from-literal=token="$STRONGHOLD_TOKEN"

An example of creating an SDSElasticStore with the Secret and Stronghold address referenced:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: SDSElasticStore
metadata:
  name: heavy
spec:
  elasticClusterRef: main
  encryption:
    mode: ServerManaged
    stronghold:
      address: https://stronghold.d8-stronghold.svc.cluster.local:8200
      tokenSecretRef:
        name: rgw-stronghold-token
      # For Stronghold with a private certificate, configure a CA Secret.
      # caSecretRef:
      #   name: stronghold-ca
EOF

Keep the following in mind when configuring KMS for Ceph RGW:

• Stronghold is used through the Vault API. Rook gets KMS_PROVIDER: vault and VAULT_ADDR, because Stronghold implements the Vault API that Ceph RGW speaks. Any other Vault-compatible store can work the same way.
• The transit mount path is not configurable. For SSE-S3, Rook builds the RGW prefix from a single engine name (/v1/transit), so a transit engine mounted elsewhere would be accepted here and never used. Mount it as transit.
• The token is copied to the d8-sds-elastic namespace. Rook expects a Secret with the token next to the CephObjectStore, not next to the storage object, so the module keeps a managed copy there. Deleting the storage also deletes the copy. After the source token is rotated, the copy is updated on the next reconcile. The CA bundle is copied the same way: the ca.crt field is converted to the cert field Rook expects.
• No key fingerprint is kept in status.encryption. The key itself lives in Stronghold and is never handed to the module, so the module cannot compute its fingerprint or detect a key change. The availability of previously encrypted data depends on the key surviving in Stronghold.

status.encryption.message and the EncryptionActive condition reflect errors applying the encryption configuration in RGW, most often a missing token Secret. For existing storage, the rest of reconcile continues. New storage with encryption enabled is not created until the KMS configuration is ready, so the first objects are never written unencrypted.

Automatic object deletion

To automatically remove outdated logs, exports, or build artifacts, use spec.lifecycle. Lifecycle rules are applied by the S3 backend itself. For every SeaweedFSStore, the module runs a singleton lifecycle worker that processes all 16 SeaweedFS shards every five minutes; without this worker, storing a lifecycle configuration alone does not delete expired objects.

An example of creating a Bucket with deletion rules in spec.lifecycle:

d8 k apply -f - <<EOF
apiVersion: storage.deckhouse.io/v1alpha1
kind: Bucket
metadata:
  namespace: team-a
  name: build-logs
spec:
  objectStoreRef: standard
  lifecycle:
    rules:
      - id: logs
        prefix: logs/
        expireAfterDays: 30
      - id: leftovers
        abortIncompleteUploadsAfterDays: 1
EOF

Every rule must have a unique id. This identifier is passed to the S3 backend and used to match rules when the configuration is updated. It is set explicitly and does not depend on the rule’s position in the list.

• expireAfterDays deletes an object this many days after it was written. On a versioned bucket, this only makes the current version noncurrent — the data stays, and expireNoncurrentAfterDays frees the space.
• expireNoncurrentAfterDays deletes a version this many days after it stopped being current.
• abortIncompleteUploadsAfterDays removes parts of multipart uploads that never completed, after the given number of days. These parts take up space but do not show up as regular objects.
• prefix limits the rule to objects whose keys start with the given prefix. If prefix is not set, the rule applies to every object in the bucket.

On every reconcile, the module builds the full set of lifecycle rules from spec and sends it to the backend. If a rule is removed from the Bucket, it is also removed from the bucket’s configuration in the backend.

Object Lock takes priority over lifecycle deletion. If a version has an active retention period, a lifecycle rule cannot delete it before that period ends.

Transitions between storage classes are not supported. The current lifecycle API only supports deletion operations. SeaweedFS 4.47 supports expiration by days and by date, deleting noncurrent versions, newer-noncurrent limits, aborting incomplete uploads, and cleaning up delete markers, but does not support the Transition action. Ceph RGW can move objects between storage classes, but a transition parameter common to both backends is not available yet.

S3 operation support

SeaweedFS and Ceph RGW support different sets of S3 operations. Before relying on an application feature, check that it is supported by the backend you chose.

The table is based on the S3 API implementation in SeaweedFS 4.39 and on the support matrix of the Ceph RGW version used by the sds-elastic module. These are the backends’ own capabilities, not an additional implementation by the sds-object module. The table needs to be re-checked whenever the backend versions are updated.

Partially implemented SeaweedFS operations. Some S3 endpoints respond successfully but do not persist any configuration. For example, the accelerate endpoint always returns Suspended, bucket logging returns an empty status, and the analytics, inventory, intelligent-tiering, and metrics listings all come back empty. A 200 response for these operations confirms the request was handled, not that the corresponding feature exists.

What the module manages, and what passes through. The module manages bucket creation, bucket policy for accessPolicy, quotas, versioning, Object Lock, and default encryption. Every other S3 operation runs directly between the application and the backend; the module neither configures nor tracks it.

If a requested feature is not supported by the chosen backend, both the tenant’s Bucket and its backing BucketContents reflect that in the informational FeaturesApplied condition. It does not gate Ready.

Example FeaturesApplied condition state:

FeaturesApplied   False   Unsupported
  backend SeaweedFS does not enforce: spec.quota.maxObjects

The table above describes the backends’ general capabilities, while the FeaturesApplied condition shows the result of applying the requested settings to a specific bucket.

SeaweedFS enforces spec.quota.maxSize asynchronously: a leader-held backend loop updates the bucket’s read-only state about once a minute. A fast writer can exceed the requested size by what it uploads during that interval. When the quota becomes active, SeaweedFS currently returns S3 500 InternalError for writes instead of a quota-specific 4xx response; SDK retries can therefore amplify the delay. Dropping below the limit re-enables writes on the same schedule. SeaweedFS does not enforce spec.quota.maxObjects, which is why that field appears in FeaturesApplied while maxSize does not.

Diagnostics

Every object carries an overall state in status.phase, with the detailed reason for the current state in status.conditions:

• Pending — the controller has not processed the object yet, or is waiting on a dependent stage that has not reported anything yet.
• InProgress — processing is underway: the stage has started but has not yet confirmed success.
• Ready — every stage confirmed success. Bucket data, BucketAccess credentials, and the storage endpoint are current only in this phase.
• Error — one of the processing stages failed; the reason is in the failing condition’s message.
• Released (BucketContents only) — the owning Bucket was deleted with reclaimPolicy: Retain; the data is preserved and will be linked again to a Bucket with the same name in the same namespace.

To see the details, run:

d8 k get bucket app-data -n my-app -o jsonpath='{.status.conditions}'

Common messages, where they appear, and what to do: