Unsealing and keys
Why is a node sealed after a restart?
With Shamir seal, this is expected: the decryption key is kept only in memory. Unseal the node with d8 stronghold operator unseal or configure auto unseal via HSM or KMS. See Troubleshooting.
Do I need to unseal every cluster node?
Yes. With Shamir seal, each node is unsealed separately after every start. With auto unseal, nodes are unsealed automatically. See Seal.
How do I change unseal key holders?
Rekey with a new number of shares and the new holders’ PGP keys. See Key management.
Do I need to keep the root token?
No. Revoke the root token after the initial setup and generate a new one only for emergency operations. In DP Automatic mode, the root token is stored in the stronghold-keys secret and used by stronghold-automatic, so do not revoke it. How to reduce the risks of this secret is described in Production checklist. Root token generation is described in Key management.
What if the root token is lost?
Generate a new one with d8 stronghold operator generate-root using the threshold number of unseal or recovery keys. See Generating a root token.
How often should the encryption key be rotated?
Configure automatic rotation via sys/rotate/config by interval or number of operations, and rotate out of schedule when compromise is suspected. See Rotating the encryption key.
Backup and restore
How do I back up Stronghold?
For integrated Raft storage, take a snapshot with d8 stronghold operator raft snapshot save. See Creating a snapshot and Automated snapshots.
Is a snapshot enough to restore?
No. To unseal a restored cluster, you need the unseal or recovery keys (or HSM/KMS access) that were valid when the snapshot was taken. See Restoring from a snapshot.
What if most Raft nodes are lost?
Recover the quorum following the guide for Linux or DP.
Monitoring and logs
How do I connect Prometheus?
Enable telemetry with prometheus_retention_time and scrape /v1/sys/metrics?format=prometheus. See Monitoring.
Which endpoint should load balancer checks use?
Use /v1/sys/health: it requires no authentication and returns different codes for active, standby, and sealed nodes. See Health checks via sys/health.
How are server logs different from audit logs?
Server logs describe the Stronghold process itself; audit logs record client requests and responses. See Server logs and Audit in Stronghold.
How do I enable debug logs without a restart?
Use the /sys/loggers endpoint or send SIGHUP after changing log_level. See Changing the log level.
Performance and load
Why does response time grow?
A frequent cause is a large number of leases and tokens or a slow Raft disk. See Slow responses and lease explosion.
How do I protect the cluster from an application that generates excessive load?
Configure rate limit quotas for an auth method, mount, or role.
How many nodes does a cluster need?
For production, 3 or 5 nodes. See Sizing.
Why do requests hang when audit has problems?
Stronghold does not serve a request if no audit device can record it. Use at least two audit devices. See Audit device blocks requests.
Upgrades and certificates
How do I upgrade a Stronghold cluster in Linux?
Take a snapshot, upgrade standby nodes one at a time, then hand over the active role with d8 stronghold operator step-down and upgrade the former active node. See Upgrade.
How do I replace a TLS certificate without downtime?
Replace the files at the same paths and send SIGHUP to the process. See TLS certificates.
How is Stronghold updated in DP?
The module is updated together with Deckhouse according to the release channel. See Platform update.
Security
What should I check before going to production?
Go through the production checklist.
What should I do if a token is compromised?
Revoke the token and leases by prefix, and review the audit logs. See Responding to compromise.