Key management

This page describes routine and emergency operations with Stronghold keys. For the key model, see Seal.

Keys in DP

In DP Automatic mode, initialization creates a single key share with a threshold of 1. The unseal key and the root token are stored in the stronghold-keys secret in the d8-stronghold namespace, in the unsealKey and rootToken fields. The stronghold-automatic Deployment uses them:

• unsealKey to unseal the nodes. The check period is 60 seconds by default.
• rootToken to create the role of the Kubernetes auth method, which stronghold-automatic uses to configure Stronghold, when it cannot log in with this role.

If unsealKey is missing from the secret or does not match the current unseal key, automatic unsealing stops working: after a restart, unseal each pod manually. If rootToken is missing or revoked, stronghold-automatic cannot restore its access and stops configuring Stronghold.

To view the stronghold-automatic logs, run the following command:

d8 k -n d8-stronghold logs deploy/stronghold-automatic

In Manual mode, the module creates neither the secret nor stronghold-automatic.

Rekeying unseal keys

Rekey when key holders change, when you change the number of shares or the threshold, or when a share is suspected to be compromised.

1. Initialize the operation with the new number of shares and threshold. In this example, the new shares are encrypted with the holders’ PGP keys, so only the holder of a share can read it. The -verify flag requires the holders to confirm receipt of the new shares before the old ones become invalid:

   d8 stronghold operator rekey -init -key-shares=5 -key-threshold=3 -pgp-keys="alice.asc,bob.asc,carol.asc,dave.asc,erin.asc" -verify

   The command returns the operation Nonce. Share it with the holders of the current shares.

2. Each current share holder runs the command and enters their share:

   d8 stronghold operator rekey -nonce=<NONCE>

3. Once the threshold is reached, Stronghold outputs the new shares (PGP-encrypted, if keys were provided) and a Verification Nonce.

4. Confirm receipt of the new shares: the threshold number of holders enter the new shares:

   d8 stronghold operator rekey -verify -nonce=<VERIFICATION_NONCE>

   The old shares remain valid until verification completes.

5. Distribute the new shares to the holders and destroy the old ones.

Check progress or cancel the operation:

d8 stronghold operator rekey -status
d8 stronghold operator rekey -cancel

Rekeying recovery keys

With auto unseal via HSM or KMS, recovery keys are used instead of unseal keys. The procedure is the same; the operation is authorized with the current recovery keys:

d8 stronghold operator rekey -init -target=recovery -key-shares=5 -key-threshold=3
d8 stronghold operator rekey -target=recovery -nonce=<NONCE>

In the API, use the /sys/rekey-recovery-key prefix.

Rotating the encryption key

Rotation adds a new encryption key to the keyring. New data is encrypted with the new key; older keys stay in the keyring to decrypt previously written data. The operation causes no downtime and requires a token with sudo on sys/rotate.

d8 stronghold operator rotate

Check the current key term and install time:

d8 stronghold operator key-status

Configure automatic rotation via /sys/rotate/config:

d8 stronghold write sys/rotate/config enabled=true interval=720h max_operations=3000000000

• interval — how long after the active key is installed to rotate it;
• max_operations — how many encryption operations to perform before rotating.

interval is a duration string (720h) or a number of seconds, with a minimum of 24 hours. max_operations accepts values from 1,000,000 to 3,865,470,566; if not set, the maximum applies.

Generating a root token

A root token is needed only for emergency operations when administrative access through auth methods is impossible. Generation requires the threshold number of unseal or recovery key shares.

With a one-time password (OTP)

1. Initialize the operation. Stronghold outputs a Nonce and an OTP. Save the OTP: it is required to decode the token:

   d8 stronghold operator generate-root -init

2. Each holder enters their share:

   d8 stronghold operator generate-root -nonce=<NONCE>

3. After the last share, Stronghold outputs an Encoded Token. Decode it:

   d8 stronghold operator generate-root -decode=<ENCODED_TOKEN> -otp=<OTP>

With a PGP key

Instead of an OTP, the token can be encrypted with the recipient’s PGP key:

d8 stronghold operator generate-root -init -pgp-key=operator.asc

After the shares are entered, decrypt the Encoded Token:

echo "<ENCODED_TOKEN>" | base64 -d | gpg --decrypt

Check progress or cancel the operation:

d8 stronghold operator generate-root -status
d8 stronghold operator generate-root -cancel

Revoking the root token

Revoke the root token after emergency operations:

d8 stronghold token revoke <ROOT_TOKEN>

Or, if the root token is used in the current session:

d8 stronghold token revoke -self

Make sure working administrative access through auth methods is configured before revocation.

Responding to compromise

Compromised unseal or recovery key share

One share is not enough to unseal when the threshold is greater than one, but the safety margin is reduced.

1. Rekey with the remaining holders.
2. If the threshold number of shares may be exposed and storage data may be accessed, rekey and rotate the encryption key, then review audit logs for suspicious operations.
3. Assess whether Raft snapshots may have been copied: a snapshot together with the threshold number of shares gives access to the data.

Compromised root token

1. Revoke the token immediately. If the token value is unknown, use its accessor from the audit log:

   d8 stronghold token revoke -accessor <ACCESSOR>

   Child tokens and leases created by this token are revoked automatically. Orphan tokens are not revoked: find them in the audit log and revoke them separately.

2. Review audit logs for operations performed with this token: changes to policies, auth methods, audit devices.

3. Rotate static secrets that may have been read.

Compromised regular token

1. Revoke the token:

   d8 stronghold token revoke <TOKEN>

2. If secrets issued through a specific path are compromised, revoke leases by prefix:

   d8 stronghold lease revoke -prefix database/creds/app-role

   The -force flag removes leases from Stronghold even if revocation fails in the external system. Use it only if the credentials have been removed from the external system manually.

3. For auth methods, revoke all tokens they issued by prefix, for example d8 stronghold lease revoke -prefix auth/approle/, and rotate the credentials (for example, the AppRole secret_id).

For more on leases, see Leases; for tokens, see Tokens.

Lost keys

Lost unseal key shares

If the holders still have the threshold number of shares, rekey to restore the required number of shares.

If fewer shares than the threshold remain, the data cannot be decrypted after the next seal or restart:

1. Do not restart or seal the running nodes.
2. While the cluster is unsealed, move secrets and configuration to a new cluster with new keys.

Lost recovery keys

With auto unseal, the cluster keeps unsealing automatically, but operations requiring recovery keys become impossible: root token generation, recovery key rekey, seal migration.

1. Keep working administrative access through auth methods.
2. Plan migrating the data to a new cluster with new recovery keys.

Stronghold cannot reissue recovery keys without the threshold number of current shares: the recovery key rekey operation reconstructs the key from the submitted shares and does not complete without them.