Auto-unseal and seal wrap with an HSM

Stronghold EE encrypts the root key with a key that is stored in an HSM and never leaves it. The HSM is accessed through the standard PKCS #11 interface. After a restart, Stronghold unseals automatically, and the most sensitive data (the keyring, the recovery key, and PKI, SSH, and Transit keys) is additionally encrypted through the same seal: the seal wrap mechanism.

Goal

Build a SoftHSM2 lab, configure seal "pkcs11", initialize Stronghold with recovery keys, verify auto-unseal and seal wrap, and then prepare to move the configuration to a hardware HSM.

Prerequisites

To follow this example, you need:

• Stronghold EE installed on Linux as described in Installation. In the examples, the configuration is in /opt/stronghold/config.hcl, and the stronghold service runs as the stronghold user.
• For the lab: Debian or Ubuntu with the softhsm2 and opensc packages (the pkcs11-tool utility).
• For production: an HSM with PKCS #11 support (for example, Rutoken ECP 3.0 or JaCarta) and the vendor PKCS #11 library.
• To move an existing server from Shamir: the threshold number of unseal keys and a data backup.

seal “pkcs11” parameters

The seal "pkcs11" stanza takes the following parameters:

Parameters can also be set with VAULT_HSM_* environment variables (for example, VAULT_HSM_PIN, VAULT_HSM_LIB) or PKCS11_WRAPPER_*; environment values take precedence over the configuration.

Step 1. Prepare a SoftHSM2 token

Perform this step on the Stronghold node.

1. Install the packages:

   sudo apt install softhsm2 opensc

2. Create a token directory and a SoftHSM2 configuration accessible to the stronghold user:

   sudo mkdir -p /opt/stronghold/softhsm/tokens
   echo "directories.tokendir = /opt/stronghold/softhsm/tokens" | sudo tee /opt/stronghold/softhsm/softhsm2.conf
   sudo chown -R stronghold:stronghold /opt/stronghold/softhsm
   sudo chmod 0700 /opt/stronghold/softhsm/tokens

3. Set the variables:

   export SOFTHSM2_CONF=/opt/stronghold/softhsm/softhsm2.conf
   HSMLIB=/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so

4. Initialize the token and set the PINs. Run the commands as the stronghold user so that the token files belong to it:

   sudo -u stronghold SOFTHSM2_CONF=$SOFTHSM2_CONF pkcs11-tool --module $HSMLIB --init-token --so-pin 1234 --init-pin --pin 4321 --label stronghold_token --login

5. Create an RSA key pair:

   sudo -u stronghold SOFTHSM2_CONF=$SOFTHSM2_CONF pkcs11-tool --module $HSMLIB --login --pin 4321 --keypairgen --key-type rsa:4096 --label stronghold-rsa-key

   In the output, the private key must have the sensitive, always sensitive, never extractable, local attributes: the key cannot be extracted from the token.

6. Check the token and keys:

   sudo -u stronghold SOFTHSM2_CONF=$SOFTHSM2_CONF pkcs11-tool --module $HSMLIB --login --pin 4321 --list-objects

Step 2. Configure Stronghold

Add the HSM seal to the Stronghold configuration:

1. Add a seal stanza to /opt/stronghold/config.hcl:

   seal "pkcs11" {
     lib           = "/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so"
     token_label   = "stronghold_token"
     pin           = "4321"
     key_label     = "stronghold-rsa-key"
     rsa_oaep_hash = "sha1"
   }

   Restrict permissions on the configuration file because it contains the PIN:

   sudo chown stronghold:stronghold /opt/stronghold/config.hcl
   sudo chmod 0600 /opt/stronghold/config.hcl

   The PIN can be passed through the VAULT_HSM_PIN or PKCS11_WRAPPER_PIN environment variable instead of the pin parameter in the configuration.

2. Pass the SoftHSM2 configuration path to the service through a systemd drop-in:

   sudo mkdir -p /etc/systemd/system/stronghold.service.d
   sudo tee /etc/systemd/system/stronghold.service.d/softhsm.conf <<'UNIT'
   [Service]
   Environment=SOFTHSM2_CONF=/opt/stronghold/softhsm/softhsm2.conf
   UNIT
   sudo systemctl daemon-reload

For a new server, go to step 3. For a server already initialized with Shamir, go to step 4.

Step 3. Initialize a new server

Start and initialize a new server with the HSM seal:

1. Start Stronghold:

   sudo systemctl start stronghold

2. Initialize Stronghold with recovery keys:

   d8 stronghold operator init -recovery-shares=5 -recovery-threshold=3

   Distribute the recovery keys among key holders and store the root token.

Step 4. Move an existing server from Shamir to the HSM

Migrate an initialized server from Shamir to the HSM seal:

1. Take a Raft snapshot:

   d8 stronghold operator raft snapshot save pre-seal-migration.snap

2. Add the seal "pkcs11" stanza (step 2) and restart Stronghold:

   sudo systemctl restart stronghold

   The log shows the message:

   core: entering seal migration mode; Stronghold will not automatically unseal even if using an autoseal: from_barrier_type=shamir to_barrier_type=pkcs11

3. Unseal with the -migrate flag, entering the threshold number of unseal keys:

   d8 stronghold operator unseal -migrate

4. Wait until the seal-wrapped entries are re-wrapped: the log must show the seal re-wrap completed message.

For an HA cluster, follow the general procedure in Seal migration.

Step 5. Check seal wrap

For supported seals, double encryption is enabled by default, and no separate configuration is needed. Check the rewrap status:

d8 stronghold read sys/sealwrap/rewrap

The response contains the process status and counters of processed entries. To force re-wrapping of all seal-wrapped entries, for example after changing the key in the HSM, start a rewrap:

d8 stronghold write -f sys/sealwrap/rewrap

To disable double encryption for all data except the root key, set disable_sealwrap = true in the server configuration. For details, see Double encryption.

Verification

Check the result:

1. Check the status:

   d8 stronghold status

   Expected values:

   Seal Type                pkcs11
   Recovery Seal Type       shamir
   Initialized              true
   Sealed                   false

2. Restart the service and make sure Stronghold unseals automatically:

   sudo systemctl restart stronghold
   sleep 10
   d8 stronghold status -format=json | jq '{type, sealed, recovery_seal}'

3. Check that Stronghold does not unseal without access to the HSM. In the lab, temporarily rename the token directory:

   sudo systemctl stop stronghold
   sudo mv /opt/stronghold/softhsm/tokens /opt/stronghold/softhsm/tokens.off
   sudo systemctl start stronghold
   d8 stronghold status
   journalctl -u stronghold.service --since "5 minutes ago" | grep -i pkcs11
   sudo systemctl stop stronghold
   sudo mv /opt/stronghold/softhsm/tokens.off /opt/stronghold/softhsm/tokens
   sudo systemctl start stronghold

   While the token is unavailable, Sealed must be true, and the log must contain a PKCS #11 error.

Moving to a hardware HSM

When you move from SoftHSM2 to a hardware HSM, take the following into account:

• Library and key. Use the vendor PKCS #11 library and create a key pair in the HSM with the vendor tools or with pkcs11-tool. An example for Rutoken ECP 3.0 is in HSM support. Choose the rsa_oaep_hash value according to the mechanisms the device supports.
• Seal change. Moving from SoftHSM2 to a hardware HSM is a migration between two auto seals: add disabled = "true" to the old seal "pkcs11" stanza, add the new stanza, and run d8 stronghold operator unseal -migrate with the recovery keys. The procedure is described in Seal migration.
• HA cluster. Each node needs access to a key with the same label. The key material must be the same on all nodes: use a shared (network) HSM or clone the key between devices with the vendor tools. Stronghold does not clone keys itself, and a key generated separately on each device (generate_key) does not suit an HA cluster.
• Key backup. Losing the key in the HSM makes Stronghold data impossible to decrypt. Use the key backup procedures provided by the HSM vendor.
• Availability. With seal wrap, the HSM is needed not only for unsealing but also at runtime: operations on seal-wrapped values go through the HSM and are slower than usual. Check that the connection to the device is stable.
• PIN. Store the PIN in the configuration file with 0600 permissions or pass it through an environment variable of the service.

Cleanup

For a lab environment:

1. Stop Stronghold: sudo systemctl stop stronghold.
2. Delete the Stronghold data, the SoftHSM2 token (/opt/stronghold/softhsm), and the /etc/systemd/system/stronghold.service.d/softhsm.conf drop-in, then run sudo systemctl daemon-reload.
3. Remove the seal "pkcs11" stanza from the configuration.

Additional resources

• HSM support
• Double encryption
• Seal and unseal
• Standalone configuration templates