A web server on a virtual machine does not need a certificate issued manually for a year. Stronghold Agent requests a certificate from the PKI secrets engine, writes it to disk, reissues it before it expires, and reloads Nginx. The private key is generated in Stronghold and is passed only to Agent on the VM.

Web server TLS certificate issuance and renewal flow

Goal

Configure Stronghold Agent on a VM so that it issues a 72-hour www.example.com certificate for Nginx from the pki_int intermediate CA, reissues it automatically, and reloads Nginx without dropping connections.

Prerequisites

Before you start, prepare the following:

  • An intermediate CA in Stronghold mounted at pki_int, as in the Internal PKI with Stronghold guide.
  • A Linux virtual machine with systemd and Nginx.
  • Stronghold Agent installed on the VM and logging in to Stronghold with AppRole, as in the Application on a VM with Stronghold Agent guide.
  • A Stronghold token with permissions to configure PKI, AppRole, and policies.

Step 1. Create a PKI role

The role restricts the names and lifetime of certificates that the web server can obtain:

d8 stronghold write pki_int/roles/web-server allowed_domains="example.com" allow_subdomains=true allow_bare_domains=true max_ttl=720h
  • allow_subdomains=true allows names in the example.com domain, such as www.example.com.
  • allow_bare_domains=true allows the example.com name itself. Without this parameter, Stronghold rejects a request that contains example.com in alt_names.

A short certificate lifetime reduces the impact of a key compromise: Agent renews the certificate automatically, so manual reissuance is not needed.

Step 2. Create a policy and an AppRole role

Grant the VM access only to certificate issuance and deliver its AppRole credentials.

  1. Create a policy that allows issuing certificates only through the web-server role:

    d8 stronghold policy write web-server-tls - <<'POLICY'
path "pki_int/issue/web-server" {
  capabilities = ["update"]
}
POLICY

  2. Create an AppRole role for the VM:

    d8 stronghold write auth/approle/role/web-server token_policies=web-server-tls token_ttl=1h token_max_ttl=24h secret_id_ttl=720h secret_id_bound_cidrs="10.0.10.20/32"

  3. Deliver role_id and a wrapped secret_id of the web-server role to the /etc/stronghold-agent/ directory on the VM, as described in step 3 of the Application on a VM with Stronghold Agent guide. In the commands of that step, replace myapp-vm with web-server.

Step 3. Create a template

Create the /etc/stronghold-agent/templates/www.pem.ctmpl file. The template writes the private key, the certificate, and the chain of intermediate CAs into a single file, so the key and the certificate always match each other:

{{ with secret "pki_int/issue/web-server" "common_name=www.example.com" "alt_names=example.com" "ttl=72h" }}
{{ .Data.private_key }}
{{ .Data.certificate }}
{{ range .Data.ca_chain }}{{ . }}
{{ end }}{{ end }}

Each secret "pki_int/issue/..." call issues a certificate with a new key. If you need separate files for the certificate and the key, as in the example from the Key features section, use a call with identical arguments in both templates.

Identical calls (same path and same arguments) are executed by Agent once and the result is reused across all templates, so the key and the certificate in separate files form a pair.

The pkiCert template function is an alternative to secret. It reads the certificate from the template destination file and requests a new certificate only when the file is missing or the certificate in it is about to expire. As a result, an Agent restart does not issue a new certificate. The function returns the .Cert, .Key, .CA, and .CAChain fields, which the template writes to the destination file.

Step 4. Configure Agent

Configure Agent to issue the certificate and reload Nginx, and grant Agent the required permissions.

  1. Create the /etc/stronghold-agent/agent.hcl file:

    stronghold {
  address = "https://stronghold.example.com"
}

auto_auth {
  method "approle" {
    mount_path = "auth/approle"
    config = {
      role_id_file_path                   = "/etc/stronghold-agent/role-id"
      secret_id_file_path                 = "/etc/stronghold-agent/secret-id"
      secret_id_response_wrapping_path    = "auth/approle/role/web-server/secret-id"
      remove_secret_id_file_after_reading = true
    }
  }
}

template {
  source               = "/etc/stronghold-agent/templates/www.pem.ctmpl"
  destination          = "/etc/nginx/tls/www.pem"
  perms                = "0600"
  command              = "/usr/bin/systemctl reload nginx"
  command_timeout      = "30s"
  error_on_missing_key = true
}
    • secret_id_response_wrapping_path: the creation path of the wrapped secret_id of the web-server role. Agent checks this path before it unwraps secret_id.
    • perms = "0600": the file contains the private key. The Nginx master process reads it with root privileges.
    • command: runs after each write of the file. On reload, the Nginx master process checks the new configuration, rereads the certificate, and does not drop current connections. If the configuration contains an error, Nginx continues to work with the previous configuration.

  2. Allow the stronghold-agent user to reload only nginx.service. The Agent unit runs as the stronghold-agent user with NoNewPrivileges=true, so sudo does not work in command. Create the /etc/polkit-1/rules.d/50-stronghold-agent-nginx.rules file:

    polkit.addRule(function(action, subject) {
    if (action.id == "org.freedesktop.systemd1.manage-units" &&
        subject.user == "stronghold-agent" &&
        action.lookup("unit") == "nginx.service" &&
        action.lookup("verb") == "reload") {
        return polkit.Result.YES;
    }
});

    polkit reads the rules files automatically. The unit and verb details are passed by systemd 226 and later.

  3. Create a directory for the certificate:

    sudo install -d -o stronghold-agent -g root -m 0750 /etc/nginx/tls

  4. Allow Agent to write to /etc/nginx/tls and /etc/stronghold-agent. The Agent unit uses ProtectSystem=strict, so add these directories to ReadWritePaths in a drop-in file. Run sudo systemctl edit stronghold-agent and add the following lines:

    [Service]
ReadWritePaths=/etc/stronghold-agent /etc/nginx/tls

    The /etc/stronghold-agent directory must belong to the stronghold-agent user. Otherwise, Agent cannot delete the secret_id file after reading it.

Step 5. Configure Nginx

Specify the same file in ssl_certificate and ssl_certificate_key:

server {
    listen 443 ssl;
    server_name www.example.com example.com;

    ssl_certificate     /etc/nginx/tls/www.pem;
    ssl_certificate_key /etc/nginx/tls/www.pem;
    ssl_protocols       TLSv1.2 TLSv1.3;

    location / {
        root /var/www/html;
    }
}

Start Agent before the first start of Nginx with this configuration so that the certificate file already exists. Then start Nginx:

sudo systemctl restart stronghold-agent
sudo systemctl start nginx

If Nginx is already running, run sudo systemctl reload nginx instead of systemctl start nginx.

Step 6. How renewal works

A PKI certificate is issued without a renewable lease. Agent tracks the certificate validity period and, before it expires, runs pki_int/issue/web-server again, overwrites the file, and runs command. Choose the ttl in the template so that reissuance happens no more often than you are ready to reload Nginx and no later than you can notice a failure.

Agent reissues the certificate after about 90% of its remaining validity period has elapsed (with a random spread of ±5% so that clients do not arrive at the same time). For ttl = 24h this is roughly 21–22 hours.

Apache HTTP Server

For Apache, use the same template and the same template block with the following differences:

  • in the virtual host configuration, specify the file in the SSLCertificateFile and SSLCertificateKeyFile directives. Apache 2.4.8 and later reads intermediate certificates from SSLCertificateFile:

    <VirtualHost *:443>
    ServerName www.example.com
    SSLEngine on
    SSLCertificateFile    /etc/apache2/tls/www.pem
    SSLCertificateKeyFile /etc/apache2/tls/www.pem
</VirtualHost>

  • in destination, specify /etc/apache2/tls/www.pem, and in command, specify /usr/bin/systemctl reload apache2 or /usr/bin/systemctl reload httpd, depending on the distribution. On reload, Apache performs a graceful restart and does not drop current connections;

  • in the polkit rule, specify the apache2.service or httpd.service unit instead of nginx.service, and in ReadWritePaths, the /etc/apache2/tls directory instead of /etc/nginx/tls.

Verification

Check that Nginx serves the certificate issued by Stronghold and that the certificate is reissued.

  1. Check the Agent log — it should contain messages about rendering the template and running the command:

    sudo journalctl -u stronghold-agent -n 50

  2. Check the certificate that Nginx serves:

    openssl s_client -connect www.example.com:443 -servername www.example.com </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -serial -dates

    The issuer must match the pki_int intermediate CA, and the validity period must match the ttl from the template.

  3. Check the chain of trust against the root CA certificate:

    curl --cacert root-ca.pem -sSI https://www.example.com

  4. Check reissuance: temporarily set ttl=10m in the template, restart Agent, and make sure that the certificate serial number from verification step 2 changes without manual actions.

Cleanup

To remove the example configuration, follow these steps:

  1. Stop Agent and remove the template block and the template file.

  2. Delete the /etc/nginx/tls/www.pem file and remove the ssl_certificate directives from the Nginx configuration.

  3. Delete the /etc/polkit-1/rules.d/50-stronghold-agent-nginx.rules file. Remove /etc/nginx/tls from ReadWritePaths in the Agent drop-in file: run sudo systemctl edit stronghold-agent.

  4. If necessary, revoke the issued certificate and delete the roles and the policy:

    d8 stronghold write pki_int/revoke serial_number=<SERIAL_NUMBER>
d8 stronghold delete pki_int/roles/web-server
d8 stronghold delete auth/approle/role/web-server
d8 stronghold policy delete web-server-tls

    <SERIAL_NUMBER> is the certificate serial number from verification step 2, for example 39:dd:2e:....

Additional resources

←
Delivering secrets to applications / Terraform and Ansible
Delivering secrets to applications / Application clients
→