Quotas

Quotas protect Stronghold from being overloaded by individual clients: a misconfigured application that authenticates on every request or creates leases uncontrollably can degrade the availability of the entire cluster.

Stronghold supports two quota types:

• rate limit quotas — limit the number of requests per unit of time;
• lease count quotas — limit the number of leases that exist at the same time. Available in Stronghold EE.

Managing quotas requires a token with permissions on sys/quotas/*.

Rate limit quotas

A quota applies globally, to a namespace, to a mount, or to an auth method role. Quota parameters are described in the /sys/quotas/rate-limit/{name} API reference:

When a quota is exceeded, Stronghold returns HTTP code 429.

Examples

A global quota for the whole cluster:

d8 stronghold write sys/quotas/rate-limit/global rate=1000

A quota on the AppRole auth method to protect against login storms:

d8 stronghold write sys/quotas/rate-limit/approle-login path=auth/approle rate=50 interval=1s

A quota on a specific role that blocks the offender for a minute:

d8 stronghold write sys/quotas/rate-limit/ci-role path=auth/approle role=ci rate=10 interval=1s block_interval=60s

A quota on a secrets engine mount:

d8 stronghold write sys/quotas/rate-limit/kv-apps path=secret/ rate=200

A quota on a namespace (Stronghold EE):

d8 stronghold write sys/quotas/rate-limit/team-a path=team-a/ rate=300

If several quotas match a request, the most specific one applies: role, then a path within a mount, then the mount, then the namespace, then the global quota.

Viewing and deleting

To list the quotas, read a quota and delete it, run the following commands:

d8 stronghold list sys/quotas/rate-limit
d8 stronghold read sys/quotas/rate-limit/approle-login
d8 stronghold delete sys/quotas/rate-limit/approle-login

General quota settings

The /sys/quotas/config endpoint sets general parameters:

By default, the following paths are exempt from rate limit quotas: sys/generate-recovery-token/attempt, sys/generate-recovery-token/update, sys/generate-root/attempt, sys/generate-root/update, sys/health, sys/seal-status and sys/unseal. Example: add sys/metrics to this list and enable response headers and audit logging:

d8 stronghold write sys/quotas/config rate_limit_exempt_paths="sys/generate-recovery-token/attempt,sys/generate-recovery-token/update,sys/generate-root/attempt,sys/generate-root/update,sys/health,sys/seal-status,sys/unseal,sys/metrics" enable_rate_limit_response_headers=true enable_rate_limit_audit_logging=true

Lease count quotas

Lease count quotas limit the number of leases that exist at the same time on a mount, in a namespace, or for a role. When the limit is reached, new requests that create leases are rejected until existing leases expire or are revoked. Lease count quotas are available in Stronghold EE.

Quotas are managed through the /sys/quotas/lease-count/{name} endpoints. Quota parameters are path, role and max_leases.

A quota that limits the number of leases on the database/ mount to 5000:

d8 stronghold write sys/quotas/lease-count/db-creds path=database/ max_leases=5000

For accurate per-role lease counting, do not enable the imprecise_lease_role_tracking server parameter.

Choosing values

1. Collect the actual load from the stronghold_core_handle_request and stronghold_expire_num_leases metrics over several weeks. The metrics are described in Monitoring.
2. Set limits with headroom above peak values.
3. Enable enable_rate_limit_audit_logging and analyze rejected requests.
4. Gradually lower the limits for clients with abnormal load.