Site Availability Monitoring

Overview

Site monitoring is a blackbox monitoring system built into DOP. It allows you to check the availability of websites, TCP services, and DNS records from external observation points (zones).

The agent is installed on a server outside the cluster, receives configuration from DOP, performs checks, and sends metrics back to DOP. Results are available on dashboards and through the alerting system.

Supported check types: HTTP, TCP/TLS, DNS, Ping (ICMP).

Quick Start

1. Make sure site monitoring is enabled (Admin → Settings → web_monitoring_enabled)
2. Create a zone: Admin → Zones → New Zone (specify a name, e.g. world)
3. Create an agent: Admin → Agents → New Agent (specify a name and zone)
4. Copy the agent token from the agent card
5. Install the agent on an external server (see Agent Installation)
6. Navigate to your project: Project → Web-monitoring → Add Site
7. Enter a domain, select zones, add a probe via the “HTTPS Check” preset, and save

Within 1–2 minutes, availability data will appear on dashboards.

Core Concepts

Zones

A zone is a geographic or network observation point. Each agent belongs to one zone. A site can be checked from multiple zones simultaneously for independent availability assessment.

Creating zones: Admin → Zones → New Zone.

Sites

A site is the monitoring target: a domain name or IP address. It belongs to a project. Each site contains one or more probes and shared settings (period, ping, zones, labels).

Probes and Checks

A probe is a single request of a specific type (HTTP, TCP, or DNS) to a site. Each probe contains one or more checks — specific verification conditions (e.g. HTTP status = 200 or response time < 5s).

The agent executes all checks on each probe run and sends a gogomonia_test metric (1 = pass, 0 = fail) for each check.

A multi-step HTTP scenario (multihttp) is a probe made of several HTTP requests executed in order — for example login → profile → logout. Steps share a cookie session, may extract variables from responses (JSON path, regex, header, cookie, CSS selector) and use them in the following steps as ${name}; every step has its own checks. See Multi-step HTTP scenarios.

Labels

Labels are arbitrary Prometheus-compatible key=value pairs added to metrics. They are used for grouping, filtering, and alert management.

Labels are set at three levels:

• Site (site_labels) — added to all site metrics
• Site (check_labels) — added to all site checks
• Check (check_labels) — per-check labels that override site-level ones

Alert management: the label alert=yes (default) enables alerts, alert=no disables them for a specific check.

Key format: Latin letters, digits, underscores; must start with a letter or underscore (/^[a-zA-Z_][a-zA-Z0-9_]*$/).

Check Types

Each probe (HTTP/TCP/DNS) contains one or more checks. The agent executes all checks on each probe run and sends a gogomonia_test metric (1 = pass, 0 = fail) for each check.

All pattern fields (bodyMatch, contentType, answerMatch, location, headers.match, cookies.match) use the /regex/ format (Go RE2-compatible). Adding ! after the closing slash (/pattern/!) inverts the result (NOT match). A value without the surrounding slashes is compared for full equality as a plain string.

HTTP Checks

status — HTTP Status Code Check

Verifies that the response code is in the list of allowed codes. Used to ensure server response correctness.

• Field: comma-separated list of codes
• Format: integers 100–599 or regex in slashes
• Examples: 200 · 200, 301, 302 · /2[0-9]{2}/ (any 2xx)

responseTime — Maximum Response Time

Verifies that the request completed faster than the threshold. The check fires in real time: if the response does not arrive within max, the test is marked as fail before the request completes.

• Field: max — maximum duration
• Format: number + unit: ms, s, m, h
• Examples: 5s · 1500ms · 1m

bodyMatch — Response Body Regex Check

Verifies that the HTTP response body contains (or does not contain) a specified pattern. Useful for checking that the page contains expected content and does not show an error.

• Field: regex
• Format: /regex/ or /regex/! (NOT match)
• Examples: /OK/ · /Welcome to/ · /error|maintenance/! (must not contain)

bodySize — Response Body Size

Verifies that the response body size falls within the specified range. Helps detect empty responses or anomalously large pages.

• Fields: min — minimum size (optional), max — maximum size (optional). At least one of min or max is required.
• Format: number + unit: b, Kb, Mb, Gb
• Examples: min 100b · max 10Mb · min 1Kb max 5Mb

contentType — Content-Type Header Check

Verifies that the Content-Type header in the response matches the expected value. Used to ensure the server returns the correct content type rather than an error page.

• Field: regex
• Format: /regex/ or /regex/!
• Examples: /text\/html/ · /application\/json/ · /image\/.*/

location — Location Header Check (Redirects)

Verifies that the Location header in the response (for 3xx redirects) matches a pattern. Useful for ensuring a redirect leads to the correct URL.

• Field: regex or exact URL
• Format: /regex/, /regex/! or a URL compared for full equality
• Examples: /https:\/\/example\.com\// · /\/login$/ · https://example.com/

A URL comparison ignores the port implied by the scheme: https://example.com:443/ and https://example.com/ are treated as the same target, in either direction. Any other port is compared as written. This does not apply to the /regex/ format, which is matched against the header as it came.

sslBasicConstraintsValid — TLS Certificate Check

Comprehensive TLS certificate verification: BasicConstraints correctness (CA certificates must have BasicConstraintsValid=true), leaf certificate validity period (NotBefore / NotAfter), hostname matching (exact and wildcard *.example.com).

• Fields: none (enabled as boolean)

certValidDays — Certificate Expiration

Verifies that the TLS certificate will not expire within the next N days. Used for early detection of expiring certificates.

• Field: min — minimum days until expiration
• Format: positive integer
• Examples: 30 (warn 30 days ahead) · 7 · 90

sslValidDays — Certificate Expiration (Alternative)

Same as certValidDays. Cannot be used simultaneously with certValidDays.

• Field: min — minimum days until expiration
• Format: positive integer

headers — Response Headers Check

Verifies that HTTP response headers contain specified values. Each pair: header name + regex for value. All pairs must match (AND logic).

• Fields: name / match pairs
• name: header name (case-insensitive), e.g. Content-Type, X-Request-Id
• match: regex /pattern/ or /pattern/!
• Examples: name X-Request-Id match /^[a-f0-9-]+$/ · name Cache-Control match /no-cache/!

cookies — Response Cookies Check

Verifies that HTTP response cookies contain specified values. Each pair: cookie name + regex for value. All pairs must match (AND logic).

• Fields: name / match pairs
• name: cookie name, e.g. session_id, csrf_token
• match: regex /pattern/ or /pattern/!
• Examples: name session_id match /^[a-f0-9]{32}$/ · name lang match /^(en|ru)$/

jsonPath — JSON Body Check

Evaluates conditions over values of a JSON response body. Each assertion: a JSONPath expression, a condition and (except for exists) an expected value. All assertions must pass (AND logic). Available for HTTP probes and for steps of multi-step scenarios.

• Fields: expression / condition / value triples
• expression: JSONPath, e.g. $.status, $.items[0].id, $.user.roles[*]
• condition: exists, equals, notEquals, contains, notContains, startsWith, endsWith, matches (regex), gt, gte, lt, lte (numeric)
• value: expected value; may reference a scenario variable as ${name}
• Examples: $.status equals ok · $.items exists · $.balance gt 0 · $.user.id equals ${user_id}

Multi-step HTTP scenarios

A multihttp probe runs several HTTP requests in order as one probe. It is added from the “Add probe” menu (probe type “Multi-step HTTP scenario”) in both form modes and is useful for flows that only make sense as a sequence: sign in and load a protected page, create and read back an object, walk a checkout. Simple mode covers the scenario interval, timeout and shared cookies plus, for every step, its name, method, scheme, URL, body (for POST/PUT/PATCH), variables and the simple checks (status, responseTime, certificate); advanced mode adds step ports, timeouts, redirect following, request headers, Basic Auth, disabling HTTP/2, logging of variable values and the remaining HTTP checks including jsonPath — the same split as the HTTP probe form.

Scenario settings:

• interval — 30s to 5m (the scenario runs at most once per interval; the range is wider than the 60s of a single probe because a scenario issues several sequential requests)
• timeout — deadline for the whole scenario (defaults to the interval minus 5s)
• shared cookies — keep cookies between steps (on by default)
• log variable values — write extracted values to agent logs (off by default: only variable names are logged)
• disable HTTP/2 for HTTPS — run every step over HTTP/1.1; the setting is scenario-wide because the steps share one HTTP client

Each step has a name, URL (absolute — with its own scheme, host and port — or a path resolved against the site host), scheme and port for a path URL, method, body, headers, Basic Auth, timeout, redirect policy, variables and checks:

• The URL is sent as written: type it already percent-encoded, query parameters included (/search?q=hello%20world). Variables are inserted as ${name} (verbatim) or ${name|urlencode} (percent-encoded per RFC 3986 — everything but A-Za-z0-9-._~, space → %20 — which is valid both in a query value and in a path segment): /me?sid=${sid|urlencode}. Any other filter is rejected when the site is saved.
• Scheme and port complete a path URL (/api/login → https://<site host>:8443/api/login), exactly as for the HTTP probe; the form starts a new step with HTTPS. A step saved without a scheme (through the API or by an older form) is completed with the scheme of the site request and otherwise with http — the form shows that effective value. An absolute URL carries its own scheme and port: the two fields show them and are locked, and sending scheme/port next to an absolute url through the API is a validation error.
• Templates [[ rndStr ]] and [[ .Site ]] work in the URL, header values and Basic Auth of a step as they do in the HTTP probe request — rendered once when the agent loads the config; the body is sent as written.
• Variables are extracted from the step response: jsonPath ($.access_token), regex (the first capture group or the whole match), header (header value), cookie (cookie value) or cssSelector (element text or an attribute). Up to 10 per step. A variable that cannot be extracted fails the step.
• ${name} references to variables of previous steps are allowed in the URL, header values, the body and in check values; a check value may also reference a variable its own step extracts (the variables are extracted before the checks run). Typing ${ in one of these fields of the editor offers the variables available to that step.
• Basic Auth is set per step (the site request defaults still apply to steps without their own); an explicit Authorization header — for example Bearer ${token} — takes precedence over it, as in the HTTP probe.
• Checks are the HTTP checks described above plus jsonPath; each step may have up to 10. A new step starts with status and responseTime, like an HTTP probe; each check type is added once per step (a type already present leaves the “Check” menu). Alert levels are configured on the scenario, not on a single step check.

The scenario stops at the first failed step: a transport error, a failed extraction or any failed check (responseTime and certificate checks included). The remaining steps are reported as skipped and all their checks fail.

Alerting works per scenario, not per step check: one GogomoniaScenarioFailed alert per probe, with the severity chosen in the probe’s “Notifications” selector, listing the step and checks that failed. Step checks therefore have no per-check notification settings (checkLabels are not accepted on them); the per-check GogomoniaCheckFailed alert applies to single-request probes only.

Metrics carry a step label: per-step series (gogomonia_test{step="login"}, gogomonia_http_response_time_ms_sum{step="login"}, …) plus scenario-level series (gogomonia_probe_request_succeeded{step=""}, gogomonia_multihttp_runs_total, gogomonia_multihttp_duration_ms_sum, gogomonia_multihttp_steps_total{result}). The site page shows every step with its checks and the scenario duration; in the “Web-monitoring Site by Instance” dashboard the collapsible “Multi-step scenarios” row gives every scenario of the site a line of three charts — scenario status (one line: ok / failed / request failed / ambiguous), run duration and step response times; the status of individual steps and their checks is in the “Checks (overall)” and “Checks (per server)” rows. The link of a GogomoniaScenarioFailed alert opens this dashboard narrowed to the failed scenario.

Multi-step scenarios require agents of version 0.3.0 or newer; older agents skip such probes.

TCP/TLS Checks

connected — TCP Connection Check

Verifies that the TCP connection (and TLS handshake if TLS is enabled) was established successfully. Basic TCP service availability check.

• Fields: none (enabled as boolean)

responseTime — Connection Establishment Time

Same as HTTP responseTime but measures TCP connection establishment time (and TLS handshake if applicable).

• Field: max — maximum duration
• Examples: 5s · 2s

sslBasicConstraintsValid — TLS Certificate Check

Same as HTTP sslBasicConstraintsValid. Checks the TCP/TLS connection certificate.

• Fields: none (enabled as boolean)

certValidDays — TLS Certificate Expiration

Same as HTTP certValidDays. Checks the TCP/TLS connection certificate.

• Field: min — days until expiration
• Examples: 30 · 7

sslValidDays — TLS Certificate Expiration (Alternative)

Same as certValidDays. Cannot be used simultaneously with certValidDays.

DNS Checks

gotAnswer — Non-Empty DNS Response

Verifies that the DNS query returned at least one record. Basic check — the domain resolves.

• Fields: none (enabled as boolean)

answerMatch — DNS Response Regex Check

Verifies that at least one record in the DNS response matches a pattern. The response contains the full record string (e.g. example.com. 300 IN A 1.2.3.4).

• Field: regex
• Format: /regex/ or /regex/!
• Examples: /1\.2\.3\.4/ · /IN A/ · /IN MX.*mail\./

responseTime — DNS Query Time

Same as HTTP responseTime but measures DNS query time.

• Field: max — maximum duration
• Examples: 2s · 500ms

Ping (ICMP)

Ping is enabled via the ping: true flag at the site level. It works completely independently of probes — it requires neither probes nor checks. The built-in ping.maxRtt test passes if RTT < 500ms (threshold is hardcoded).

Monitoring Setup

Creating Zones

A zone is a geographic or network observation point. Each agent belongs to one zone.

Create a zone: Admin → Zones → New Zone. Specify the zone name.

Creating Sites

Path: Project → Web-monitoring → Add Site.

Two Form Modes

The site creation/editing form operates in two modes.

Simple mode (default for new sites):

• Basic fields shown: host, period, ping, zones
• Adding probes via presets (ready-made templates)
• Available checks are limited (see table below)
• TCP: simplified TLS toggle

Advanced mode (“Advanced mode” toggle at the bottom of the form):

• All additional fields: hosts, probe_all_resolved_ips, site_labels, check_labels, request_defaults
• Full set of check types
• Check grouping
• Per-probe labels and period
• TCP: full TLS configuration (insecureSkipVerify, serverName)

Switching from advanced to simple mode is blocked if the form contains data available only in advanced mode. For existing sites, the mode is chosen automatically based on saved settings.

Available Checks by Mode

Check Grouping

In advanced mode, the “Group conditions into checks” toggle is available.

Without grouping (default): each check is a separate verification with its own result (pass/fail) and its own alert.

Each check type is added once: a type that is already present (as a separate check or inside a group) disappears from the “Check” menu of the probe or step, and the button is disabled once every type is in use. In an HTTP probe, a second check of the same type — say two bodyMatch patterns with different severities — is added as a separate group with grouping enabled.

With grouping: multiple conditions (status, bodyMatch, responseTime, etc.) are combined into a single logical check with a shared name and shared alert. A severity level is assigned to the group.

Example: a group “https-health” may include status=200 + bodyMatch="/OK/" + responseTime max=5s. If any condition is violated, a single shared alert fires.

Grouping is enabled automatically if the site already contains checks with multiple conditions in a single config.

Site Parameters

Spreading the agents over the interval

Every agent of a zone runs a probe once per interval, but the agents of one zone do not hit the site at the same moment: the platform numbers the registered agents of the zone (in creation order, whether they are online right now or not) and agent k of N starts the probe k × interval / N seconds after the first one. With a 60 s interval and three agents the site gets a request every 20 s, with two agents every 30 s; a single agent behaves as before. The site form shows the resulting spacing for the selected zones under the interval field. The spreading applies to HTTP, TCP and multi-step probes; DNS probes and ping are not affected.

Slots are computed from the system clock, so the clocks of the agents of a zone must be synchronized (NTP). The agent compares its clock with the platform’s on every configuration poll: the difference is exported as gogomonia_agent_clock_skew_seconds, and a difference above 5 s is logged as a warning. Spreading requires agents 0.6.0 or newer; older agents keep running probes at their previous moments.

Presets

Presets are ready-made probe templates available when creating a site. They allow adding a standard configuration with a single click.

Run Now

Every saved probe on the site page has a Test button. It executes the probe immediately on the agents you choose — any available agent, all agents of one zone, all agents of every zone, or one specific agent (the select lists every live agent of the site zones as Agent <zone> / <name>) — and shows the result in a modal. The probe runs against every IP address of the host (all A records, as a traceroute does), whatever the site’s “probe all resolved IPs” option says; a site with its own hosts list is probed at one IP per entry.

Under the agent’s tab every probed IP is a row — status, the IP with the host it came from in parentheses, the run time — folding the details: the request that was sent (GET https://… with sensitive headers such as Authorization and API keys masked, cookie values shown by their first and last four characters as sid=abcd…wxyz, and every value taken from the secrets store shown as the ${secret:<path>#<key>} reference it came from), the response (200 HTTP/2.0, headers, body up to 50 KB; when the step followed redirects — the final URL and a table of the hops: status, from, to, the cookies each hop set; a TCP probe reports the made connection and its time, a DNS probe row leads with the queried host and names the resolver), extracted variables and the expected/actual value of every check (for bodyMatch the actual value describes the page: title, size, type and the first visible text).

For multi-step scenarios every step is a card of its own headed by “Step N of M” and the step name, shows its response time next to the status, plus the scenario state after it: all variables collected so far and the cookie jar of every host as host: name=value — what the next step starts with.

The Logs button of the site page opens the agent records of its probes. A failed request or scenario step leaves three kinds of error records, each readable on its own: the request record — its message names the first failed test and its failure (check 'logged_in/http.bodyMatch' failed: expected /Deckhouse UI/, got page "Deckhouse" (3522 bytes, text/html) at https://dex.example.com/auth after 4 redirects (+1 more failed test)), the fields carry the request and response with masked secrets (Cookie: _oauth2_proxy=<redacted>, Authorization: Bearer <redacted>), probe.url (as requested), probe.final_url and probe.redirects when redirects were followed (one line per hop, 302 https://a → https://b; set-cookie: session=<redacted>), probe.redirect_count, probe.response.summary, checks.failed and tests.failed.

When the request got no answer at all (timeout, refused connection), the message names the URL the request died on — a redirect hop is marked (after redirect) and repeated in probe.final_url — and the test lists are left out: nothing was tested.

Then one record per failed test (test '…' failed: expected …, got …, with test.expect, test.actual — for bodyMatch the page summary with a text excerpt — and the same response context); for scenarios the run summary multihttp scenario failed with the status of every step, probe.failed_step and probe.failure.

The same button is available for every probe in the site form (create and edit): the probe runs exactly as it is currently filled in, without saving the form — edit the configuration, run, inspect the result and edit again. The form has to be valid (errors are shown in the run modal).

Runs are on demand only: they do not produce metrics or alerts and the result is discarded when the modal is closed. Only agents seen in the last 90 seconds are eligible; an HTTP probe (single request or scenario) runs on agents of version 0.7.0 or newer — older ones report the trace with the secret values and the cookies in plain text, so they are not offered.

Traceroute

The Traceroute button in the site header traces the network path from the agents to the site: pick where to run (any available agent, all agents of one zone, all agents of every zone, or one specific agent able to trace), the protocol and the mode, and every agent reports the route to every IPv4 address of the site hosts (all A records, resolved on the agent itself without cache).

• Protocol: ICMP echo (like traceroute -I) or TCP — connects to a port with a limited TTL, like tcptraceroute, so the probes follow the same path and pass the same firewalls as the monitored service. The port is prefilled from the site probes (an explicit probe port, otherwise 80/443 by scheme) and can be changed.
• Mode: Standard — 30 hops, 3 probes per hop, 1 s per hop; Thorough — 30 hops, 5 probes per hop, 3 s per hop. Probes of one hop run in parallel, hops run one after another up to the maximum, without stopping early on silent hops.
• Result: one tab per agent (zone / agent name, with a dot for the worst of its results and a spinner while it has not answered yet), and in the tab a row per target — the traced IP with the host it came from in parentheses, the duration and whether the target was reached — followed by a compact table of hops: hop number, responder IP, its reverse DNS name and the round-trip time of every probe (a lost probe is a red *; when a hop lost some probes it also shows lost/sent). The chevron in front of a target folds its table. The target is green when it was reached without loss, yellow (with loss) when it was reached but probes were lost — on the target itself or partially on an intermediate hop (hops that never answer are routers that do not send ICMP errors and are not counted as loss) — and red when it was not reached within the hop limit. The clipboard icon of a target puts the classic traceroute output into the clipboard.

Reading the result

The same legend opens from How to read the result under the agent tabs of a run.

• A row is one hop — a router on the way to the target; # is its position. The agent sends several probes to every hop (3 or 5 by mode); the RTT column lists the answer time of each of them, Host is the reverse DNS name of the address (empty when it has none).
• * instead of a time — the probe got no answer within the timeout of the mode (1 s or 3 s).
• A greyed-out row without an IP, every probe * — the hop answered none of its probes. Usually a router that does not send ICMP Time Exceeded at all (a policy of the operator, MPLS core, a cloud gateway); it is not a problem while the hops after it answer, and it is not counted as loss.
• A highlighted row with 1/3 — the hop answered some probes but not all: lost/sent. On an intermediate hop this is usually ICMP rate limiting on that router rather than traffic loss — real traffic does not need the router to answer. Loss on the target itself, or on every hop from some point on, is a path problem.
• The green bold last row — the target answered itself; the route is complete.
• Target statuses: ok — reached, every probe answered; with loss — reached, but probes were lost on the target or on an intermediate hop; failed — not reached in N hops: nothing answered after the last responding router, so the target ignores ICMP echo or a firewall drops it (try the TCP mode on the service port); error — the trace could not run (no raw-socket permission, IPv6 address, agent error), the reason is on the target row.
• A site with several A records gets a row per address; the addresses may be hosted in different places, so their paths and statuses differ.

Only IPv4 is traced: an IPv6 host is listed with a note and skipped. Like probe tests, traceroutes need agents seen in the last 90 seconds and agents of version 0.4.0 or newer; the agent must run as root or with CAP_NET_RAW (the installer’s systemd unit does), otherwise every target is reported as an error. Results are discarded when the modal is closed.

A path where no hop answers at all (* * * everywhere) is what the network delivers, not a failure of the agent: many cloud networks and NAT gateways drop the ICMP Time Exceeded messages routers send back, and the system traceroute on the agent host shows the same stars. Use the TCP mode there — the target itself still answers the connect, so its reachability and RTT are shown even when every intermediate hop is silent. To compare, run the system traceroute -T -p 443 <host> on the agent host: it takes the same path the agent does.

Security considerations

An on-demand run and a traceroute go out from the monitoring agents, not from the browser, and nothing restricts where they may go:

• the agents of a zone are shared by every project of the platform, so a run reaches whatever their network reaches — the internal networks of the site the agents are installed at, the loopback of the agent host, the metadata service of the cloud it runs in;
• the operator sees the response of that request, up to 50 KB of the body, and a TCP traceroute tells whether a port is open;
• redirects are followed to any host, so the target of a run is not necessarily the one it was started for.

Starting a run needs edit rights on the sites of the project (the user and admin roles, the AI assistant acting for such a user), the same rights that let the configuration send a request anywhere. Treat these roles accordingly, and place the agents in a network segment whose reachability matches what the projects of the platform may see. Viewers can only read the result of a run somebody else started. The destinations of a run are deliberately not restricted by the platform: this is an accepted part of the trust model of these roles, and the boundary is the network segment the agents live in.

A run started from the site form executes the probe as it is filled in, without saving it, and leaves no record in the audit log: the audit log tracks saved configurations only. Such a run may reference the secrets of the project like a saved site does (the same edit rights read them either way), so the fact that a secret was used in a run that was never saved is not traceable afterwards; the AI assistant is not allowed to use secret references in a run it composes itself, only in saved sites.

Secrets

Passwords, tokens and API keys used by checks (basic auth, Authorization and other headers, request bodies, scenario steps, expected values of checks) do not have to be stored in the platform: any string field of a site configuration may contain a secret reference that the backend resolves from a HashiCorp Vault-compatible secrets store — Deckhouse Stronghold or HashiCorp Vault — only when the configuration is handed to the agents. The platform database, the public API, the audit log and webmon-cli exports contain the reference, never the value.

${secret:<path>#<key>}

• path is the path of a KV v2 secret inside the single KV engine configured for the platform (ui.secretsStore.kv.mount, secret by default); key is the key inside that secret. Path segments may contain letters, digits, _ . -.
• A relative path (${secret:app/prod#token}) is confined to the project: it resolves to <mount>/<pathPrefix>/<space name>/<project name>/app/prod (pathPrefix is dop/web-monitoring by default). Every project can use relative references without any extra permission. Renaming a space or a project changes this prefix — move the secrets accordingly.
• An absolute path starts with / (${secret:/team-a/prod/db#password}) and resolves to <mount>/team-a/prod/db. Absolute references are allowed only under the path prefixes a platform administrator has listed in the project settings (Settings → Web Monitoring → Allowed absolute secret paths, one input per prefix, e.g. /team-a/; / allows the whole engine; the section is shown only while the secrets store is configured). The list is matched on segment boundaries: /team-a/ allows /team-a/prod/db but not /team-abc/x. An empty list (the default) forbids absolute paths.
• The name secret is reserved for this syntax and cannot be used as a scenario variable.
• References belong to request fields and to the expected values of checks. Metric labels (siteLabels, checkLabels), names and variable extractors are rejected: a value substituted there would become a label of every series the probe writes and would be visible to everyone who can read the metrics of the project.

Who can read a secret. A reference is resolved into a request the agents send, so anyone who can edit the sites of the project (the user and admin roles, an API token with web monitoring access, the AI assistant acting for such a user) can point a check at a host they control and receive the value there. Editing sites is therefore equivalent to reading every secret the project may reference: grant the role accordingly and keep the list of allowed absolute paths as narrow as possible. Viewers cannot edit sites, start ad-hoc runs or list the secret catalog. What the platform itself never shows is the value: the database, the public API, the audit log, webmon-cli exports, the agent logs and the traces of ad-hoc runs carry the reference in its place.

In the site form, typing ${ in a field that accepts references (request path and URL, headers, request body, basic auth, expected values of checks) opens a list of what can go there: inside a scenario step — the variables extracted by the previous steps, as ${name}; when the secrets store is configured — the secrets under the project prefix and the allowed absolute prefixes, as ${secret:path#key} (paths and key names only, never values). Keep typing to narrow the list, pick with the arrow keys and Enter or with a click. When the store does not let the platform list a prefix, the list says so — type the path manually.

When a site is saved, every new reference is read from the store: a path or key that does not exist, a path the platform is not allowed to read, an absolute path outside the project allowlist or an unreachable store are validation errors — the configuration is not saved. Editing a site without changing its references never contacts the store.

At run time the backend reads the values when it renders the configuration for the agents and sends them next to it; the agent puts them into the request it makes, so a value that happens to look like a template or a variable reference is sent as data:

• a probe is run only when every reference of it has a value: a secret is never replaced by a placeholder, so a probe that has no value is left out of the configuration the agent runs — its other probes and the ping of the site go on. The site page names the reference and the reason, and the GogomoniaSiteRejected alert fires for the project;
• if the store is unreachable, the project keeps the configuration it has and the agents go on probing it with the secret values of the last successful render; nothing else in the zone is affected. A probe an agent has no values for at all (it was restarted while the store was down, or the reference is new) is not run until the store answers; the WebMonitoringSecretsStoreUnavailable alert of the platform covers that — the project cannot fix it. Agents older than 0.8.2 do not resolve references themselves: for them the state endpoint answers 503 and they keep their current configuration until the store answers again;
• if a single reference cannot be resolved anymore (the secret was deleted or moved, the policy changed, the project allowlist was narrowed), the probes that use it are not run and the site page names the reference; the rest of the site keeps being monitored. For an agent older than 0.8.2 the whole site is left out of its state instead — it cannot skip a single probe;
• rotated secrets are picked up automatically: every 5 minutes the backend compares the KV versions of the referenced secrets with the last seen ones and, when a version changed, bumps the configuration version so the agents re-fetch it. Ad-hoc runs (Test) resolve the references when an agent claims the job; a reference that cannot be resolved at that moment ends the run with an error that names it.

The same references can be used in conf.yaml managed with webmon-cli (see Git-Based Configuration Management): the API validates them on sync exactly like the form does, and exports return the references, not the values.

Existing plain-text credentials are not migrated automatically: put the value into the store and replace it in the site configuration with a reference.

Platform configuration

The setup order with Deckhouse Stronghold in the same cluster; the differences for HashiCorp Vault are at the end of the section. Every ui.secretsStore parameter is described in the module configuration reference. The examples use the KV engine dop-secrets, the prefix web-monitoring and the policy dop-web-monitoring.

1. Administrative access to Stronghold. The commands use the d8 stronghold CLI (compatible with vault, reads VAULT_ADDR and VAULT_TOKEN). An administrator listed in management.administrators logs in through the oidc_deckhouse method (d8 stronghold login -method=oidc -path=oidc_deckhouse). In the management.mode: Automatic mode the root token is kept in the Secret stronghold-keys of the d8-stronghold namespace (key rootToken); it is suitable for the initial setup from a console — do not print it:

export VAULT_ADDR=https://stronghold.<publicDomainTemplate>   # the address of the stronghold module Ingress
export VAULT_TOKEN="$(kubectl -n d8-stronghold get secret stronghold-keys -o jsonpath='{.data.rootToken}' | base64 -d)"
d8 stronghold status                                            # Initialized true, Sealed false
d8 stronghold auth list                                         # kubernetes_local/ is configured by the module

2. The store address. The backend verifies the TLS certificate against the system CA bundle and the host name. When the API on port 8200 serves the Ingress certificate (issued for the public name), set the public address https://stronghold.<publicDomainTemplate> without caCert. The internal address https://stronghold.d8-stronghold.svc:8200 is suitable only when the certificate is issued for that name; its CA goes to ui.secretsStore.caCert. Checking the certificate from inside the cluster:

kubectl -n d8-observability-platform run tls-check --rm -i --restart=Never --image=alpine/openssl -- \
  s_client -connect stronghold.d8-stronghold.svc:8200 -servername stronghold.d8-stronghold.svc </dev/null 2>/dev/null \
  | openssl x509 -noout -subject -issuer -ext subjectAltName

3. The KV v2 engine and the path layout. The platform works with a single KV v2 engine (ui.secretsStore.kv.mount). Project secrets live under <mount>/<pathPrefix>/<space>/<project>/... and are available to the projects through relative references. Secrets shared by several projects get a prefix of their own (for example shared/): projects reference it with absolute paths allowed in the project settings.

d8 stronghold secrets list                                          # existing engines
d8 stronghold secrets enable -path=dop-secrets -version=2 kv        # when there is no KV v2 yet

4. The platform policy. data is for reading the values when the agent configuration is rendered; metadata is for rotation detection (current_version) and for the list of secrets in the site form (list; without it the paths are typed manually). The policy must cover the pathPrefix and every absolute prefix you plan to allow to projects:

# dop-web-monitoring.hcl
path "dop-secrets/data/web-monitoring/*"     { capabilities = ["read"] }
path "dop-secrets/metadata/web-monitoring/*" { capabilities = ["read", "list"] }
path "dop-secrets/data/shared/*"             { capabilities = ["read"] }
path "dop-secrets/metadata/shared/*"         { capabilities = ["read", "list"] }

d8 stronghold policy write dop-web-monitoring dop-web-monitoring.hcl

5. Platform authentication. Two methods are supported.

Option A — the Kubernetes method (auth.method: Kubernetes, the default). The backend logs in to the store with the projected token of the ServiceAccount backend-secrets of the d8-observability-platform namespace (created by the module, needs no RBAC) and receives a store token with the policy of the role; the token is renewed automatically. The role in the kubernetes_local method:

d8 stronghold write auth/kubernetes_local/role/dop-web-monitoring \
  bound_service_account_names=backend-secrets \
  bound_service_account_namespaces=d8-observability-platform \
  policies=dop-web-monitoring ttl=1h

Checking the role and the policy before the integration is enabled, with a token of the same ServiceAccount:

JWT=$(kubectl -n d8-observability-platform create token backend-secrets --duration=10m)
d8 stronghold write -field=token auth/kubernetes_local/login role=dop-web-monitoring jwt="$JWT" > /tmp/dop.token
VAULT_TOKEN=$(cat /tmp/dop.token) d8 stronghold kv list -mount=dop-secrets web-monitoring/     # a list or an empty answer, but not 403
rm /tmp/dop.token

Option B — a static token (auth.method: Token). The backend calls the store with a token from a Kubernetes Secret (its name is auth.token.secretName, secrets-store-token by default; key token). No login is performed and the ServiceAccount backend-secrets is not used. Issue an orphan token with the platform policy through a role of the token method and put it into the Secret:

d8 stronghold write auth/token/roles/dop-web-monitoring \
  allowed_policies=dop-web-monitoring orphan=true renewable=true token_type=service
d8 stronghold token create -role=dop-web-monitoring -policy=dop-web-monitoring \
  -display-name=dop-web-monitoring -ttl=768h -field=token \
  | kubectl -n d8-observability-platform create secret generic secrets-store-token --save-config --from-file=token=/dev/stdin

The token lifetime is limited by the max_lease_ttl of the token method (768 hours by default). The platform does not renew the token: before it expires, issue a new one, update the Secret and restart backend and backend-scheduler — the Secret value is passed to an environment variable when the pod starts. Revoke the previous token after the restart (d8 stronghold token revoke -accessor <accessor>; d8 stronghold token lookup -accessor shows the accessor and display_name of each token listed by d8 stronghold list auth/token/accessors).

d8 stronghold token create -role=dop-web-monitoring -policy=dop-web-monitoring \
  -display-name=dop-web-monitoring -ttl=768h -field=token \
  | kubectl -n d8-observability-platform create secret generic secrets-store-token --from-file=token=/dev/stdin \
      --dry-run=client -o yaml | kubectl apply -f -
kubectl -n d8-observability-platform rollout restart deployment/backend deployment/backend-scheduler

The store does not distinguish an expired or revoked token from a policy denial and answers 403: no secret reference resolves, the probes that use secrets are not run, the site page shows the reason access denied, and sites with new references cannot be saved until the token is replaced.

6. Enabling the integration in the module. mount and pathPrefix must match the policy; the auth section follows the chosen option:

# kubectl edit moduleconfig observability-platform
spec:
  settings:
    ui:
      secretsStore:
        enabled: true
        address: https://stronghold.<publicDomainTemplate>   # see step 2
        auth:
          method: Kubernetes            # option A
          kubernetes:
            path: kubernetes_local
            role: dop-web-monitoring
          # method: Token               # option B
          # token:
          #   secretName: secrets-store-token
        kv:
          mount: dop-secrets
          pathPrefix: web-monitoring

Deckhouse rolls out backend and backend-scheduler: the pods get the SECRETS_STORE_* variables from the ConfigMap backend-envs; with option A also the ServiceAccount backend-secrets and a projected token in /var/run/secrets/secrets-store/token, with option B the SECRETS_STORE_TOKEN variable from the Secret. The integration is ready when typing ${ in a password field of the site form offers ${secret:…} references (or, for a project without secrets yet, a note naming its prefix dop-secrets/web-monitoring/<space>/<project>).

7. A secret and a reference to it. The secret is stored under the project prefix and referenced in the site configuration; on save the backend reads every new reference from the store:

d8 stronghold kv put -mount=dop-secrets web-monitoring/<space>/<project>/httpbin user=monitor password='...'
d8 stronghold kv put -mount=dop-secrets shared/api token='...'      # for the absolute reference ${secret:/shared/api#token}

request:
  basicAuth:
    username: ${secret:httpbin#user}          # relative to the project prefix
    password: ${secret:httpbin#password}
  headers:
    - name: Authorization
      value: Bearer ${secret:/shared/api#token}   # absolute path: needs the /shared/ prefix in the project settings

A platform administrator allows absolute paths in the project settings (Settings → Web Monitoring → Allowed absolute secret paths, for example /shared/; project administrators see the list read-only). A reference to a missing key, a path denied by the policy or an absolute path outside the list is rejected by the form and the API with the reference named.

8. Rotation and failures. A new version of a secret (kv put) is picked up automatically: every 5 minutes backend-scheduler compares the current_version of the referenced secrets and bumps the configuration version when it changed, the agents re-read the configuration on their next poll; independently of that, an agent requests the full configuration every 10 minutes and applies it when the content changed.

When a secret is deleted or the access to it is revoked, the probes that reference it stop being run and the site page names the reference — a secret is never replaced by a placeholder. When the store is unreachable, the agents keep probing with the values they already hold, the probes they have no values for are not run, and sites with new references cannot be saved until the store is back (agents older than 0.8.2 receive 503 for the whole state and stay on their current configuration). Secret values in URL paths are not protected: the agent writes the request line to the check log and masks only the Authorization, Cookie and similar headers — use references in basicAuth, headers and request bodies, not in path.

Typical errors: 403 on read — the policy does not cover the path (data/) or its metadata (metadata/), or the static token has expired or been revoked; 404 — no such path or key (a KV v2 policy path carries the data/ segment, kv commands do not); login failed with HTTP 403 in the backend log — the Kubernetes method role is not bound to backend-secrets/d8-observability-platform, the role has an audience different from ui.secretsStore.auth.kubernetes.audience, or the store lacks access to the TokenReview API (for an in-cluster Vault — the ClusterRole system:auth-delegator for its ServiceAccount); certificate verify failed — the address does not match the names in the certificate (step 2).

HashiCorp Vault. The differences: the KV engine must be version 2 and carry the name used in the policy and in ui.secretsStore.kv.mount; the Kubernetes method is enabled and configured manually and its path is kubernetes (ui.secretsStore.auth.kubernetes.path: kubernetes); ui.secretsStore.address is the Vault address (inside the cluster — http(s)://<service>.<namespace>.svc:8200). A static token is issued the same way with vault commands:

vault secrets enable -version=2 -path=dop-secrets kv
vault auth enable kubernetes
# Vault inside the cluster: the backend pod token is validated through the TokenReview API
# on behalf of Vault's own ServiceAccount, which needs the ClusterRole system:auth-delegator:
#   kubectl create clusterrolebinding vault-auth-delegator \
#     --clusterrole=system:auth-delegator --serviceaccount=<Vault namespace>:<Vault ServiceAccount>
vault write auth/kubernetes/config kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
# Vault outside the cluster: point it at the cluster and either give it a reviewer token
# (token_reviewer_jwt) or set ui.secretsStore.auth.kubernetes.grantTokenReview: true so the
# platform ServiceAccount may call the TokenReview API itself.
vault policy write dop-web-monitoring dop-web-monitoring.hcl
vault write auth/kubernetes/role/dop-web-monitoring \
  bound_service_account_names=backend-secrets \
  bound_service_account_namespaces=d8-observability-platform \
  policies=dop-web-monitoring ttl=1h

If the role has an explicit audience, set the same value in ui.secretsStore.auth.kubernetes.audience. ui.secretsStore.namespace sends the X-Vault-Namespace header for Stronghold namespaces or Vault Enterprise; leave it empty otherwise. A custom CA of the store goes to ui.secretsStore.caCert.

What the agents report about a configuration

The platform validates a site when it is saved, but the last word is the agent’s: it compiles the expressions of the scenario variables and checks with its own engines (Go RE2, ojg, cascadia), computes the effective timeouts and holds the secret values. A site or a probe it refuses produces no metrics at all, so no check alert can fire for it — the verdict of each agent is therefore shown on the site page and counted as a badge in the list of sites:

• the whole site was rejected — the configuration did not pass the agent’s validation (an expression it cannot compile, a timeout above the limit it computes, a key this agent version does not know). Nothing of the site is monitored by that agent;
• a probe is not run — a ${secret:...} reference of it has no value, because the reference does not resolve or the secrets store could not be read. The other probes of the site and its ping keep running;
• a probe runs with the secret values of an earlier configuration — the secrets store is unreachable and the agent keeps probing with what it already holds.

An agent applies a saved configuration and reports back within a minute, so the verdict appears shortly after a save; reload the site page to see it. A rejected configuration also raises GogomoniaSiteRejected for the project, and an unreadable secrets store raises WebMonitoringSecretsStoreUnavailable for the platform — the owner of a project cannot fix the store. Agents older than 0.8.2 do not report verdicts: for them a refused site is visible only in their log and in gogomonia_agent_config_problem_sites.

Project Requirements

For site monitoring to work, the project must have an API token with “Metrics: write” (write_metrics) permission. This token is created automatically when the project is created.

If the token is deleted, a warning is displayed in the Web-monitoring section, and the agent does not receive the configuration for this project.

Agent Installation

Requirements

• Linux (amd64)
• Network access to DOP API (api.<domain>)

Manual Installation

1. Create an agent in Admin UI: Admin → Agents → New Agent (specify name and zone)
2. Copy the token from the agent card
3. On the target server:

# Download binary
curl -o /usr/local/bin/gogomonia-agent \
  "https://update.<domain>/gogomonia-agent/storage/latest/linux/amd64/gogomonia-agent"
chmod +x /usr/local/bin/gogomonia-agent

# Config
mkdir -p /usr/local/gogomonia-agent/etc
cat > /usr/local/gogomonia-agent/etc/config.yaml <<EOF
dop_api_url: "https://api.<domain>"
EOF

# Token
echo -n "<TOKEN>" > /usr/local/gogomonia-agent/agent.token

# Run
/usr/local/bin/gogomonia-agent

Agent Configuration

File config.yaml:

dop_api_url: "https://api.dop.example.com"
proxy:
  url: ""          # HTTP proxy (optional)
  no_verify_tls: false
poll_interval: 60s        # polling for configuration changes
force_poll_interval: 10m  # full state request: picks up changes made without a version bump, not shorter than poll_interval

Environment Variables

Git-Based Configuration Management

The converter utility allows storing site configurations in a git repository in a declarative YAML format with Go template support, and synchronizing them with DOP via the API. This is a GitOps approach: validate on merge request, sync on merge to master.

UI Edit Protection

By default, every site in DOP is editable from the UI — including sites created by webmon-cli. To explicitly declare in git that a particular site is managed via the API only (so that an operator does not silently change it in the UI and cause drift between git and DOP), set the uiEditable: false flag in YAML. Available in webmon-cli v0.5.1 and later:

defaults:
  uiEditable: false       # every site in the file is API-only
targets:
  - name: managed-site
    host: managed.example.com
    # uiEditable is inherited from defaults → false

  - name: editable-site
    host: editable.example.com
    uiEditable: true      # explicit exception: keep UI editing enabled

Semantics:

• uiEditable: true — default behaviour: the site is editable both in the UI and through the API.
• uiEditable: false — the UI shows an “API only” badge next to the site name and the edit button becomes greyed out; deletion (UI and API) and API updates remain available.
• Field absent in YAML — webmon-cli does not send the field, so the server keeps the previously stored value (for new sites that is true).
• Precedence: targets[].uiEditable → defaults.uiEditable → value already stored in DOP.

See API → Site parameters for the full API-contract description of the field.

Download

Binary: webmon-cli.

curl -o /usr/local/bin/webmon-cli \
  "https://update.<domain>/webmon-cli/storage/latest/linux/amd64/webmon-cli"
chmod +x /usr/local/bin/webmon-cli

Commands

The first argument is the command: sync or validate. Parameters are set via environment variables only.

sync — Synchronize Configs to DOP

Creates, updates, and (with DELETE_ORPHANS=true) deletes sites in DOP based on YAML file contents.

export DOP_API_URL="https://api.<domain>/api/v1/observability"
export CONFIGS_DIR=/path/to/configs
export DOP_WEB_MON_TOKEN_MYPROJECT="<API-token>"
webmon-cli sync

validate — Local Validation

Validates configs without applying: field formats, allowed keys, deprecated fields, duplicates. Loads the zone list from the API for validation.

export DOP_API_URL="https://api.<domain>/api/v1/observability"
export CONFIGS_DIR=/path/to/configs
webmon-cli validate

Important: DOP_API_URL is required for validate as well (zone list loading). A token is not required for validate.

Global Environment Variables

Project Mapping and Tokens

Environment variables are set for each project directory. The suffix is derived from the folder name: all non-alphanumeric characters are replaced with _, and the result is uppercased.

Examples: folder myapp → MYAPP, folder my-client → MY_CLIENT.

Repository Structure

{team-repo}/
  .helpers/helpers.tpl       # shared Go templates (Sprig)
  {project}/conf.yaml        # site configuration
  .gitlab-ci.yml

• Project = any folder containing *.yaml files
• Templates .tpl are loaded from COMMON_TEMPLATES and .helpers/ directories up the path from the project folder
• Templates use Go text/template syntax with the Sprig library
• Configs are rendered as templates before YAML parsing

conf.yaml Format

# Unit-level labels — added to all sites
siteLabels:                      # optional, map[string]string
  env: production
checkLabels:                     # optional, map[string]string
  alert: "yes"

# Default values for all sites
defaults:                        # optional
  ping: "true"                   # "true"/"yes" → enable ping
  period: "20s"                  # format: \d+(s|m), max 60s
  uiEditable: false              # optional, bool: false = UI marks the site as API-only
  request:                       # HTTP request defaults
    scheme: https                # http / https
    timeout: 10s                 # format: \d+(ms|s|m), max 300s
    headers:                     # array of {name, value}
      - name: User-Agent
        value: "monitoring/1.0"

# List of monitoring targets (required, at least 1)
targets:
  - name: example-main           # REQUIRED: unique, no spaces/quotes/parens/pipes
    host: example.com            # REQUIRED: domain or IP
    hosts:                       # optional: fixed IPs instead of DNS
      - 93.184.216.34
    probeAllResolvedIPs: "true"  # optional: check all IPs from DNS resolve
    period: 30s                  # period override
    ping: true                   # ping override
    uiEditable: true             # optional: override defaults.uiEditable
    siteLabels:                  # merge with unit-level
      env: staging
    checkLabels:                 # merge with unit-level
      alert: "no"
    request:                     # override request defaults
      scheme: https
      timeout: 5s
    probes:                      # >= 1 probe if ping=false
      - name: https-main         # REQUIRED: unique, no spaces/quotes/parens/pipes
        period: 30s              # optional: probe period override
        siteLabels:              # optional: merge with site-level
          component: api
        checkLabels:             # optional: merge with site-level
          alert: "yes"
        request:                 # full HTTP configuration
          scheme: https
          path: /healthz
          method: GET            # GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS, TRACE
          port: 443              # 1-65535
          timeout: 5s            # max 300s
          headers:
            - name: Accept
              value: application/json
          basicAuth:
            username: user
            password: pass
          data: '{"key":"value"}'
          disableHttp2ForHttps: "true"   # quoted string: "true" or "false"
        checks:                  # REQUIRED: >= 1 check
          - name: status-ok      # REQUIRED: unique, no spaces/quotes/parens/pipes
            status: [200, 301]
          - name: response-time
            responseTime:
              max: 5s
          - name: body-check
            bodyMatch: "/OK/"
          - name: size-check
            bodySize:
              min: 100b
              max: 10Mb
          - name: content-type
            contentType: "/text\\/html/"
          - name: ssl-check
            sslBasicConstraintsValid: true
          - name: cert-expiry
            certValidDays:
              min: 30
            checkLabels:
              alert: "no"

Legacy format: instead of targets, you can use sites with site (instead of host/name) and servers (instead of hosts). Both formats are supported, but targets is recommended. Mixing site + host or servers + hosts in one entry will cause an error.

TCP probe (instead of request):

        tcp:
          port: 443
          timeout: 5s
          tls:
            insecureSkipVerify: false
            serverName: example.com

DNS probe (instead of request):

        dns:
          type: A                # A, AAAA, CNAME, MX, NS, TXT, SOA, SRV, PTR, CAA
          query: example.com     # REQUIRED: domain to query
          timeout: 2s

Multi-step HTTP scenario (multihttp: instead of request; the probe itself has no checks, every step carries its own). Steps run in order sharing a cookie jar; ${name} refers to a variable extracted by a previous step. A login behind Dex (OIDC) followed by a protected page:

      - name: dex-login
        period: 60s                    # 30s..5m; the scenario timeout defaults to period − 5s
        multihttp:
          timeout: 50s                 # optional, deadline for all steps together, <= period
          cookies: "true"              # optional, share cookies between steps (default true)
          logVariables: "false"        # optional, log variable values, not only names (default false)
          disableHttp2ForHttps: "false" # optional, HTTP/1.1 for every step (one client serves the scenario)
          steps:
            - name: open_console       # 1. console -> redirect chain -> Dex "choose a login method" page
              url: /?from=console      # path → site host; absolute http(s):// URLs are allowed too; sent as written
              followRedirects: "true"
              variables:
                - name: login_link     # href of the "Email" connector, carries the one-time state
                  type: cssSelector    # jsonPath | regex | header | cookie | cssSelector
                  expression: 'a[href^="/auth/local"]'
                  attribute: href      # cssSelector only: attribute instead of element text
              checks:
                - name: chooser
                  status: [200]
                  bodyMatch: "/select a login method/"
            - name: open_login_form    # 2. the login form; its action carries the request state
              url: "https://dex.example.com${login_link}"
              followRedirects: "true"
              variables:
                - name: login_action
                  type: cssSelector
                  expression: form.login-form
                  attribute: action
              checks:
                - name: form
                  status: [200]
            - name: submit             # 3. POST credentials -> callback sets the session cookie -> home page
              url: "https://dex.example.com${login_action}"
              method: POST
              headers:
                - name: Content-Type
                  value: application/x-www-form-urlencoded
              data: "login=monitor%40example.com&password=secret"
              timeout: 20s             # optional, per-step timeout <= scenario timeout
              followRedirects: "true"
              checks:
                - name: logged_in
                  status: [200]
                  bodyMatch: "/Deckhouse UI/"
                  responseTime:
                    max: 15s
            - name: protected_page     # 4. opens with the session cookie, no redirect to Dex
              url: "/?state=${login_link|urlencode}"  # variable percent-encoded for the query
              basicAuth:               # optional, credentials of this step only
                username: monitor
                password: "${secret:app/prod#password}"
              checks:
                - name: authorized
                  status: [200]

Step fields: name (required, unique in the probe), url (required; sent as written, so already percent-encoded and with the query parameters inside — there is no separate query list), scheme (http/https) and port (1..65535) that complete a path url ahead of the site request (scheme falls back to http; both are rejected next to an absolute url), method (default GET), headers (list of {name, value}), basicAuth ({username, password} of the step; an explicit Authorization header wins over it), data (request body), timeout (the site request.timeout when the step sets none, else 10s — the form shows that effective value as the placeholder), followRedirects, variables (up to 10 per step), checks (up to 10 per step; the same checks as for HTTP probes plus jsonPath; checkLabels are not allowed on step checks — alert labels of a scenario are set on the probe).

${name} is allowed in url, headers values, data and in check values; ${name|urlencode} inserts the value percent-encoded (RFC 3986), any other filter is a validation error. The scenario stops at the first failed step; the remaining steps are reported as failed. Requires agents 0.3.0 or newer; step basicAuth, scheme, port, [[ ]] templates in steps, disableHttp2ForHttps and ${name|urlencode} require agents 0.5.0 or newer (older agents do not receive such scenarios).

Validation rules:

• name and host — required fields of each target
• If ping: false and probes is empty — validation error
• Probe type is determined by key presence: tcp: → tcp, dns: → dns, multihttp: → multihttp, otherwise http
• multihttp: 1..10 steps with unique names, at least one check across the steps, no probe-level checks; period 30s..5m; timeout <= period, step timeout <= scenario timeout; ${name} may only reference variables of previous steps (in check values — of the step itself too) and the only filter is urlencode; the regex, jsonPath and cssSelector expressions of variables and the jsonPath of checks must compile — the agent would otherwise reject the whole site; steps[].query is rejected (unknown key "query": put the parameters into url and encode variable values with ${name|urlencode}); variable type is one of jsonPath, regex, header, cookie, cssSelector (attribute only with cssSelector)
• Zones are determined from siteLabels.zone; empty zones = all zones
• certVerify / sslVerify — deprecated, produces a warning (use sslBasicConstraintsValid instead)
• Unknown keys at any level — parse error (rejected by the YAML parser)
• Site-level request (under defaults or targets) only allows scheme, timeout, headers. Probe-specific keys (port, method, path, etc.) must be set in probe-level request
• certValidDays and sslValidDays — cannot be used simultaneously
• bodySize supports min and/or max
• responseTime supports only max (not min); max format: \d+(ms|s|m) (no h)
• Effective period (probe override → site → unit default) must be >= every check’s responseTime.max

CI/CD Integration

The converter is designed for use in GitLab CI: validate on MR, sync on merge to master.

stages:
  - validate
  - apply

variables:
  DOP_CONVERTER_URL: "https://update.${DOP_BASE_DOMAIN}/webmon-cli/storage/${DOP_CONVERTER_VERSION}/linux/amd64/webmon-cli"

.download_converter: &download_converter
  image: alpine:${DOP_WEB_MONITORING_ALPINE_VERSION}
  before_script:
    - apk add --no-cache curl
    - curl -sSL -o webmon-cli "${DOP_CONVERTER_URL}"
    - chmod +x webmon-cli

validate:
  <<: *download_converter
  stage: validate
  variables:
    DOP_API_URL: "${DOP_API_URL}"
    CONFIGS_DIR: "."
  script:
    - ./webmon-cli validate

apply:
  <<: *download_converter
  stage: apply
  variables:
    DOP_API_URL: "${DOP_API_URL}"
    CONFIGS_DIR: "."
  script:
    - ./webmon-cli sync
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'

CI/CD Variables:

Important: replace {FOLDER} with the project directory name after sanitization. Folder myapp → DOP_WEB_MON_TOKEN_MYAPP.

Metrics

Metric names use the technical prefix gogomonia_ — this is the internal component name, preserved for backward compatibility.

Probe Metrics (per-project tenant)

Labels: instance, zone, project, site, server, probe, check, test

Agent Self-Monitoring Metrics

Labels: instance, zone, uuid, hostname, version

Dashboards

When site monitoring is enabled, the following dashboards automatically appear in the project:

• Overall Status — summary table of all project sites: availability, response times, check status
• Zone Status — zone-level details: per-zone metrics, availability comparison across zones
• Site by Instance — site drill-down: per-server/per-instance metrics, response time scatter plot, the “Multi-step scenarios” row (status, duration and step response times of every scenario)

Additional system dashboards (Go Runtime, Logs) are available for monitoring agent health.