Server monitoring and alerts

Every server you connect to Synq is sampled in the background: CPU, memory, root disk, load and network every 5 minutes, and the requests its edge serves every minute. You read it on the server's Overview tab, all servers together on the Monitoring page, and you get told when a server crosses a limit you set.

Live health

At the top of the Overview tab, three dials show CPU (with the load average), Memory and Disk ("X of Y"), next to Uptime since the last boot. The dials use the same colours as the Monitoring page: red at the server's alert threshold, amber when getting close. Below them, the Components card lists Docker, OpenResty and the other components with Healthy / Unhealthy markers (see Servers).

Resource history

Resource history charts the stored samples. Pick a Time range: 1h, 24h, 7d or 30d.

The first points appear about 5 minutes after the server is connected: "No samples yet. Server metrics are recorded every 5 minutes." Samples are taken by the Resource usage sampling job (resources:sample) and kept for 30 days.

Alert thresholds

The Alert thresholds card decides when Synq raises an alert for this server: "Open an issue and send a notification when this server crosses a limit. It resolves on its own when usage drops back."

How the alerts behave, so they stay quiet enough to leave on:

• CPU must be sustained. A build or a cron job pins every core for a minute several times a day; only consecutive samples spanning Sustained for that are all above the line count. Memory and disk are levels, so the latest sample decides.
• One alert per problem. Each server and metric has at most one open incident. It opens once, stays open while the problem lasts, and resolves once.
• It resolves with a margin. An alert clears only when the value drops 5 percentage points below the threshold, so a disk hovering at 85% doesn't open and close an incident on every sample.

When an alert opens, you see it:

• as an issue on the Monitoring page: Server disk almost full, Server memory high or Server CPU high;
• as a notification, if you're subscribed to Server disk almost full, Server memory high or Server CPU high (all on by default), with an all-clear such as "Server disk back to normal" when it resolves. See Notifications.

Changing thresholds needs write access to the server.

Traffic on this server

The Traffic panel ("Requests the edge on this server handled, across every domain it routes") has two parts.

Live (the last 60 seconds, refreshed every 5 seconds):

A line under the tiles gives nginx's connection counts: "N open connections: X serving, Y idle". If the edge doesn't answer, the panel says "The edge on this server isn't answering. Live numbers resume when it does."

Over a range (1h, 24h, 7d, 30d): a summary row (Requests, Avg rate, Peak rate, Data in, Data out, 5xx rate, 4xx rate), the charts Requests per second, Bandwidth and Response time (p50, p95, p99), then Busiest domains (with Requests, Data out, 5xx and Now columns) and Top countries ("Daily totals for the days in this window").

From the CLI

# Requests, bandwidth and hourly periods for a project on this server
synq edge traffic -p <projectId>

# Visitors, countries and top paths for one domain
synq edge analytics -p <projectId> --domain app.example.com

See The edge for more, and the Analytics API for GET /api/analytics/server/{serverId}/metrics, /traffic, /traffic/live and /alert-thresholds.

Related