# Server monitoring and alerts
URL: https://synquic.tech/docs/guides/server-monitoring.md

Watch a server's CPU, memory, disk, load and network over time, set the thresholds that open an alert, and read the traffic its edge handles.

import { Step, Steps } from 'fumadocs-ui/components/steps';

Every server you connect to Synq is sampled in the background: CPU, memory, root disk, load and network every
5 minutes, and the requests its edge serves every minute. You read it on the server's **Overview** tab, all
servers together on the [Monitoring page](/docs/dashboard/monitoring), and you get told when a server crosses a
limit you set.

<Callout title="Where this lives">
Open **Servers** in the sidebar and click a server. The **Overview** tab holds everything on this page, top to
bottom: the live health dials, **Resource history**, **Alert thresholds** and the **Traffic** panel.
</Callout>

## Live health

At the top of the **Overview** tab, three dials show **CPU** (with the load average), **Memory** and **Disk**
("X of Y"), next to **Uptime** since the last boot. The dials use the same colours as the Monitoring page: red
at the server's alert threshold, amber when getting close. Below them, the **Components** card lists Docker,
OpenResty and the other components with **Healthy** / **Unhealthy** markers (see [Servers](/docs/dashboard/servers)).

## Resource history

**Resource history** charts the stored samples. Pick a **Time range**: **1h**, **24h**, **7d** or **30d**.

| Chart | What it shows |
| --- | --- |
| **CPU** | CPU use in percent. |
| **Memory** | Memory in use, in percent. |
| **Disk** | Root disk use, in percent. |
| **Load (1m)** | The 1-minute load average. |
| **Network** | Bytes received (**In**) and sent (**Out**). |

The first points appear about 5 minutes after the server is connected: "No samples yet. Server metrics are
recorded every 5 minutes." Samples are taken by the **Resource usage sampling** job (`resources:sample`) and
kept for 30 days.

## Alert thresholds

The **Alert thresholds** card decides when Synq raises an alert for this server: "Open an issue and send a
notification when this server crosses a limit. It resolves on its own when usage drops back."

| Field | Default | Meaning |
| --- | --- | --- |
| **Alerts enabled** | on | Switch all three alerts for this server on or off. |
| **Disk above (%)** | 85 | Alert when the root disk's latest sample is at or above this. |
| **Memory above (%)** | 90 | Alert when the latest memory sample is at or above this. |
| **CPU above (%)** | 90 | Alert when CPU stays above this... |
| **Sustained for (minutes)** | 10 | ...for this long, across consecutive samples. `0` alerts on a single sample. Up to 1,440 (one day). |

<Steps>

<Step>

### Change a value

Type the new number. Each field shows its default underneath ("Default 85"). Percentages must be between 1 and
100.

</Step>

<Step>

### Save

Press **Save**. The new limits apply from the next sample. **Reset to defaults** puts all four values back.

</Step>

</Steps>

How the alerts behave, so they stay quiet enough to leave on:

- **CPU must be sustained.** A build or a cron job pins every core for a minute several times a day; only
  consecutive samples spanning **Sustained for** that are all above the line count. Memory and disk are levels,
  so the latest sample decides.
- **One alert per problem.** Each server and metric has at most one open incident. It opens once, stays open
  while the problem lasts, and resolves once.
- **It resolves with a margin.** An alert clears only when the value drops 5 percentage points below the
  threshold, so a disk hovering at 85% doesn't open and close an incident on every sample.

When an alert opens, you see it:

- as an issue on the [Monitoring page](/docs/dashboard/monitoring#the-issues-list): **Server disk almost full**,
  **Server memory high** or **Server CPU high**;
- as a notification, if you're subscribed to **Server disk almost full**, **Server memory high** or **Server CPU
  high** (all on by default), with an all-clear such as "Server disk back to normal" when it resolves. See
  [Notifications](/docs/guides/notifications).

Changing thresholds needs write access to the server.

## Traffic on this server

The **Traffic** panel ("Requests the edge on this server handled, across every domain it routes") has two parts.

**Live** (the last 60 seconds, refreshed every 5 seconds):

| Tile | Meaning |
| --- | --- |
| **Active requests** | In flight right now. |
| **Requests/s** | Rate over the last 60 seconds. |
| **Inbound** / **Outbound** | Bandwidth per second. |
| **Error rate** | Share of `5xx` responses. Amber from 1%, red from 5%. |
| **Avg response** | Mean response time over the last 60 seconds. |

A line under the tiles gives nginx's connection counts: "N open connections: X serving, Y idle". If the edge
doesn't answer, the panel says "The edge on this server isn't answering. Live numbers resume when it does."

**Over a range** (**1h**, **24h**, **7d**, **30d**): a summary row (**Requests**, **Avg rate**, **Peak rate**,
**Data in**, **Data out**, **5xx rate**, **4xx rate**), the charts **Requests per second**, **Bandwidth** and
**Response time** (p50, p95, p99), then **Busiest domains** (with **Requests**, **Data out**, **5xx** and **Now**
columns) and **Top countries** ("Daily totals for the days in this window").

<Callout title="Where the numbers come from" type="info">
The edge counts every response in shared memory as it goes out, which costs visitors nothing. The **Traffic
analytics collection** job copies the minute totals into the database every 30 minutes, and opening a chart
fetches the newest minutes first, so charts are current. Minute data is kept for 90 days and daily country
totals for 400 days.
</Callout>

## From the CLI

```bash
# Requests, bandwidth and hourly periods for a project on this server
synq edge traffic -p <projectId>

# Visitors, countries and top paths for one domain
synq edge analytics -p <projectId> --domain app.example.com
```

See [The edge](/docs/guides/the-edge#watching-your-traffic) for more, and the
[Analytics API](/docs/api/analytics) for `GET /api/analytics/server/{serverId}/metrics`,
`/traffic`, `/traffic/live` and `/alert-thresholds`.

## Related

<Cards>
  <Card title="Monitoring page" href="/docs/dashboard/monitoring" description="All servers and apps on one screen." />
  <Card title="App monitoring" href="/docs/guides/app-monitoring" description="One project's resources and traffic." />
  <Card title="Notifications" href="/docs/guides/notifications" description="Where alerts are sent." />
  <Card title="Servers" href="/docs/dashboard/servers" description="Connect and manage servers." />
</Cards>
