Channels and routes
Notifications have two parts, both on the Notifications page.
- A channel is one destination: a connection plus its settings, such as a PagerDuty service, a Jira project, a Slack channel or a Telegram chat.
- Routes are one ordered list for the whole organization. Each alert’s notices walk the list from the top and go to the channels of the routes they match.
Channels
Section titled “Channels”| Channel | Alerts | Reports |
|---|---|---|
| PagerDuty | Yes | No |
| Jira Cloud | Yes (one issue per alert; its key is saved on the alert) | No |
| Slack, Microsoft Teams, Discord, Google Chat, Telegram | Yes | Yes |
| Email (SMTP) | Yes | Yes |
| AWS SNS topic | Yes | Yes |
| Webhook (signed) | Yes | Yes |
To add one, first add the connection on Connections, then select New channel on Notifications, choose the connection, name the channel, and fill in its settings (for example a Telegram channel’s Chat id, or a Slack channel name). Several channels can share one connection. Step-by-step setup for each is in Send alerts to your team.
Routes
Section titled “Routes”Each route has:
| Part | What it does |
|---|---|
| Severity | Matches alerts of these severities. Empty: any. |
| Labels | Every label listed must be on the alert with one of the given values, such as team=payments. Empty: any. |
| Source (API) | Matches alerts from these sources, such as check, job or job_finding. |
| Monitors and Clients (API) | monitorIds, clientIds: alerts of these monitors, or of these clients’ monitors and jobs. |
| Send to | The channels. Empty means record only. |
| Keep matching routes below this one | Without it, the first matching route decides. With it, matching goes on and the channels add up. |
An alert that matches no route is recorded but notifies nobody. A route with an empty match matches every alert: put it last, as a catch-all.
Use Try an alert beside the list to see which route an alert with a given severity and labels would take, before you select Save routes. New notifications follow the saved routes.
Example
Section titled “Example”- Severity
critical→ PagerDuty on-call, keep matching. - Labels
team=payments→ #payments in Slack. - Empty match → Ops email.
A critical payments alert pages on-call and posts to #payments. A medium payments alert posts to #payments only. Every other alert goes to email.
Delivery
Section titled “Delivery”- In order. One alert’s notices reach a channel strictly in order, so a resolve never overtakes its firing.
- Follow the alert. Acknowledgements and resolves go to every channel the alert already reached, whatever the routes say now. An escalation adds the channels its new severity routes to.
- Retries. A failed send is retried with back-off from 5 seconds to 10 minutes, up to 8 attempts, and honours
Retry-After. Bad credentials fail at once with the reason. - Visible. Every attempt shows under Notifications on the alert’s page.
- Guarded. Webhooks and other calls to addresses you configure can’t reach private, loopback, link-local or cloud metadata addresses.
Silences and flapping can hold notices back without stopping alerts being recorded.