Skip to content

Receive alerts from other tools

SREInsights can take alerts that other tools raise. Each becomes an SREInsights alert, labelled connection=<connection name>, and resolves when the tool says it recovered. Your routes, silences and severity handling apply as for any other alert.

Tool Sends
Prometheus Alertmanager (also Rancher Monitoring) Each Prometheus alert, per pod or container
Kibana alerting Each alert of a rule, when it starts and when it recovers
Dynatrace Each problem, until Dynatrace closes it

On Connections, select Connect on the tool’s card and save. SREInsights shows:

  • the ingest URL, https://console.sreinsights.com/api/v1/ingest/<type>/<connection id>,
  • a token (pit_…), shown once. SREInsights keeps only its hash.

The connection’s setup drawer shows the exact steps and the payload or configuration to paste, filled in with your URL.

Every call needs Authorization: Bearer pit_… and Content-Type: application/json.

receivers:
- name: sreinsights
webhook_configs:
- url: https://console.sreinsights.com/api/v1/ingest/alertmanager/<connection id>
send_resolved: true
http_config:
authorization:
type: Bearer
credentials: pit_…

Keep send_resolved: true, or alerts never resolve in SREInsights. The alert’s severity label sets its severity (critical and page → critical, warning → medium, and so on); other labels become alert labels.

In Stack Management → Connectors, create a Webhook connector with method POST, the ingest URL and both headers. On each rule, add an action with it for the firing action group and one for Recovered, with the body shown in the connection’s drawer. Tag a rule severity:critical to set its severity; tags such as team:payments become labels.

In Settings → Integration → Problem notifications, add a Custom integration with the ingest URL and both headers, turn on Call webhook if problem is closed, and paste the payload from the connection’s drawer. Choose the severity for each kind of problem on the connection.

To catch problems whose webhook was lost, also add a dynatrace check: a runner lists open problems and SREInsights updates the alerts as the webhook would.

Incoming alerts have the tool as their source (alertmanager, kibana, dynatrace). Route them by severity or labels like any alert (Send alerts to your team).

On the connection, Create a new token? makes a new one. The old token stops working at once, so update the tool straight after.

  • 401 INGEST_UNAUTHORIZED: the token is missing or wrong, or the connection doesn’t exist. The answer is the same in each case, so it never confirms which connections exist.
  • 400 INGEST_PAYLOAD_INVALID: the body isn’t what that tool sends. The answer says what to fix in the tool’s setup.

Fields and payloads in full: Where alerts come from.