Move a scheduled job into SREInsights
A script on a schedule somewhere (a crontab line, a launchd agent on a Mac, an EventBridge rule in front of a Lambda function) runs until the day it silently stops. Moved into SREInsights it gets a run history, Run now, the logs of each run, and an alert the first time it fails.
This guide moves one script. Repeat it for each.
1. Choose the shape
Section titled “1. Choose the shape”| Your script | Move it in as |
|---|---|
| Does a task: refreshes a token, publishes a post, cleans up, calls an API | A monitor with a script check. This guide. |
| Collects data day by day into storage, and should be able to backfill | A job on your own job type. See the job type contract. |
| Only checks something (a page, a row count) | A monitor with a built-in check type, no script needed. |
A script check runs Node, Python or shell, for up to 10 minutes a run. The script passes when it exits 0, or when its assertion holds.
2. Choose where it runs
Section titled “2. Choose where it runs”| Runner | When |
|---|---|
| A Lambda runner | The script only needs the internet and AWS. Nothing runs between runs. |
| A Docker runner | The script needs your private network, a local file system, or Docker. |
| Hosted runners | Not for scripts: hosted runners don’t run them. |
The runner image needs your script’s interpreter and libraries. The standard images have Node and a shell. For Python or extra packages, extend the image (see Lambda runner, step 1) and copy your code in, for example to /opt/jobs/.
3. Translate the schedule
Section titled “3. Translate the schedule”SREInsights schedules are five-field cron expressions with a time zone, which you set on the monitor.
| You have | SREInsights cron |
|---|---|
crontab */15 * * * * |
*/15 * * * * (the same) |
launchd StartInterval 900 |
*/15 * * * * |
launchd StartCalendarInterval Hour 6, Minute 30 |
30 6 * * * |
launchd StartCalendarInterval Weekday 1, Hour 9, Minute 0 |
0 9 * * 1 |
EventBridge rate(15 minutes) |
*/15 * * * * |
EventBridge rate(1 hour) |
0 * * * * |
EventBridge cron(0 6 * * ? *) (UTC) |
0 6 * * * with time zone UTC |
The shortest interval is 30 seconds (60 seconds on an instance that runs as Lambda functions). A launchd RunAtLoad has no equivalent: use Run now once after you save.
4. Decide what counts as a failure
Section titled “4. Decide what counts as a failure”By default a script fails when it exits with a code other than 0, and the failure message quotes the last lines of its output. That is usually all you need: keep the script’s own error handling, and make sure it exits non-zero when something goes wrong.
For more, the script can return JSON, and an assertion can judge it. Write the JSON to the file named in SREINSIGHTS_OUTPUT_FILE:
import json, os
result = {"refreshed": True, "expiresInDays": 59}with open(os.environ["SREINSIGHTS_OUTPUT_FILE"], "w") as f: json.dump(result, f)The assertion then reads it as data, for example exitCode == 0 && data.expiresInDays > 7. A number in data (or data.value) is recorded as the run’s value, for threshold rules and charts.
5. Handle secrets
Section titled “5. Handle secrets”Don’t put secrets in the script’s code or in the monitor’s env: both are saved with the monitor, in plain text, for anyone in the organization to read.
A script never sees connection secrets or the runner’s own secrets. It gets a short, fixed environment (PATH, HOME, locale, time zone, temp folder, CA and proxy settings), its own env, and whatever the runner’s operator lets it inherit with SREINSIGHTS_SCRIPT_ENV_PASSTHROUGH. Use that list:
-
On a Lambda runner: pass the function’s role through, and read secrets from AWS Secrets Manager in the script:
Terminal window SREINSIGHTS_SCRIPT_ENV_PASSTHROUGH=AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY,AWS_SESSION_TOKEN,AWS_REGIONGive the function’s role
secretsmanager:GetSecretValueon just the secrets your scripts need. Read the warning about what else that role can reach. -
On a Docker runner: set the variable on the runner container and list its name:
Terminal window -e INSTAGRAM_APP_SECRET=… -e SREINSIGHTS_SCRIPT_ENV_PASSTHROUGH=INSTAGRAM_APP_SECRET
The list applies to runtime: local scripts only. Names starting SREINSIGHTS_SECRET_, the registration token and the runner’s identity are never passed.
6. Create the monitor
Section titled “6. Create the monitor”-
Monitors → New monitor → Check types → Script (under Code).
-
Name it after what it does:
Refresh Instagram token. -
Choose the language, keep the runtime local, and write the script. To run code you copied into the image, keep the script short:
Terminal window set -ecd /opt/jobspython3.12 refresh_token.pyWrite with ADAM can draft or change the script from a description.
-
Set the timeout: the default is 60 seconds, at most 10 minutes.
-
Schedule (cron) and Time zone from step 3.
-
Runner group: your Lambda group or Docker runner group.
-
Select Create monitor, then Run now on its page. Expand the run to see the exit code, the output and the returned data.
7. Get told when it fails
Section titled “7. Get told when it fails”The monitor already raises a high alert called Check failing when a run fails, and resolves it on the next success. Route it to a channel (Send alerts to your team). If one failure is too noisy for a job that runs often, add a rule such as “2 of the last 3 runs failed” (Alert rules).
Watch for these too:
skipped/no_runner: no runner took the run. It fires no alert. On a Lambda group, look at the function’s CloudWatch logs.timeout: the script ran past its timeout.
8. Switch the old schedule off
Section titled “8. Switch the old schedule off”Once a few runs have passed in SREInsights, stop the old one, so the work doesn’t run twice:
# cron: remove the linecrontab -e
# launchd: unload the agentlaunchctl bootout gui/$(id -u) ~/Library/LaunchAgents/com.example.refresh-token.plist
# EventBridge: disable the ruleaws events disable-rule --name refresh-token-every-15-minKeep the old definition somewhere for a while, in case you need to go back.
A worked example
Section titled “A worked example”A Python script refreshes a long-lived API token every day at 03:00 and stores it in AWS Secrets Manager. It used to be a launchd agent on a laptop.
- Extend the Lambda runner image with Python and
boto3, copy the script to/opt/jobs/refresh_token.py, and push it under a new tag. - On the function, set
SREINSIGHTS_SCRIPT_ENV_PASSTHROUGHto the four AWS variables, and give its role read and write access to that one secret. - Create a monitor Refresh API token: script, shell,
python3.12 /opt/jobs/refresh_token.py, timeout 120 000 ms, schedule0 3 * * *, your time zone, runner groupjobs. - Run now and check the output.
- Route Check failing alerts for it to your chat.
launchctl bootoutthe old agent.
The laptop can now be off at 03:00.