Job type contract
This page is for people who build a runner image that provides job types. It covers the manifest that lists the types, the input file and environment the type’s command gets, the result it writes, and the rules it must follow. It is also the contract you use to move an existing scheduled script into SREInsights as a job type (see Move a scheduled job in).
To use the job types an image provides, see Job types.
How it fits together
Section titled “How it fits together”- SREInsights knows a job type only as data: its name, version, settings and connection roles. The list is on Job types.
- A runner image carries the code, and lists the types it provides in a manifest.
- The runner reads the manifest when it starts and reports each type as a capability,
job:<name>@<version>. - A job’s run goes only to a runner that reported its type at its version.
- For each run, the runner writes an input file, starts the manifest’s command, and reads the result the command wrote to an output file.
sync is not in any manifest: the runner provides it itself.
The manifest
Section titled “The manifest”The runner reads the manifest from /etc/sreinsights/job-types.json, or from the file the environment variable SREINSIGHTS_JOB_TYPES_MANIFEST names.
{ "manifestVersion": 1, "jobTypes": [ { "name": "search_console", "version": 1, "command": ["/opt/jobs/bin/collect", "search_console"], "timeoutMs": 600000 }, { "name": "shopify_orders", "version": 1, "command": ["/opt/jobs/bin/collect", "shopify_orders"], "cwd": "/opt/jobs" } ]}| Field | Type | Required | Default | What it does |
|---|---|---|---|---|
manifestVersion |
1 |
yes | The manifest’s format. Always 1. |
|
jobTypes |
array, at most 100 | yes | The types this image provides. | |
jobTypes[].name |
text | yes | The type’s name: lower-case letters, digits and _, starting with a letter, up to 30 characters. |
|
jobTypes[].version |
integer, 1 to 1,000 | yes | The type’s version. Each name and version may be listed once. | |
jobTypes[].command |
array of 1 to 50 strings | yes | What to run, as argv. There is no shell, so nothing in it is interpreted. The runner adds no arguments: the run’s input is in the input file. | |
jobTypes[].cwd |
path | no | the run’s own temporary directory | The command’s working directory. |
jobTypes[].timeoutMs |
integer, 1,000 to 900,000 | no | The longest a run of this type may take on this image, whatever the job asks for. |
- An image without a manifest provides no job types.
- A manifest that
SREINSIGHTS_JOB_TYPES_MANIFESTnames and that is missing, or any manifest that is not valid, stops the runner from starting. The error says what is wrong, field by field. - The runner logs the types it provides at start, for example
this image provides job typeswithsearch_console@1. - To move jobs from one version of a type to the next, list both versions while the jobs move.
The input file
Section titled “The input file”For each run, the runner writes input.json in a fresh directory and names it in SREINSIGHTS_INPUT_FILE. The file is mode 0600, the directory 0700, and both are deleted after the run.
Connection secrets are in this file and nowhere else: not in the environment, not in argv, not in the runner’s logs.
{ "inputVersion": 1, "jobType": { "name": "shopify_customers", "version": 1 }, "job": { "id": "0190c3c4-0000-7000-8000-00000000b001", "name": "Shopify customers · acme-shop" }, "orgId": "0190c3c4-0000-7000-8000-000000000001", "client": { "id": "0190c3c4-0000-7000-8000-00000000c001", "name": "Acme" }, "run": { "id": "0190c3c4-0000-7000-8000-00000000d001", "attempt": 1, "scheduledFor": "2026-11-22T09:05:00.000Z", "trigger": "schedule", "deadline": "2026-11-22T09:14:58.000Z" }, "params": { "site": "acme-shop" }, "window": null, "cursor": "2026-11-21T09:05:12Z", "connections": { "shopify": { "id": "0190c3c4-0000-7000-8000-00000000a001", "type": "shopify_admin", "name": "Acme store", "config": { "shopDomain": "acme.myshopify.com", "apiVersion": "2025-10" }, "secrets": { "accessToken": "shpat_…" } }, "destination": { "id": "0190c3c4-0000-7000-8000-00000000a002", "type": "s3_destination", "name": "Acme data", "config": { "bucket": "acme-data-lake", "prefix": "orgs/0190c3c4-0000-7000-8000-000000000001/clients/0190c3c4-0000-7000-8000-00000000c001/", "region": "us-east-1", "accessKeyId": "AKIA…" }, "secrets": { "secretAccessKey": "…" } } }, "hosted": false}| Field | Type | What it holds |
|---|---|---|
inputVersion |
1 |
The input file’s format. |
jobType |
{ name, version } |
The type and version this run is for. |
job |
{ id, name } |
The job. |
orgId |
text or null |
The run’s organization, from the job. Use it to refuse a destination prefix that isn’t this organization’s. |
client |
{ id, name } or null |
The job’s client inside the organization, or null for the organization’s own. |
run.id |
text | The run. |
run.attempt |
number | 1 for the first try. A run is tried again after a lost lease, up to three attempts in all. |
run.scheduledFor |
ISO time | When the run was due. |
run.trigger |
schedule, manual or backfill |
What started it: its schedule, Run now, or Backfill. |
run.part |
number | Only on the second and later parts of a Backfill: which part this is. |
run.deadline |
ISO time | When the runner stops the command. Write the output file before then. |
params |
object | The job’s settings, in their declared types (text, number, boolean). Run now and Backfill may override them for one run. |
window |
{ start, end } or null |
Days to collect, YYYY-MM-DD, both included. Set by Backfill. null: decide from the cursor or your own default. |
cursor |
text or null |
What the job’s last run that wrote data reported. Not sent with a window. |
connections |
object: role → connection | Each connection the job chose, opened for this run: { id, type, name, config, secrets }. config has the fields that aren’t secret, secrets the secret ones. See Connection types. |
hosted |
boolean | true on a hosted runner. See Egress on hosted runners. |
A type on its runner’s own credentials (credentials: "runner") gets no connections: it uses the variables the runner passes on.
Environment variables
Section titled “Environment variables”The command gets a short fixed list from the runner’s environment, and nothing else of it:
PATH,HOME,LANG,LC_ALL,TZ,TMPDIRSSL_CERT_FILE,NODE_EXTRA_CA_CERTS,HTTPS_PROXY,HTTP_PROXY,NO_PROXY
Plus the variables the runner’s operator lists in SREINSIGHTS_SCRIPT_ENV_PASSTHROUGH (comma-separated names). A hosted runner passes none. Connection secrets kept on the runner (SREINSIGHTS_SECRET_*), the registration token and the runner’s identity are never passed, whatever the list says. See Runner settings.
And these, set for the run:
| Variable | Value |
|---|---|
SREINSIGHTS_INPUT_FILE |
The input file’s path. |
SREINSIGHTS_OUTPUT_FILE |
Where to write the result. |
SREINSIGHTS_WORKDIR |
The run’s temporary directory, which holds both files. It is deleted after the run. |
SREINSIGHTS_JOB_TYPE |
The type’s name, for example shopify_customers. |
SREINSIGHTS_JOB_TYPE_VERSION |
The type’s version, for example 1. |
SREINSIGHTS_JOB_DEADLINE |
The same as run.deadline. |
SREINSIGHTS_RUNNER_HOSTED |
true or false, the same as hosted. |
The output
Section titled “The output”The command writes its result as JSON to SREINSIGHTS_OUTPUT_FILE before the deadline.
{ "status": "completed", "rows": 9511, "window": { "start": "2026-11-18", "end": "2026-11-21" }, "cursor": "2026-11-22T09:06:40Z", "tables": { "gsc_queries": 9100, "gsc_pages": 411 }, "details": { "apiCalls": 42 }}| Field | Type | Required | Limits | What it does |
|---|---|---|---|---|
status |
completed, partial or failed |
yes | See Statuses. | |
rows |
integer ≥ 0 | no (default 0) | Rows written, all tables together. | |
window |
{ start, end } |
no | YYYY-MM-DD, start not after end |
The days the run’s data covers. |
cursor |
text | no | up to 2,000 characters | Where the next run carries on from, such as the last updated_at seen. Stored on the job and passed to its next scheduled or Run now run. Ignored on a Backfill run. |
nextDay |
YYYY-MM-DD |
no | For a run given a window that stopped before its end, with status partial: the first day it did not collect. SREInsights queues the next part of the Backfill from here. |
|
error |
text | no | up to 2,000 characters | What went wrong, or what is missing from a partial run. |
tables |
object: table → rows | no | at most 50 tables, names up to 100 characters | Rows per table. |
details |
object | no | at most 8 KB as JSON | Anything else small worth showing on the run. |
findings |
array | no | at most 50 | Only for a type with findings. See Findings. |
The whole result must be at most 48 KB (49,152 bytes) as JSON. That is the runner protocol’s 64 KB result limit, less room for the summary and the command’s last output lines that travel with it.
A result over any limit is refused, and the run is an error that says which field. For example: shopify_customers wrote a result SREInsights can't read: details: must be at most 8000 bytes.
Findings
Section titled “Findings”Only a type marked with findings on Job types (today anomaly_scan) may report them. A run of any other type that reports findings is an error. Each finding becomes an alert while runs keep reporting it. See Findings.
{ "status": "completed", "rows": 0, "findings": [ { "key": "clicks_drop:/products/ring", "title": "Clicks to /products/ring fell 62% week on week", "severity": "high", "detail": "412 clicks last week, 157 this week.", "value": 62, "data": { "page": "/products/ring", "before": 412, "after": 157 } } ]}| Field | Type | Required | Limits | What it does |
|---|---|---|---|---|
key |
text | yes | 1 to 200 characters, no control characters, unique in the result | Stable across runs for the same thing. What comes before the first : names the detector, unless data.detector does. |
title |
text | yes | 1 to 300 characters | One line: the alert’s title. |
severity |
critical, high, medium or low |
yes | The alert’s severity. | |
detail |
text | no | up to 2,000 characters | Why, in words: the alert’s detail. |
value |
number | no | finite | The number the finding is about, such as a drop in percent. |
data |
object | no | at most 2 KB as JSON | Anything else small to keep with the alert. |
Statuses
Section titled “Statuses”| Status | Meaning | The run is |
|---|---|---|
completed |
Everything in the window was collected and written. | ok, data arrived |
partial |
Some of it was written. error says what is missing. |
ok, data arrived |
failed |
Nothing usable was written. | failed |
completed and partial count as data arriving: they move the cursor, resolve the job’s failed alert and keep its no new data alert quiet. Only a completed run resolves findings that it no longer reports.
What decides success
Section titled “What decides success”The result decides, not the exit code. A command may exit non-zero after writing partial, and the run is still partial.
- A command that exits without writing a result is an
error: “type exited with code 1 and it wrote no result to SREINSIGHTS_OUTPUT_FILE”, followed by its last lines of stderr (or stdout). - An output file that is not JSON is an
errorthat says so. - A command that isn’t in the image is an
errorthat names it and says to check the manifest’scommand. - The last 2,000 characters of stdout and stderr are kept with the run.
Timeouts
Section titled “Timeouts”A run’s time limit is the smaller of the job’s Time limit and the manifest’s timeoutMs for the type. The job’s limit is at most 15 minutes, which is also when a Lambda function stops.
run.deadlineandSREINSIGHTS_JOB_DEADLINEare 2 seconds before the limit, to leave the runner time to read the result and report it.- A command still running at the limit is stopped, and the run is recorded as
timeout. - For a long range, stop before the deadline and write
partialwithnextDay. SREInsights runs the rest as the next part of the Backfill.
What a job type must do
Section titled “What a job type must do”- Be idempotent per window. A lost lease runs it again, up to three attempts in all. Running the same window twice must give the same data, not twice the rows.
- Print no secrets. The ends of stdout and stderr are kept with the run and shown on its page.
- Write only to its destination. A type on its runner’s credentials writes only where those allow.
- Check the destination prefix. See Destination prefix.
- Connect only where its egress allows on a hosted runner. See below.
Destination prefix
Section titled “Destination prefix”By convention a run’s data goes under one prefix per organization, and one per client inside it:
| Job | Prefix |
|---|---|
| A client’s job | orgs/<orgId>/clients/<clientId>/ |
| The organization’s own job | orgs/<orgId>/ |
Take orgId and client.id from the input file. A job type should refuse a destination whose prefix isn’t this, and the destination’s credentials should allow nothing else, so no client’s run writes where another client’s data is.
Egress on hosted runners
Section titled “Egress on hosted runners”The runner’s egress guard covers the runner process, not the command it starts. So a job type guards itself when hosted is true. A type declares egress:
provider: it calls only its provider’s own API hosts, such as Google, Shopify or S3.public: it may also fetch public URLs that its settings lead to, such as a site’ssitemap.xmland the child sitemaps it lists, which may be on a CDN.
Either way the command must never connect to private, loopback, link-local (cloud metadata), CGNAT, reserved or multicast addresses. Check the address when you connect, after DNS, so a name that resolves to a private address is refused too.
Connection fields that name a host are checked for shape when they are saved. For example, a Shopify store must be a *.myshopify.com address.
A complete example
Section titled “A complete example”This example adds a command for shopify_customers version 1 to a runner image. Replace the collect function with your own code, such as the body of the scheduled script you are moving in.
The manifest
Section titled “The manifest”Save as job-types.json:
{ "manifestVersion": 1, "jobTypes": [ { "name": "shopify_customers", "version": 1, "command": ["/nodejs/bin/node", "/opt/jobs/shopify-customers.mjs"], "timeoutMs": 600000 } ]}/nodejs/bin/node is Node’s path in the Docker runner image. In the Lambda runner image, use "node".
The command
Section titled “The command”Save as jobs/shopify-customers.mjs:
import { readFileSync, writeFileSync } from "node:fs";
const input = JSON.parse(readFileSync(process.env.SREINSIGHTS_INPUT_FILE, "utf8"));
async function main() { // Write only under this job's own prefix. const destination = input.connections.destination; const expected = input.client ? `orgs/${input.orgId}/clients/${input.client.id}/` : `orgs/${input.orgId}/`; if (destination.config.prefix !== expected) { return { status: "failed", error: `The destination's prefix is "${destination.config.prefix}", not this job's own (${expected}).` }; }
const { rows, cursor } = await collect(input); return { status: "completed", rows, cursor, tables: { shopify_customers: rows } };}
// Your code: read from input.connections.shopify, write to the destination's// bucket and prefix with its config and secrets, and return what you wrote.async function collect({ cursor, connections }) { return { rows: 0, cursor: cursor ?? new Date().toISOString() };}
let result;try { result = await main();} catch (err) { // stderr is kept with the run: never print a secret. console.error(err.message); result = { status: "failed", error: String(err.message).slice(0, 2000) };}writeFileSync(process.env.SREINSIGHTS_OUTPUT_FILE, JSON.stringify(result));The image
Section titled “The image”Extend the Docker runner image (ghcr.io/bintree-softwares/sreinsights-runner):
FROM ghcr.io/bintree-softwares/sreinsights-runner:latestCOPY job-types.json /etc/sreinsights/job-types.jsonCOPY jobs/ /opt/jobs/The Docker runner image is distroless Node 22, running as a non-root user. It has no shell and no package manager, so RUN steps don’t work in it: add files with COPY, and bring your command’s dependencies with it. A Python or shell command needs an image that has Python or a shell.
Or extend the Lambda runner image, for a Lambda runner group. It carries Node and a shell, and you can install more:
FROM 123456789012.dkr.ecr.us-east-1.amazonaws.com/sreinsights-runner-lambda:latest# Only if your command is in Python:# RUN dnf install -y python3.12 python3.12-pip && dnf clean allCOPY job-types.json /etc/sreinsights/job-types.jsonCOPY jobs/ /opt/jobs/- Keep the base image’s handler and entry point: an extension adds files only.
/var/taskand/optare read-only at run time. Write under/tmp; the run’s own directory is there.- Lambda takes a single-platform image without attestations, from ECR. Build with
docker buildx build --platform linux/arm64 --provenance=false … --push.
Check it
Section titled “Check it”- Start a runner from the new image. Its log shows
this image provides job typeswithshopify_customers@1. - In the job form, the runner’s group no longer shows · no runner with this type yet for the type.
- Save a job of the type on that group, and press Run now. The run’s result and the last lines of the command’s output show on the job’s page.