Launch one run per target as a group (async)
Launches ONE run per target — attached project environments, or attached named hosts — under a single server-minted runGroupId, then detaches execution and responds 202 with a per-target receipt.
This is the only endpoint with grouped-launch semantics. POST /eval-runs also accepts a runGroupId, but purely as a display label: it groups sibling rows in the UI and gives N separate launches no group treatment. Here the server bounds the fan-out (max 10 targets), validates EVERY target before launching any of them (an unattached target, a heterogeneous target list, or a target whose host selects an unavailable harness returns 400 with zero runs started), and holds exactly ONE organization concurrency slot for the whole group — released only when the last sibling finishes.
Targets fan out along ONE axis: all environmentId or all namedHostId, never a mix. Duplicates are deduplicated by id, preserving order.
A per-target failure does NOT abort its siblings, so read outcome rather than treating the 202 as “everything started”: started means every target launched, partial means some did, failed means none did.
Send Idempotency-Key (or idempotencyKey) to make the launch replayable: the group id is DERIVED from the key, and each target carries a run key derived from that, so a retry after a crash mid-launch returns the original run ids instead of double-launching the targets that already started.
x-mcpjam-launcher and x-mcpjam-ci are read once and applied to EVERY sibling: one fan-out is one launch by one process, so the targets badge identically.
Authorizations
MCPJam API key (sk_…). Create one at Settings → API keys. Guest sessions cannot use the API, and API keys cannot manage other API keys.
Headers
Which vocabulary this request and its response speak. Absent means 1, which is byte-for-byte today's contract: the same request fields, the same refusals, the same response projection. 2 is the canonical vocabulary. Any other value is a 400 with code: "VALIDATION_ERROR".
Today it decides one thing: the spelling of an evaluator's policy role. Vocabulary 1 accepts and returns gating; vocabulary 2 accepts both spellings and returns the canonical required. Sending required without the header is a 400, deliberately — vocabulary 1 is not widened to meet vocabulary 2 half way, because a boundary that accepts a spelling it does not announce is one two implementations can disagree about.
A response that varies by vocabulary sends Vary: x-mcpjam-eval-vocabulary.
1, 2 The launching process's own DECLARED identity, as compact JSON: {"kind":"cli"|"mcp"|"github_action","client":"…","version":"…"}. Stored on the run as launcher and used only as a display label — source stays api and the audit trail reads the platform-minted attribution instead.
A header rather than a body field so that sending it never breaks against an older deployment: both eval-run bodies reject unknown properties, while an unknown header is ignored everywhere. For the same reason a malformed value, or a kind outside the three above, is DROPPED rather than rejected — a label must never fail a launch. Max 512 bytes.
The CI job this run was launched from, as compact JSON. Recognized keys: provider, pipelineId (or runId), jobId (or job), runUrl, branch, commitSha. Stored on the run as ciMetadata, which is what makes a run resolvable by commit for baseline comparison. Unrecognized keys are dropped.
A header for the same compatibility reason as x-mcpjam-launcher, and dropped rather than rejected when malformed. Max 2048 bytes.
Path Parameters
ID of the hosted project that contains the server.
Body
Launch one run per target under a single server-minted group id.
targets fan out along ONE axis: every entry is an environmentId, or every entry is a namedHostId. A mixed list returns 400 with details.reason: "HETEROGENEOUS_TARGETS" — mixing them would have to answer whether a host runs inside an environment or beside it, and every answer to that is a cross product nobody asked for. Duplicates are deduplicated by id, preserving order.
The knobs below apply UNIFORMLY to every run in the group. serverIds and refreshSnapshot are deliberately absent: an environment supplies its own closed server set, a host derives one from its config, and last-writer-wins on a frozen suite snapshot is never what a fan-out means. Inline tests are absent for the same reason — a group runs a persisted suite.
The suite every target runs.
One entry per run. Every target must be ATTACHED to the suite; an unattached one returns 400 with zero runs started.
1 - 10 elements- Option 1
- Option 2
Override the per-case runs count for every run in the group.
1 <= x <= 10Narrow every run in the group to these suite cases.
1Match options for every run in the group. Same public-or-internal vocabulary as POST /eval-runs.
Run every target as the "without skills" arm.
exclude The floor a run must clear, as a PERCENT in [0, 100] — 80 means 80%. minimumPassRatePercent is the canonical spelling (the unit is in the name); minimumPassRate is the deprecated alias for it. Send exactly one. THE NAME DISAMBIGUATES THE UNIT: a value in (0, 1) is accepted on minimumPassRatePercent, where it unambiguously means a sub-1% floor the evaluator can act on, and REJECTED on the bare minimumPassRate, where 0.8 cannot be told apart from a fraction sent by mistake and would silently make the gate unfailable. Neither field ever reinterprets a value. A per-case passThreshold IS a fraction; this suite/run floor is not.
- Option 1
- Option 2
Makes the whole launch replayable. The runGroupId is DERIVED from this key and each target's run key is derived from that, so a retry after a crash mid-launch returns the original run ids rather than double-launching the targets that already started. The Idempotency-Key header carries the same value and wins over this field.
256When true, each environment target may be a project-scoped, non-archived environment that is NOT attached to the suite. The launch never mutates the suite. Absent / false keeps the membership check. Probe GET /environments/capabilities (ephemeralEnvironmentLaunch) before sending — older servers reject the unknown field.
Approve approximated imported cases for THIS RUN ONLY. The caller supplies a hosted test-case id and a reason and nothing else: the approver's identity and the approval time are derived server-side and frozen into the run's snapshot, because a caller-supplied approver would file one person's approval under another's name and a caller-supplied timestamp could be backdated past the edit that invalidated the claim. Nothing persists on the case — the next run of the same approximation needs a new approval. On a grouped launch the SAME approvals go to every target: a case's approximation is approximated the same way on each of them, so one human decision covers the whole fan-out.
1Response
Group accepted. Each target's outcome is in targets; execution continues in the background.
One entry per target. A per-target failure does NOT abort its siblings, so read outcome rather than treating the 202 as "everything started".
Server-minted. Every sibling run carries it, and the hosted UI collapses them into one row.
started — every target launched; partial — some did and some did not; failed — none did (still a 202: the group itself was valid, and the per-target reasons are in targets).
started, partial, failed Discriminated on status, so a target that failed can never be mistaken for one that started with an unread runId.
Mirror of the FIRST started run, so readers written against the single-run receipt keep working. Absent when nothing started. Read targets instead — this describes one run out of several.
See runId.
See runId.
See runId.
See runId.

