Runs
The core primitive. POST /v1/runs accepts a kind (run | eval | verify | matrix | threshold | compare | transfer) with kind-specific parameters mirroring the SDK.
POST /v1/runs
Create a run (any kind)
Creates a physical evaluation job. The `kind` field selects the primitive: `run` (default), `eval`, `verify`, `matrix`, `threshold`, `compare`, or `transfer`. Episode counts accept an integer or `auto(ci=…, moe=…)`, which sizes n for a target margin of error (Wilson interval). `max_budget_usd` is a hard stop, metered live.
| Parameter | In | Type | Description |
|---|---|---|---|
X-Roborama-Force | header | string | Test mode only: force this documented error code on any request — the response has the code's real shape and HTTP status. Three codes are in-band on POST /v1/runs (`cell_fault`, `estop_triggered`, `budget_exceeded`): the created run demonstrates the documented behavior end to end instead of returning an immediate error. Ignored with live keys. |
Idempotency-Key | header | string | Makes the POST safely retryable: the same key + body always returns the ORIGINAL run. In test mode the id (including its creation time) becomes a pure function of (key, body); the derived creation time sits in the past, so a retried request typically finds the run already completed. |
kind | body | run | eval | verify | matrix | threshold | compare | transfer | The primitive this run executes. |
robot | body | string | `model@firmware`; `@latest` allowed, resolved pin recorded. |
robots | body | array<string> | For eval | verify | matrix. |
environment | body | string | |
environments | body | array<string> | For matrix. |
policy | body | Policy | Policy packaging. Declared contracts (`action_space`, `observation_contract`) are validated before any motor moves. The runtime currently accepts only the reference pickup subset documented in runtime/deploy/PICKUP.md; other packaging is not installed there. |
policies | body | object | For compare: name → policy. |
policy_stream | body | object | For threshold: a webhook that receives checkpoint pushes. |
task | body | string | |
suite | body | string | For eval. |
publish | body | private | leaderboard | Leaderboard publication is opt-in only. |
scenarios | body | object | For verify: sim-flagged initial conditions. A bare list of scenarios, a {format, source, count} descriptor, or upload the JSONL and pass scenarios_upload_id. |
audit_sample | body | number | For verify: fraction of random episodes run against the sim's blind spots. |
return_ground_truth | body | boolean | For verify: ship mocap 6-DoF poses + commanded-vs-executed, formatted to recalibrate your simulator. |
source | body | string | For transfer: embodiment the policy was trained on. |
target | body | object | For transfer: destination embodiment. For threshold: the target object. |
iterate_on | body | string | For threshold: commodity tier to iterate on. |
escalate_to | body | string | For threshold: verification-tier embodiment for confirmation. |
monthly_cap_usd | body | number | For threshold: hard monthly cap. |
on_verified | body | object | threshold(): where to deliver run.completed / threshold.crossed once the target verifies. All documented shapes are accepted. |
paired | body | boolean | For compare: same initial conditions per episode pair — halves the n needed to separate two policies. |
episodes | body | EpisodesSpec | A positive integer, or "auto(ci=..., moe=...)": run until the Wilson interval at the requested confidence fits the target margin of error. Sizing is a genuine sequential stopping rule on the run's own episode stream (checked from n=50 with a look penalty so repeated looks don't eat coverage), hard-capped at the worst-case p=0.5 planning n. The delivered interval therefore meets the request with margin. The published canonical runs reproduce their fixture results exactly (fixture parity). |
episodes_per_cell | body | EpisodesSpec | A positive integer, or "auto(ci=..., moe=...)": run until the Wilson interval at the requested confidence fits the target margin of error. Sizing is a genuine sequential stopping rule on the run's own episode stream (checked from n=50 with a look penalty so repeated looks don't eat coverage), hard-capped at the worst-case p=0.5 planning n. The delivered interval therefore meets the request with margin. The published canonical runs reproduce their fixture results exactly (fixture parity). |
perturbation | body | Perturbation | Seeded, versioned, replayable perturbation schedule. The schedule IS the product spec; realized perturbations are recorded in `/meta`. |
data | body | DataPosture | Customer IP posture, explicit per run. |
priority | body | burst | standard | soak | Maps to the rate-card priority tiers. |
max_budget_usd | body | number | Hard stop, metered live. |
interventions | body | none | on_stall | scripted | Intervention policy, declared before the run. Counts and timestamps ship in results, and any intervened episode is flagged in the statistics. |
scenarios_upload_id | body | string | An upl_ id from POST /v1/uploads holding scenario JSONL (one scenario per line). |
limits | body | object |
Example request
kind=run — the core primitive
{
"kind": "run",
"robot": "g1-edu-pro@fw2.3",
"environment": "kitchen-std@v1.2",
"policy": {
"type": "container",
"image": "ghcr.io/acme/skill:v4",
"action_space": "joint_delta_50hz",
"observation_contract": "droid-3cam"
},
"task": "load_dishwasher@v2",
"episodes": "auto(ci=0.95, moe=0.03)",
"perturbation": {
"layout_jitter_mm": 25,
"lighting": [
"3000K",
"5600K"
],
"distractors": "set-B",
"seed": 42
},
"data": {
"retention": "30d",
"train_on_failures": false
},
"max_budget_usd": 4000
}kind=eval — versioned, citable suite
{
"kind": "eval",
"policy": {
"type": "checkpoint",
"hf": "acme/skill-v4",
"runtime": "openpi"
},
"suite": "rbr-manip-core@v3",
"robots": [
"g1-edu-pro@fw2.3"
],
"publish": "private"
}kind=verify — the interlock with simulation
{
"kind": "verify",
"policy": {
"type": "checkpoint",
"hf": "acme/skill-v4",
"runtime": "openpi"
},
"scenarios": {
"format": "layout-replay",
"source": "sim_flagged_p1.json",
"count": 318
},
"robots": [
"g1-edu-pro@fw2.3",
"g1-edu-plus@fw2.3"
],
"audit_sample": 0.1,
"return_ground_truth": true
}kind=matrix — embodiments × environments
{
"kind": "matrix",
"policy": {
"type": "checkpoint",
"hf": "acme/skill-v4",
"runtime": "openpi"
},
"robots": [
"g1-edu-pro@fw2.3",
"g1-edu-pro@fw2.4",
"g1-edu-plus@fw2.3",
"aloha2-pro@fw1.1",
"spot-arm@fw4.1",
"tiago-pro@fw2.0",
"fr3-bench@fw5.2",
"stretch3@fw1.9",
"nori-a3@fw1.0",
"so-101@fw1.2"
],
"environments": [
"cell-a",
"kitchen-std@v1.2",
"warehouse-std@v2.0"
],
"task": "pick_place_class3",
"episodes_per_cell": "auto(ci=0.95, moe=0.05)"
}kind=threshold — the soak-to-verify outcome contract
{
"kind": "threshold",
"policy_stream": {
"webhook": "https://acme.ai/ckpt"
},
"target": {
"success_rate": 0.99,
"ci": 0.95,
"task": "bin_pick@v1"
},
"iterate_on": "soak",
"escalate_to": "g1-edu-pro@fw2.3",
"monthly_cap_usd": 12000,
"on_verified": {
"webhook": "https://acme.ai/release-gate"
}
}kind=compare — paired A/B
{
"kind": "compare",
"policies": {
"v3": {
"type": "checkpoint",
"hf": "acme/skill-v3",
"runtime": "openpi"
},
"v4": {
"type": "checkpoint",
"hf": "acme/skill-v4",
"runtime": "openpi"
}
},
"paired": true,
"robot": "g1-edu-pro@fw2.3",
"task": "shelf_restock@v1"
}kind=transfer — cross-embodiment gap
{
"kind": "transfer",
"policy": {
"type": "checkpoint",
"hf": "acme/skill-v4",
"runtime": "openpi"
},
"source": "aloha-bimanual@fw3.1",
"target": "g1-edu-pro@fw2.3",
"task": "fold_towel@v2"
}Task-pinned matrix with interventions declared (espresso@v1)
{
"kind": "matrix",
"task": "espresso@v1",
"policy": {
"type": "checkpoint",
"hf": "pi/espresso-v7",
"runtime": "openpi",
"inference": {
"action_horizon": 50,
"chunk_size": 50,
"temp": 0
},
"action_space": "joint_delta_50hz",
"observation_contract": "droid-3cam"
},
"robots": [
"g1-edu-pro@fw2.3",
"g1-edu-pro@fw2.4",
"g1-edu-plus@fw2.3",
"stretch3@fw1.9"
],
"environments": [
"kitchen-std@v1.2",
"kitchen-replica@v2.0"
],
"episodes_per_cell": "auto(ci=0.95, moe=0.05)",
"interventions": "none"
}Example response (201)
{
"id": "run_8842",
"object": "run",
"kind": "run",
"status": "queued",
"created": "2026-08-28T14:02:11Z",
"robot": "g1-edu-pro@fw2.3",
"environment": "kitchen-std@v1.2",
"task": "load_dishwasher@v2",
"episodes": "auto(ci=0.95, moe=0.03)",
"priority": "standard",
"max_budget_usd": 4000
}GET /v1/runs
List runs
Test mode is stateless: without `run_ids` this returns the published canonical runs (the docs' worked examples), not your account's history — the response says so in-band. Pass `run_ids=<id,id,...>` to list the runs you created; `kind`, `status` and `limit` filter either listing. Live mode lists account runs, newest first.
| Parameter | In | Type | Description |
|---|---|---|---|
run_ids | query | string | Comma-separated run ids to list (stateless test mode cannot see runs it wasn't shown). GET /v1/usage?run_ids=... aggregates the same way. |
kind | query | run | eval | verify | matrix | threshold | compare | transfer | Filter by primitive kind. |
status | query | queued | scheduling | running | completed | halted_budget | halted_facility | failed | stopped | |
limit | query | integer |
Example response (200)
{
"object": "list",
"url": "/v1/runs",
"data": [
{
"id": "run_8842",
"object": "run",
"kind": "run",
"status": "completed",
"created": "2026-08-28T14:02:11Z",
"robot": "g1-edu-pro@fw2.3",
"environment": "kitchen-std@v1.2",
"task": "load_dishwasher@v2",
"result": {
"n": 612,
"success_rate": 0.874,
"ci95": [
0.846,
0.898
],
"interval_method": "wilson",
"failure_clusters": {
"grasp_slip": 41,
"perception_miss": 22,
"collision": 9
},
"robot_hours": 20.4,
"environment_hours": 20.4,
"cost_usd": 3812
}
}
]
}GET /v1/runs/{run_id}
Retrieve a run
Returns the run, its resolved pins (`@latest` is allowed on create, but the resolved pin is recorded), and — once completed — the result with n, rate, CI, failure clusters, both meters, and artifact links.
| Parameter | In | Type | Description |
|---|---|---|---|
run_id * | path | string |
Example response (200)
{
"id": "run_8842",
"object": "run",
"kind": "run",
"status": "completed",
"created": "2026-08-28T14:02:11Z",
"completed": "2026-08-29T11:37:48Z",
"robot": "g1-edu-pro@fw2.3",
"environment": "kitchen-std@v1.2",
"task": "load_dishwasher@v2",
"policy": {
"type": "container",
"image": "ghcr.io/acme/skill:v4",
"action_space": "joint_delta_50hz",
"observation_contract": "droid-3cam"
},
"episodes": "auto(ci=0.95, moe=0.03)",
"priority": "standard",
"perturbation": {
"layout_jitter_mm": 25,
"lighting": [
"3000K",
"5600K"
],
"distractors": "set-B",
"seed": 42
},
"data": {
"retention": "30d",
"train_on_failures": false
},
"max_budget_usd": 4000,
"resolved_pins": {
"robot": "g1-edu-pro@fw2.3",
"environment": "kitchen-std@v1.2",
"task": "load_dishwasher@v2",
"perturbation_rev": "set-B@v1.4"
},
"result": {
"n": 612,
"success_rate": 0.874,
"ci95": [
0.846,
0.898
],
"interval_method": "wilson",
"failure_clusters": {
"grasp_slip": 41,
"perception_miss": 22,
"collision": 9
},
"robot_hours": 20.4,
"environment_hours": 20.4,
"cost_usd": 3812,
"artifacts": {
"mcap": "https://api.roborama.com/v1/runs/run_8842/episodes?artifact=mcap",
"video": "https://api.roborama.com/v1/runs/run_8842/episodes?artifact=video",
"ground_truth": "https://api.roborama.com/v1/runs/run_8842/episodes?artifact=ground_truth",
"report_pdf": "https://api.roborama.com/v1/runs/run_8842/report?format=pdf"
}
}
}GET /v1/runs/{run_id}/episodes
List episodes for a run
Every episode carries video, an MCAP recording (channels: `/joint_states_measured`, `/joint_targets_commanded`, `/camera/*`, `/ft_wrist`, `/gt/object_poses`, `/events`, `/meta`) and a replay spec — the initial conditions, re-runnable or sim-loadable. `/meta` carries reproducibility metadata: firmware, calibration age, actuator cycle counts, thermal state, seed, and realized (not just requested) perturbations.
| Parameter | In | Type | Description |
|---|---|---|---|
run_id * | path | string | |
outcome | query | success | failure | Filter by outcome. |
cluster | query | string | Filter failures by cluster, e.g. `grasp_slip`. |
Example response (200)
{
"object": "list",
"url": "/v1/runs/run_8842/episodes",
"data": [
{
"id": "ep_17",
"object": "episode",
"index": 17,
"outcome": "success",
"duration_s": 118,
"video_url": "https://artifacts.roborama.com/run_8842/ep_17/video.mp4",
"mcap_url": "https://artifacts.roborama.com/run_8842/ep_17/episode.mcap",
"replay_spec": {
"environment": "kitchen-std@v1.2",
"layout_seed": 42017,
"lighting": "5600K",
"distractors": "set-B"
}
}
]
}GET /v1/runs/{run_id}/report
Retrieve the verification report
The deployment verification report: conformance evidence with n, rate, CI, failure clusters, resolved pins, and the perturbation schedule as realized. JSON by default; `format=pdf` returns the citable PDF.
| Parameter | In | Type | Description |
|---|---|---|---|
run_id * | path | string | |
format | query | json | pdf |
Example response (200)
{
"object": "report",
"run_id": "run_8842",
"kind": "verification",
"result": {
"n": 612,
"success_rate": 0.874,
"ci95": [
0.846,
0.898
],
"interval_method": "wilson"
},
"resolved_pins": {
"robot": "g1-edu-pro@fw2.3",
"environment": "kitchen-std@v1.2",
"task": "load_dishwasher@v2",
"perturbation_rev": "set-B@v1.4"
},
"citation": "Roborama run run_8842, 2026-08-29, g1-edu-pro@fw2.3 × kitchen-std@v1.2",
"pdf_url": "https://api.roborama.com/v1/runs/run_8842/report?format=pdf"
}POST /v1/runs/{run_id}/export
Export run data
Packages the run's episodes for training or analysis. Formats: `lerobot`, `rlds`, `mcap-bundle`. Respects the run's `data.train_on_failures` posture. Test mode returns `status: "ready"` with an immediate `download_url` (exports are structurally-correct 3-episode samples); live mode may return `preparing` first.
| Parameter | In | Type | Description |
|---|---|---|---|
run_id * | path | string | |
format * | body | lerobot | rlds | mcap-bundle |
Example request
{
"format": "lerobot"
}Example response (202)
{
"object": "export",
"id": "exp_ab12cd34ef",
"run_id": "run_s...",
"format": "lerobot",
"status": "ready",
"download_url": "https://api.roborama.com/v1/runs/run_s.../export.zip?e=...&sig=...&format=lerobot",
"note": "test-mode exports are structurally correct samples (first 3 episodes)."
}GET /v1/runs/{run_id}/events
Watch a run (server-sent events)
Live event stream: episode ticker, status transitions, and WebRTC stream URLs for the cell. Episode indices are 0-based (`episode: 0` is the first). Late subscribers get past events replayed instantly, then the future on schedule. Equivalent to `run.watch()` in the SDKs.
| Parameter | In | Type | Description |
|---|---|---|---|
run_id * | path | string |
Example response (200)
"event: episode\ndata: {\"episode\": 213, \"of\": 612, \"outcome\": \"success\", \"duration_s\": 121}\n\nevent: status\ndata: {\"status\": \"running\", \"cell\": \"cell-g1-04\", \"webrtc\": \"wss://streams.roborama.com/cells/cell-g1-04/webrtc\"}\n\n"POST /v1/runs/{run_id}/poke
Deliver any due webhook events for a run now
Evaluates the run's timeline and synchronously delivers any webhook events that are due but undelivered, returning the delivery records. The documented CI idiom in test mode, where a stateless service has no cron; harmless in production, where deliveries are already push-based.
| Parameter | In | Type | Description |
|---|---|---|---|
run_id * | path | string |
Example response (200)
{
"object": "list",
"run_id": "run_s...",
"status": "completed",
"data": [
{
"event": "run.completed",
"id": "evt_ab12cd34ef",
"url": "https://ci.example.com/hooks/roborama",
"signature": "t=1757621000,v1=6c3f...",
"status": 200,
"delivered": true
}
]
}