# matrix() One policy swept across embodiments × environments — rate and interval per cell, regression diffs, and a verification report. A policy that works is a claim about one robot in one scene. A release decision needs the grid. `roborama.matrix()` sweeps the same task across every embodiment × environment pair you name and returns a rate with its interval per cell — plus the diffs and the document that release engineering actually asked for. ## The call *Example: roborama.matrix()* **Python** ```python import roborama # reads ROBORAMA_API_KEY from the environment policy = roborama.Policy.checkpoint(hf="acme/skill-v4", runtime="openpi") job = roborama.matrix( policy=policy, robots=["g1-edu-pro@fw2.3", "g1-edu-pro@fw2.4", "g1-edu-plus@fw2.3", "aloha2-pro@fw1.1", "spot-arm@fw4.1", "tiago-pro@fw2.0", "fr3-bench@fw5.2", "stretch3@fw1.9", "nori-a3@fw1.0", "so-101@fw1.2"], environments=["cell-a", "kitchen-std@v1.2", "warehouse-std@v2.0"], task="pick_place_class3", episodes_per_cell="auto(ci=0.95, moe=0.05)", ) job.heatmap() # embodiment × environment grid of rate±CI job.regressions(vs="run_8821") # cells that degraded vs. a prior release job.verification_report(format="pdf") # the release-gate deliverable ``` **TypeScript** ```typescript import Roborama from "@roborama/sdk"; // reads ROBORAMA_API_KEY const roborama = new Roborama(); const job = await roborama.runs.create({ kind: "matrix", policy: { type: "checkpoint", hf: "acme/skill-v4", runtime: "openpi" }, robots: [ "g1-edu-pro@fw2.3", "g1-edu-pro@fw2.4", "g1-edu-plus@fw2.3", "aloha2-pro@fw1.1", "spot-arm@fw4.1", "tiago-pro@fw2.0", "fr3-bench@fw5.2", "stretch3@fw1.9", "nori-a3@fw1.0", "so-101@fw1.2", ], environments: ["cell-a", "kitchen-std@v1.2", "warehouse-std@v2.0"], task: "pick_place_class3", episodes_per_cell: "auto(ci=0.95, moe=0.05)", }); const heatmap = await job.heatmap(); // embodiment × environment, rate±CI const regressions = await job.regressions({ vs: "run_8821" }); const report = await job.verificationReport({ format: "pdf" }); console.log(heatmap, regressions, report.url); ``` **cURL** ```bash curl https://api.roborama.com/v1/runs \ -H "Authorization: Bearer $ROBORAMA_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "kind": "matrix", "policy": { "type": "checkpoint", "hf": "acme/skill-v4", "runtime": "openpi" }, "robots": ["g1-edu-pro@fw2.3", "g1-edu-pro@fw2.4", "g1-edu-plus@fw2.3", "aloha2-pro@fw1.1", "spot-arm@fw4.1", "tiago-pro@fw2.0", "fr3-bench@fw5.2", "stretch3@fw1.9", "nori-a3@fw1.0", "so-101@fw1.2"], "environments": ["cell-a", "kitchen-std@v1.2", "warehouse-std@v2.0"], "task": "pick_place_class3", "episodes_per_cell": "auto(ci=0.95, moe=0.05)" }' ``` **Agent (tool-use payload)** ```json { "type": "tool_use", "name": "roborama_matrix", "input": { "policy": { "type": "checkpoint", "hf": "acme/skill-v4", "runtime": "openpi" }, "robots": [ "g1-edu-pro@fw2.3", "g1-edu-pro@fw2.4", "g1-edu-plus@fw2.3", "aloha2-pro@fw1.1", "spot-arm@fw4.1", "tiago-pro@fw2.0", "fr3-bench@fw5.2", "stretch3@fw1.9", "nori-a3@fw1.0", "so-101@fw1.2" ], "environments": [ "cell-a", "kitchen-std@v1.2", "warehouse-std@v2.0" ], "task": "pick_place_class3", "episodes_per_cell": "auto(ci=0.95, moe=0.05)" } } ``` ## Reading the grid Task pick_place_class3, episodes_per_cell=auto(ci=0.95, moe=0.05), gate: ci95 lower bound ≥ 0.8. | robot | cell-a | kitchen-std | warehouse-std | | --- | --- | --- | --- | | g1-edu-pro@fw2.3 | 0.937 (n=190, ci95 0.893–0.964) pass | 0.892 (n=240, ci95 0.846–0.925) pass | 0.862 (n=210, ci95 0.809–0.902) pass | | g1-edu-pro@fw2.4 | 0.921 (n=190, ci95 0.874–0.952) pass | 0.787 (n=240, ci95 0.731–0.835) fail | 0.838 (n=210, ci95 0.782–0.882) fail | | g1-edu-plus@fw2.3 | 0.953 (n=190, ci95 0.912–0.975) pass | 0.921 (n=240, ci95 0.880–0.949) pass | 0.890 (n=210, ci95 0.841–0.926) pass | | aloha2-pro@fw1.1 | 0.937 (n=190, ci95 0.893–0.964) pass | 0.871 (n=240, ci95 0.823–0.908) pass | 0.843 (n=210, ci95 0.788–0.886) fail | | spot-arm@fw4.1 | 0.884 (n=190, ci95 0.831–0.922) pass | 0.746 (n=240, ci95 0.687–0.797) fail | 0.790 (n=210, ci95 0.730–0.840) fail | | tiago-pro@fw2.0 | 0.905 (n=190, ci95 0.855–0.939) pass | 0.808 (n=240, ci95 0.753–0.853) fail | 0.724 (n=210, ci95 0.660–0.780) fail | | fr3-bench@fw5.2 | 0.947 (n=190, ci95 0.905–0.971) pass | 0.821 (n=240, ci95 0.768–0.864) fail | 0.790 (n=210, ci95 0.730–0.840) fail | | stretch3@fw1.9 | 0.879 (n=190, ci95 0.825–0.918) pass | 0.729 (n=240, ci95 0.670–0.781) fail | 0.629 (n=210, ci95 0.561–0.691) fail | | nori-a3@fw1.0 | 0.800 (n=190, ci95 0.737–0.851) fail | 0.537 (n=240, ci95 0.474–0.599) fail | 0.467 (n=210, ci95 0.400–0.534) fail | | so-101@fw1.2 | 0.742 (n=190, ci95 0.675–0.799) fail | 0.463 (n=240, ci95 0.401–0.526) fail | 0.410 (n=210, ci95 0.346–0.478) fail | Each cell is an independent run: `episodes_per_cell="auto(ci=0.95, moe=0.05)"` sizes every cell for a ±5-point margin of error at 95% confidence. Cells with rates near 0.5 need more episodes to hit that margin than cells near the extremes, so cell sizes differ — the grid spends episodes where the uncertainty is. ## Task-pinned sweeps A matrix can pin a frozen [Task Spec](/docs/concepts/tasks-and-method-transfer/) instead of a catalogue task: `task="espresso@v1"` sweeps the customer's own signed protocol across embodiments, and the per-cell results carry the spec's `stage_rates` — so the grid shows not just *which* body degrades but *where in the task* it does. The [advanced quickstart](/docs/quickstart-advanced/) runs exactly this sweep, with `interventions="none"` declared so every cell's statistics are untouched autonomy (see [run()](/docs/primitives/run/) for intervention semantics). ## The pass gate Declare the gate up front. In this example a cell passes when its Wilson 95% lower bound is at or above 0.80 — the pessimistic end of the interval clears the bar, not the point estimate. Point estimates flatter small samples; lower bounds don't. A cell whose point estimate clears the bar on thin n still fails the gate until enough episodes accumulate to prove the 0.80 floor. ## Catching a firmware regression Same policy, same kitchen, one firmware step: ```text kitchen-std@v1.2 × g1-edu-pro@fw2.3 rate=0.892 n=240 ci95=(0.846, 0.925) kitchen-std@v1.2 × g1-edu-pro@fw2.4 rate=0.787 n=240 ci95=(0.731, 0.835) ``` A −10.5 pt drop with non-overlapping intervals — not noise. The failure delta concentrates in the `grasp_slip` cluster, and the fw2.4 release notes mention a wrist controller change. That is a firmware regression caught by a grid, in one sweep, before it reached a customer site. `job.regressions(vs="run_8821")` automates exactly this comparison: every cell that degraded against a prior release, with cluster deltas attached, and a `regression.detected` webhook if you'd rather be told than ask. ## The verification report `job.verification_report(format="pdf")` is the deliverable for release sign-off: the grid with per-cell n and intervals, pass-gate outcomes, every pin (firmware, environment revisions, perturbation schedules, seeds), and the failure clusters. It is conformance evidence a reviewer can re-derive from the episode artifacts — hand it to release engineering or attach it to the contract, and anyone who doubts a cell can replay it. ## Where next - [Runs & statistics](/docs/concepts/runs-and-statistics/) — why the gate uses the lower bound. - [CI gates](/docs/guides/ci-gates/) — run the sweep on `release/*` automatically. - [transfer()](/docs/primitives/transfer/) — when one column of the grid is a new embodiment.