matrix()

One policy swept across embodiments × environments — rate and interval per cell, regression diffs, and a verification report.

A policy that works is a claim about one robot in one scene. A release decision needs the grid. roborama.matrix() sweeps the same task across every embodiment × environment pair you name and returns a rate with its interval per cell — plus the diffs and the document that release engineering actually asked for.

The call

import roborama  # reads ROBORAMA_API_KEY from the environment

policy = roborama.Policy.checkpoint(hf="acme/skill-v4", runtime="openpi")

job = roborama.matrix(
    policy=policy,
    robots=["g1-edu-pro@fw2.3", "g1-edu-pro@fw2.4", "g1-edu-plus@fw2.3",
            "aloha2-pro@fw1.1", "spot-arm@fw4.1", "tiago-pro@fw2.0",
            "fr3-bench@fw5.2", "stretch3@fw1.9", "nori-a3@fw1.0",
            "so-101@fw1.2"],
    environments=["cell-a", "kitchen-std@v1.2", "warehouse-std@v2.0"],
    task="pick_place_class3",
    episodes_per_cell="auto(ci=0.95, moe=0.05)",
)

job.heatmap()                    # embodiment × environment grid of rate±CI
job.regressions(vs="run_8821")   # cells that degraded vs. a prior release
job.verification_report(format="pdf")  # the release-gate deliverable

Reading the grid

pass = Wilson 95% lower bound ≥ 0.80

Each cell is an independent run: episodes_per_cell="auto(ci=0.95, moe=0.05)" sizes every cell for a ±5-point margin of error at 95% confidence. Cells with rates near 0.5 need more episodes to hit that margin than cells near the extremes, so cell sizes differ — the grid spends episodes where the uncertainty is.

Task-pinned sweeps

A matrix can pin a frozen Task Spec instead of a catalogue task: task="espresso@v1" sweeps the customer's own signed protocol across embodiments, and the per-cell results carry the spec's stage_rates — so the grid shows not just which body degrades but where in the task it does. The advanced quickstart runs exactly this sweep, with interventions="none" declared so every cell's statistics are untouched autonomy (see run() for intervention semantics).

The pass gate

Declare the gate up front. In this example a cell passes when its Wilson 95% lower bound is at or above 0.80 — the pessimistic end of the interval clears the bar, not the point estimate. Point estimates flatter small samples; lower bounds don't. A cell whose point estimate clears the bar on thin n still fails the gate until enough episodes accumulate to prove the 0.80 floor.

Catching a firmware regression

Same policy, same kitchen, one firmware step:

kitchen-std@v1.2 × g1-edu-pro@fw2.3   rate=0.892  n=240  ci95=(0.846, 0.925)
kitchen-std@v1.2 × g1-edu-pro@fw2.4   rate=0.787  n=240  ci95=(0.731, 0.835)

A −10.5 pt drop with non-overlapping intervals — not noise. The failure delta concentrates in the grasp_slip cluster, and the fw2.4 release notes mention a wrist controller change. That is a firmware regression caught by a grid, in one sweep, before it reached a customer site. job.regressions(vs="run_8821") automates exactly this comparison: every cell that degraded against a prior release, with cluster deltas attached, and a regression.detected webhook if you'd rather be told than ask.

The verification report

job.verification_report(format="pdf") is the deliverable for release sign-off: the grid with per-cell n and intervals, pass-gate outcomes, every pin (firmware, environment revisions, perturbation schedules, seeds), and the failure clusters. It is conformance evidence a reviewer can re-derive from the episode artifacts — hand it to release engineering or attach it to the contract, and anyone who doubts a cell can replay it.

Where next

• Runs & statistics — why the gate uses the lower bound.
• CI gates — run the sweep on release/* automatically.
• transfer() — when one column of the grid is a new embodiment.