Real robots as an API.
Statistically defensible evaluation on physical hardware. Simulation screens; physical testing verifies.
import roborama # auth: ROBORAMA_API_KEY
run = roborama.runs.get("run_demo")
for event in run.watch(): # SSE ticker
print(event)
print(run.result())# requests & SSE eventsland here in step with the footage
For policy & model teams
Package a checkpoint, container, or endpoint; pin a robot and a versioned scene; get back a success rate with its n and Wilson 95% interval. episodes="auto(ci=0.95, moe=0.03)" sizes the run for the precision you need. Start with the quickstart.
For release & fleet verification
Sweep one policy across every embodiment and environment you ship to — each cell a rate with its interval, gated against a declared threshold, compared against prior runs. Wire it into CI gates and webhooks for release sign-off.
For simulation vendors
Send the scenarios your simulator flags; verify() runs them on physical hardware and returns the disagreements with full ground truth — mocap poses, commanded-vs-executed. Calibration exports ship under sim-calibration-v1. See sim triage.
The product
The product is a function call.
pip install roborama-sdk — then hand us a policy, pin a robot and an environment, and get back a result you can put in front of a customer, a safety board, or a CI gate.
import roborama # auth: reads ROBORAMA_API_KEY from the environment
run = roborama.run(
robot="g1-edu-pro@fw2.3",
environment="kitchen-std@v1.2",
policy=roborama.Policy.checkpoint(hf="acme/skill-v4", runtime="openpi"),
task="load_dishwasher@v2",
episodes="auto(ci=0.95, moe=0.03)",
max_budget_usd=4000,
)
print(run.result())The discipline
Every result ships with error bars. A success rate arrives with its n and a Wilson interval — a bare percentage is treated as a bug, in our API and on this site.
The testing surface compounds: every embodiment multiplies against every policy and every scheduled re-run. The matrix is a primitive, not a spreadsheet.
In Physical Intelligence's published online-RL result, 7 of 8 minutes of wall-clock went to resets and overhead, not robot data. Scripted resets are the economic core of the facility.
Sources and arithmetic on the research page.
The pipeline
Quote to verified, one thread.
Pin the claim, price it before a single motor moves, watch it meter live, and end at a number you can defend — every value below is the run_8842 fixture, end to end.
600 episodes est · 20 robot-hours × 20 env-hours · queue 6h
20.4 robot-hours · 20.4 env-hours · metered $3,812 against the $3,740 quote
ci95=(0.846, 0.898) · wilson · gate ci95 lower ≥ 0.80 → pass
The matrix
Embodiments × environments, resolving all day.
One policy, ten bodies, three scenes — every cell a rate with its interval. The full grid lives on the product page.
pass = Wilson 95% lower bound ≥ 0.80
10 embodiments × 3 environments · pick_place_class3 · episodes_per_cell=auto(ci=0.95, moe=0.05)
Transfer
One brain, many bodies — measured.
The leading foundation models train overwhelmingly on arms and mobile bases, so every cross-embodiment claim ships with an unverified transfer to the bodies that matter most. transfer() measures the gap before a customer does — the same weights on two embodiments, Wilson intervals on both ends, and the new failure clusters named.
sourcealoha-bimanual@fw3.1
0.942
targetg1-edu-pro@fw2.3
0.715
grasp_slipwrist_singularityopened at the target · zero at source
Δ −22.7 pts — intervals on both ends, clusters named in between
The number nobody in the market can currently produce is the one every deployment decision needs — the worked example, with sources, on the research page.
Observability
Watch it run.
A WebRTC stream for every cell, an episode ticker, stage rates with Wilson intervals — and when the run completes, the artifacts are yours: mcap, video, ground truth, a citable report.
cell-g1-04 — episode 341/412 · scripted reset between episodes
result.stage_rates — Wilson 95%
failure_clusterscup_misplace 24spill 19timeout 12
mcapvideoground_truthreport_pdfwss://streams.roborama.com/cells/cell-g1-04/webrtcThe evidence
Reliability compounds against you.
Useful physical work is a chain of steps, and per-step reliability compounds across it — the entire economic value of deployment lives in the last decimals. A demo reel is one survivor-selected take; a 20-trial eval cannot distinguish 85% from 99%.
0.9930 = 0.74
99% per step — deployable
0.9530 = 0.21
95% per step — a demo reel
rule of three: a failure rate below 1-in-1,000 at 95% confidence takes ~3,000 clean episodes; 1-in-10,000 takes ~30,000 — the price of evidence is set by mathematics, not by us
Agent-native
Built for your agents too.
Every page on this site has a markdown twin — append .md. The docs ship whole as /llms.txt and /llms-full.txt, the API as an OpenAPI 3.1 spec with a runnable example on every operation, and ⌘K jumps anywhere. An agent can go from zero to a priced, running evaluation without rendering a pixel.
$ curl -s https://roborama.com/docs/quickstart.md # Quickstart Key → run → result in under five minutes. You'll create an API key, launch a physical run on a humanoid in a kitchen replica, and read back a result with a confidence interval. Nothing here requires hardware knowledge — the defaults are the point. $