Advanced quickstart
The end-to-end flow — package a checkpoint, ship a kit, freeze the claim as espresso@v1, verify the sim's doubts, and prove it across embodiments.
The quickstart proves the plumbing with a catalogue task. This is the real shape of an engagement: a policy trained elsewhere, a claim involving your own hardware, and evidence strong enough to release on. Seven calls, start to finish.
The whole flow
one claim, proven end to end
import roborama # reads ROBORAMA_API_KEY from the environment
# ── 1. Package the learnings ──────────────────────────────
policy = roborama.Policy.checkpoint(
hf="pi/espresso-v7", # weights + config + norm stats
runtime="openpi",
inference={"action_horizon": 50, "chunk_size": 50, "temp": 0.0},
action_space="joint_delta_50hz",
observation_contract="droid-3cam",
)
# ── 2. Ship the hardware the claim depends on ─────────────
kit = roborama.kits.register(
name="pi-breville", items=[{"desc": "Breville BES870", "qty": 2}])
kit.shipping_label() # -> received -> tracked -> available
# ── 3. Formalize the claim as a Task Spec ──────────────────────────────
task = roborama.tasks.create(
name="espresso", visibility="private",
initial_conditions={
"machine": {"object": "kit/pi-breville",
"pose": "P1", "tol_mm": 20},
"cup": {"object": "catalogue/cup-std-08", "randomize": "zone-A"},
},
instructions={
"sampled_per_episode": ["make an espresso", "brew me a coffee",
"fais un espresso"],
"held_out": ["prepare a single shot"],
},
stages=[{"grasp_cup": "gt.pose(cup) in gripper"},
{"cup_placed": "gt.pose(cup) within drip_zone"},
{"extraction": "scale.delta > 0 within 60s"},
{"served": "all success_predicates"}],
success_predicates=[{"cup_on_tray": "gt.pose(cup) within tray_zone"},
{"liquid_mass": "scale.delta between 25 and 40 g"},
{"no_spill": "vision.spill_area < 2 cm2"},
{"t_complete": "episode.duration < 180 s"}],
envelope={"lighting": ["3000K", "5600K"], "distractors": "set-B",
"out_of_scope": ["oat_milk"]},
baseline={"internal_trials": 30, "internal_rate": 0.87},
)
# ── 4. Agree on the method, then freeze it ──────────────────────────────
pilot = task.pilot(robot="g1-edu-pro@fw2.3", episodes=25)
print(pilot.stream_url) # watch the calibration live
task.freeze() # -> espresso@v1, immutable, co-signed
# ── 5. Screen in your simulator; verify the flagged tail physically ─────
# the sim's uncertain slice, verified physically
scenarios = roborama.scenarios.from_file("sim_flagged_p1.json")
report = roborama.verify(
policy=policy, task="espresso@v1",
scenarios=scenarios,
robots=["g1-edu-pro@fw2.3"],
audit_sample=0.10, return_ground_truth=True,
)
# ── 6. Then prove the cross-embodiment claim itself ───────
job = roborama.matrix(
policy=policy, task="espresso@v1",
robots=["g1-edu-pro@fw2.3", "g1-edu-pro@fw2.4",
"g1-edu-plus@fw2.3", "stretch3@fw1.9"],
environments=["kitchen-std@v1.2", "kitchen-replica@v2.0"],
episodes_per_cell="auto(ci=0.95, moe=0.05)",
interventions="none",
)
# ── 7. The evidence ──────────────────────────────
r = job.result()
r.heatmap() # embodiment × environment, rate ± CI per cell
r.stage_rates # where in the task each embodiment degrades
r.instruction_breakdown # held-out phrasings reported separately
r.verification_report(format="pdf") # cites espresso@v1, fw pins,
# out-of-scope verbatim
job.export(format="lerobot") # episodes back into trainingWhat each step buys you
- Package the learnings.
Policy.checkpointpins more than weights: theinferencedict (action_horizon,chunk_size,temp) is recorded with the run, because a decoding change can move a rate as much as a training change. Details in Policies. - Ship the hardware. The espresso machine arrives as a
kit: tracked (mocap markers,
mass, mesh scan) and referenceable as
kit/pi-breville. - Formalize the claim. The Task Spec turns "makes espresso" into
instrument-bound predicates, a phrasing distribution with held-out
paraphrases, staged progress scoring, and declared boundaries
(
out_of_scope: ["oat_milk"]appears verbatim on the report). - Agree, then freeze. The pilot is watched live; the freeze produces
espresso@v1— immutable, mutually signed, cited by every report. - Spend reality where the sim is unsure. verify() runs the sim-flagged tail plus a random audit sample, and returns ground truth to recalibrate the simulator.
- Prove the claim itself. A task-pinned matrix()
across four embodiments and two kitchens,
interventions="none"declared up front so the statistics are untouched by helping hands. - Collect the evidence. Rate ± CI per cell,
stage_ratesshowing where each embodiment degrades,instruction_breakdownwith held-out phrasings separated, a citable verification report, and the episodes back inlerobotformat for the next training round.
Reading the espresso evidence
The canonical single-cell result, from /fixtures/runs.json — overall 0.850 (n=412, ci95 0.812–0.881), zero interventions:
stage_rates
grasp_cup 0.966 n=412 ci95=(0.944, 0.980)
cup_placed 0.932 n=412 ci95=(0.904, 0.953)
extraction 0.893 n=412 ci95=(0.860, 0.919)
served 0.850 n=412 ci95=(0.812, 0.881)
instruction_breakdown
"make an espresso" 0.873 n=118 ci95=(0.801, 0.921)
"brew me a coffee" 0.866 n=112 ci95=(0.791, 0.917)
"fais un espresso" 0.848 n=99 ci95=(0.765, 0.906)
"prepare a single shot" 0.795 n=83 ci95=(0.696, 0.868) held-out
The stages localize the loss: grasping is nearly solved, extraction gives up seven points, serving five more. The held-out phrasing runs about eight points behind the sampled set — the gap between "understands the instructions it trained on" and "understands the task."
Where next
- Tasks & method transfer — the Task Spec in full.
- CI gates — make the matrix a release check.
- threshold() — buy the outcome instead of the runs.