Package code as a container

Build policy code into an OCI image in CI, push it to a registry, and submit the run by digest — code and weights pinned as one artifact.

GitHubGitLabDocker

Code and weights usually live in different places — a Git repository and a checkpoint store. This guide packages the code into a runnable image; weights arrive by COPY at build time (see Import from private storage) or by a Hub download inside the build. Either way the output is one OCI image, and its digest pins everything the run executes.

Prerequisites

• An inference entry point — the image boots a server that speaks the cell interface: it receives the observations named by the declared observation_contract and returns actions in the declared action_space, at the declared rate. Interface details: Run your container.
• Pinned dependencies — a requirements.txt or lockfile the build installs verbatim; an unpinned base layer is a reproducibility hole.
• The declared contracts — know which action_space and observation_contract your code implements; both are validated at submission, before any motor moves (policies).
• The checkpoint's location — weights are an input to the build, not part of the repository; decide where the build reads them from.

Connect

A minimal packaging — base image, pinned dependencies, code, weights, and an entrypoint that starts the inference server:

FROM pytorch/pytorch:2.4.1-cuda12.4-cudnn9-runtime

WORKDIR /app

# Dependencies first — pinned, cached as their own layer.
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Weights can instead be fetched from the Hub at build time.
COPY src/ ./src/
COPY checkpoints/skill-v4/ ./checkpoints/skill-v4/

# Boot the server the cell runtime drives.
ENTRYPOINT ["python", "-m", "src.serve", \
    "--checkpoint", "checkpoints/skill-v4"]

The workflow builds that image, pushes it to GHCR with the built-in GITHUB_TOKEN, and submits the run by digest, tagged with the source commit. workflow_dispatch keeps the trigger manual: evaluating every checkpoint automatically should be a decision you opt into, not a default.

name: evaluate-policy

# Manual trigger. To evaluate every release checkpoint instead:
# on: {push: {tags: ["policy-v*"]}}
on:
  workflow_dispatch:

permissions:
  contents: read
  packages: write

jobs:
  evaluate:
    runs-on: ubuntu-latest
    timeout-minutes: 360
    steps:
      - uses: actions/checkout@v4

      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        id: push
        uses: docker/build-push-action@v6
        with:
          context: .
          push: true
          tags: ghcr.io/acme/skill:${{ github.sha }}

      - name: Submit the run by digest
        env:
          ROBORAMA_API_KEY: ${{ secrets.ROBORAMA_API_KEY }}
          DIGEST: ${{ steps.push.outputs.digest }}
        run: |
          run_id=$(curl -sS --fail \
            https://api.roborama.com/v1/runs \
            -H "Authorization: Bearer $ROBORAMA_API_KEY" \
            -H "Content-Type: application/json" \
            -H "Idempotency-Key: eval-$GITHUB_SHA" \
            -d '{
              "robot": "g1-edu-pro@fw2.3",
              "environment": "kitchen-std@v1.2",
              "policy": {
                "type": "container",
                "image": "ghcr.io/acme/skill@'"$DIGEST"'",
                "action_space": "joint_delta_50hz",
                "observation_contract": "droid-3cam"
              },
              "task": "load_dishwasher@v2",
              "episodes": "auto(ci=0.95, moe=0.03)",
              "max_budget_usd": 4000
            }' | jq -r '.id')
          echo "RUN_ID=$run_id" >> "$GITHUB_ENV"

      - name: Wait for the result
        env:
          ROBORAMA_API_KEY: ${{ secrets.ROBORAMA_API_KEY }}
        run: |
          while true; do
            run=$(curl -sS \
              "https://api.roborama.com/v1/runs/$RUN_ID" \
              -H "Authorization: Bearer $ROBORAMA_API_KEY")
            status=$(echo "$run" | jq -r '.status')
            case "$status" in
              completed) echo "$run" | jq '.result'; exit 0 ;;
              failed|stopped) exit 1 ;;
              *) sleep 120 ;;
            esac
          done

Two details are load-bearing:

• Immutability is the deliverable. The image tag is the commit sha and the submission uses steps.push.outputs.digest, so the run pins ghcr.io/acme/skill@sha256:4e5b1a7c9d0f2a3b5c6d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a3b4c5d6e7f8a90 — the code and the artifact, not whatever a tag means next week.
• max_budget_usd is the CI-safe default — a bad checkpoint stops at the cap with partial results instead of buying the full suite. The Idempotency-Key means a re-run of the job returns the original run instead of scheduling it twice (REST).

GitLab CI

Same shape: manual trigger, build, push to the project registry, submit by digest. Set ROBORAMA_API_KEY as a masked CI/CD variable.

evaluate:
  image: docker:27
  services:
    - docker:27-dind
  when: manual   # same stance as workflow_dispatch above
  variables:
    IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY
    - docker build -t "$IMAGE" .
    - docker push "$IMAGE"
    - |
      REF=$(docker inspect \
        --format '{{index .RepoDigests 0}}' "$IMAGE")
      curl -sS --fail https://api.roborama.com/v1/runs \
        -H "Authorization: Bearer $ROBORAMA_API_KEY" \
        -H "Content-Type: application/json" \
        -H "Idempotency-Key: eval-$CI_COMMIT_SHA" \
        -d '{
          "robot": "g1-edu-pro@fw2.3",
          "environment": "kitchen-std@v1.2",
          "policy": {
            "type": "container",
            "image": "'"$REF"'",
            "action_space": "joint_delta_50hz",
            "observation_contract": "droid-3cam"
          },
          "task": "load_dishwasher@v2",
          "episodes": "auto(ci=0.95, moe=0.03)",
          "max_budget_usd": 4000
        }'

Keys live in CI secrets — repository or environment secrets on GitHub, masked variables on GitLab — never in the image, never echoed to logs. Scope the key to runs:write: a leaked CI key then can't purge data or read streams (billing & budgets).

Validate

Nothing above needs robot time to debug. Mock mode first (ROBORAMA_MOCK=1 — the full loop against packaged fixtures, no API key, no charge); then admission, where the declared contracts are validated server-side and a rejection (contract_validation_failed) costs nothing; then cell startup, which pulls the image and boots your server. The three layers are detailed in Run your container.

Quote

A quote resolves both meters and the dollars before the workflow buys anything — the fixture job, 600 episodes of g1-edu-pro in kitchen-std, prices at $3,740. Quotes expire after seven days: if you quote in one job and submit from a later one, expect quote_expired — re-quote, don't retry (billing & budgets).

import roborama  # reads ROBORAMA_API_KEY from the environment

quote = roborama.quote(
    robot="g1-edu-pro",
    environment="kitchen-std",
    episodes=600,
)
print(quote)
# {robot_hours: 20.0, env_hours: 20.0, usd: 3740, queue_eta: "6h"}

Evaluate

The submit step is an ordinary run(). The wait step holds a runner; a suite that outlasts the six-hour hosted cap wants the webhook pattern from CI gates instead — nothing waits, and the verdict still lands on the commit.

Inspect results

The wait step exits on the verdict. The result carries n, the rate, the Wilson interval, failure clusters, both meters, and the cost; pull episodes and the citable PDF with run.episodes and run.report(format="pdf"). Retention and exports: data contract.

Compare versions

Two commits are two digests. Submit both under the same declared configuration and let compare() pair the episodes — same scenes for both arms, roughly half the episodes to separate them. The digest makes "same code?" a checkable question.

Troubleshoot

Where next

• Run your container — the image reference, digest pinning, and what the cell provides at runtime.
• Import from private storage — getting weights into the build without making them public.
• CI gates — turn this workflow into a regression gate with fail_if and webhooks.
• Billing & budgets — key scopes, quotes, and the two hard stops.