Before you start
RunFP is OORB's local-first software for recording robot run evidence and comparing episodes. Start with two MCAP recordings from the same task, a robot project directory, and an OORB-provided installation.
Access is arranged through OORB onboarding or a reliability audit. These instructions assume you already have RunFP; they do not require a public package registry or a public source repository.
For an approved source checkout, install its environment with uv sync and prefix the commands below with uv run. For an OORB-provided CLI installation, use runfp directly. Check the installed version and available commands with:
runfp --help
1. Initialize the robot project
Run this inside the project directory whose software state you want associated with captures:
runfp init --name robot-evaluation --root .
The workspace root determines where project episodes are stored and which Git workspace is inspected. Choose the robot project, rather than an unrelated directory, if commit provenance matters.
2. Import comparable recordings
Use your own files and give runs of the same task the same label:
runfp snapshot recordings/baseline.mcap --label pick-place --json
runfp snapshot recordings/candidate.mcap --label pick-place --json
Save the episode IDs returned by these commands. A shared label defines a run group; it does not by itself prove that task conditions are comparable. Keep the task, success definition, robot configuration, and test conditions explicit.
3. Set a baseline and compare
Replace the angle-bracket placeholders with those returned IDs:
runfp baseline set <baseline-episode-id> --name pick-place
runfp diff <baseline-episode-id> <candidate-episode-id>
runfp history pick-place
The baseline is the reference artifact for subsequent checks. The comparison shows recorded changes; history connects group attempts with available commit and outcome evidence.
4. Choose the question you want to answer
- What changed between runs? Continue with run comparison.
- Did the task succeed often enough? Use recorded outcomes and confidence intervals. Imported telemetry alone may have no task outcome.
- Should a new run block a release? Configure a regression check.
Need an installation or help choosing a valid baseline? Request an OORB audit.