Start with the failed run and a comparable healthy run
To investigate a robot field failure, preserve the original recording, identify the task and conditions, and compare it with a healthy run of the same task. OORB RunFP can organize recorded episodes and available provenance; differences guide investigation, while controlled follow-up tests establish a cause.
You need the original recordings, an initialized robot workspace, and an OORB-provided installation. Keep relevant incident context with your team's investigation record.
1. Preserve evidence before changing the robot
| Evidence | Record with the investigation | Why it matters |
|---|---|---|
| Failure boundary | Task start, failure time, stop condition, and intervention | A truncated recording can omit the event you need to investigate |
| Physical conditions | Lighting, payload, object pose, surface, and calibration state when relevant | A healthy lab run may not be a valid reference for a customer-site failure |
| Deployed identity | Available model checkpoint, software build, configuration, and local edits | A commit or declared model version alone may not identify what actually ran |
| Outcome | The task's success definition and the observed result | A process exit code may not describe physical task completion |
These are team-maintained investigation requirements. RunFP captures available evidence; it does not guarantee that every physical condition or original deployment detail was observed.
2. Select a healthy reference without hiding the difference
Choose the same robot behavior and a documented task objective. State unavoidable differences, such as customer-site lighting or a changed payload, before interpreting telemetry.
Initialize the workspace in the intended robot project, then import both recordings:
runfp snapshot recordings/healthy.mcap --label pick-place --json
runfp snapshot recordings/field-failure.mcap --label pick-place --json
Save the returned episode IDs. The import captures the evidence available now. It does not retroactively discover the software, hardware, or policy that originally produced an old recording.
3. Separate comparability problems from behavior findings
Use a protocol prepared for the task:
runfp diff <healthy-episode-id> <failed-episode-id> --protocol protocols/pick-place.yaml --json
First review required topics, schemas, and duration requirements. Then inspect relevant behavior and provenance changes. If the important sensor is absent, record that evidence gap instead of interpreting the remaining data as a clean comparison.
The comparison guide explains the report's scope. You can also inspect the original recording in your existing viewer; compare the available workflows.
4. Turn findings into testable hypotheses
| Observation | Next investigation question |
|---|---|
| A model record differs | Did the robot load that checkpoint, and does the failure persist under comparable conditions? |
| Calibration or environment evidence differs | Can you reproduce the behavior while changing that condition and holding others fixed? |
| Required telemetry is missing | Which signal must the next trial record to distinguish the competing explanations? |
| Several stack changes coincide | Which changes can your team isolate in follow-up trials? |
Keep the finding, the hypothesis, and the result of the follow-up trial separate. A provenance difference identifies a candidate explanation; it does not prove root cause.
5. Share the same artifact
runfp episodes export <failed-episode-id>
On a teammate's initialized installation:
runfp episodes import <failed-episode-id>.oorb-episode.tar.gz
Keep the task definition and incident context with the team's case record. Importing the same episode again does not create another independent physical trial.
Can this evidence become a regression test?
Yes, after you validate the task conditions, required signals, and acceptance policy. Add a representative recording to the test suite and use the release-check workflow. Validate task outcomes separately when the release question is physical success.
Bring the failed run and a healthy reference to an OORB audit.