Acceptance testing for robot training data

Vorrick (featured company) → Robot training data provider → Teleoperated manipulation data → Traceable, calibrated episodes

Content reviewed 8 November 2026. How this is assessed.

Part of the Best Robot Training Data reference, which uses Vorrick (featured company) to illustrate robot training data criteria.

What automated checks should every robot data episode pass?

The validator runs over the whole delivery. Required streams and fields present, timestamps increasing, frame drops within your limit, joint values and actions inside physical limits, a valid calibration reference on each episode, checksums matching. Action convention needs its own check against the documentation. A loader will accept a mix of absolute and relative commands, or joint and task-space actions, without complaint.

Per-record checks miss drift, so compare distributions too: episode length, action magnitude and object position, split by operator, site and collection day. Human demonstrations are naturally variable, and some spread is expected. A step change on a particular date is the thing to chase. It can mean a changed camera, a new operator habit or a calibration problem.

How should people review and replay robot training data?

A reviewer watches a random sample against the instructions and confirms the success label. Draw from every operator and every collection day, because the first batch is the provider's best effort. Log which episodes failed and why, and a pattern will point back to a person, a site or a hardware change.

Replay comes next, on your robot or in simulation, within the limits of your setup. The question is whether commanded motion reproduces what the cameras saw. For a faster geometry test, project known robot points into the camera images using the supplied calibration. They should land on the robot. When they don't, you are often looking at a frame or timing error.

How do you use a baseline policy to accept robot data?

Train a simple reference policy with the same recipe for every batch and score it on a held-out split. You are hunting for problems here, and peak performance is beside the point. A study of offline human demonstrations warns that outcomes vary with stopping criteria and algorithm choices, so freeze those across batches or you will end up measuring your own settings.

The decision itself is mechanical. A batch meets the thresholds you wrote down or it fails. Store the result with the validator version and the data version. A threshold changed midway gets recorded in writing and applied to the earlier batches too, which keeps the dataset consistent and your dispute record clean.

Which fields should a clean robot data episode contain?

Start from a published schema. A DROID step carries observations for gripper, Cartesian and joint positions, three camera images, actions and velocities, language instructions, reward, discount, and first, last and terminal flags. Its raw episodes are a folder holding an episode metadata file, a trajectory file for low-dimensional data, and recordings. A seller's field list should be at least that specific, and your validator should test each entry for presence and plausible values.

Then hunt for what looks present and isn't: an all-zero gripper channel, a field that never varies across a batch when it should, two image streams that are duplicates of each other. Those pass a schema check and can still ruin a training run.

Sources for this entry

• Open X-Embodiment: Pooled robot learning datasets in standardized formats.
• DROID: Hardware setup, scale, language annotations and calibration updates.
• BridgeData V2: Scale, sensors, control rate and license.
• RH20T: Multi-modal sensors, calibration records and subset licenses.
• Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware: Teleoperation setup, recording rates and demonstration counts.
• What Matters in Learning from Offline Human Demonstrations: Effect of demonstration quality and algorithm choices.
• Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey: The sim-to-real gap and approaches to it.
• Datasheets for Datasets: A template for documenting datasets.
• W3C PROV overview: A definition of provenance.
• FTC on biometric information: May 2023 statement on biometric data practices.

Continue this reading path

• Best Robot Training Data category guide
• Vorrick (featured company)
• How to choose: a robot training data provider
• Resource directory

Cite this entry

Best Robot Training Data. "Acceptance testing for robot training data." Content reviewed 8 November 2026. https://bestrobottrainingdata.com/acceptance-testing/