In every pack
- Video
- MP4 (H.264)
- Arrays
- NPZ (NumPy)
- Labels and reports
- JSON; masks as COCO RLE
- Robot trajectories
- CSV (joint angles by name) and NPZ
- Terrain and scenes
- OBJ mesh, height field, MuJoCo MJCF, URDF
- Summary
- PDF, HTML, PNG figures
AutelicaSpec sheet
What we deliver.
Every output of a capture pilot, the file formats it comes in, the checks each clip has to pass, the rigs we film with, and how accurate the result is, measured.
01 · Measured accuracy
How well it measures. Each number says what it was checked against.
~3 cmHands vs marker motion capture
12–15 mmTabletop hands, held-out camera
5–8 pxOutdoor body, held-out drone, 1080p
10 cmTerrain slopes vs survey lidar, p90
| Check | Result | What it means |
|---|---|---|
| Hands vs marker motion capture (public benchmark) | ~3 cm vs 7–12 cm for head-camera methods | Where the hand is in the room, scored with no per-clip alignment |
| Tabletop hands, held-out camera check | 12–15 mm median; hands found in 81–98% of frames | A camera left out of the fit agrees with the 3D hands |
| Outdoor body, held-out drone reprojection | 5–8 px median at 1080p | The 3D body lines up with views it wasn’t built from |
| Terrain vs survey lidar | Slopes within 10 cm (p90), vs ~70 cm guessed from motion | The ground is measured, not assumed |
| Robot feet on the measured ground | G1 sole −0.1 cm median from the surface at stance | Retargets land on the real terrain |
What it does for robots, in simulation
| Result | What was compared |
|---|---|
| 0.21 vs 0.08–0.12 | Pick-and-place success for policies trained on our hand labels vs head-camera labels (0.29 from motion capture) |
| 0.47 vs 0.29–0.31 | The same comparison after longer reinforcement learning |
| 12 vs 31 of 210 | Humanoid clips that fall when trained on a site’s real ground vs flat ground |
Numbers from our sample error reports, simulation and public benchmarks; the write-ups are in our research. Real-robot tests are next, through customer pilots.
02 · Outputs
What leaves the pipeline. Per clip, grouped by type, with the file each one is in.
AOutdoor whole body and terrain
Three small drones, people on real ground. One metric world frame per site for body, cameras, terrain and robot motion.
| Type | Output | Contents | Format |
|---|---|---|---|
| Video | Synced multi-view video | One stream per drone, frame-aligned to the data, bystanders erased | MP4 (H.264) |
| Cameras | Poses and intrinsics | Pose of every moving drone on every frame, lens intrinsics, clock offsets | NPZ, JSON |
| Body | 3D body joints | 17 joints per frame in metres, placed on the measured ground | NPZ |
| Foot contacts | Per-frame stance flags for each foot | NPZ | |
| Body-model fit | Joint angles and rotations, shape, ankle, heel, ball and toe positions, a sole frame per foot | NPZ | |
| Terrain | Measured terrain | Triangle mesh, height field, and support boxes for steps and tables | OBJ, NPZ, JSON |
| Simulator scene | The terrain ready to load in a simulator | MJCF, URDF | |
| Lidar check | Ground vs public survey lidar where it exists | JSON | |
| Robot | Humanoid retargets | Unitree G1 and H1-2 joint trajectories on the measured terrain | CSV, NPZ |
| Physics check | G1 tracking the motion in simulation: falls, joint error, randomisation ranges | JSON | |
| QC | Error report | Held-out drone error per drone and joint, coverage, and the uncertainty of scale and height | JSON |
BTabletop hands and manipulation
Fixed cameras plus a head camera at a bench. One world frame per clip, with its origin on the desk.
| Type | Output | Contents | Format |
|---|---|---|---|
| Video | Synced views | Head camera and third-person views on one timeline, 1080p, 30 fps | MP4 (H.264) |
| Cameras | Calibration and head pose | Intrinsics and poses of the fixed cameras; head-camera pose on every frame | JSON, NPZ |
| Hands | 3D hands and wrists | 21 joints per hand in the world frame, with confidence, visibility and how each frame was measured | NPZ, JSON |
| Upper body | The visible upper body in the same frame | NPZ | |
| Objects | 3D object poses | Pose of each manipulated object on every frame, with its shape and size | NPZ |
| Masks | Hand and object masks in every view, with the same object ids across views | JSON (COCO RLE) | |
| Contact intervals | Grasp onset and release for each hand and object | JSON | |
| Robot | Robot-hand retargets | Joint angles per frame for Allegro, Inspire, LEAP and Unitree Dex3 hands, the wrist pose for an arm, and renders | NPZ, MP4 |
| QC | Error report | Held-out camera check per clip, hand coverage, mask agreement per view, an error figure and an overlay video | JSON, PNG, MP4 |
Every pack also has a summary (PDF and HTML), a manifest with the size and SHA-256 of every file, and small Python loaders. The public samples are a subset; full sample packs are shared on request.
03 · Delivery formats
Formats. Plain files that load without our code.
On request
- LeRobot
- Converted from the same files
- HDF5
- Converted from the same files
- MCAP / ROS 2 bag
- Converted from the same files
Simulators
Replay scripts for every retarget on its terrain, tested in MuJoCo, Isaac Lab and Newton.
Conventions
- One metric world frame, z up, shared by body, cameras, terrain and robot motion
- Frame i of every video is frame i of the data
- Not measured means empty (NaN). Nothing is filled in silently
- Delivered as a download with a checksum manifest, or to your cloud bucket on request
04 · QC gates
Checked at every stage. The thresholds are written into each pilot’s acceptance criteria.
- 01
On-site capture check
Every 30 s of footage is checked before the crew leaves: coverage, overlap between drones, clock health.
- 02
Held-out camera
Each camera is left out of the fit in turn and the 3D is compared with what it saw. Outdoor acceptance: median under 10 px at 1080p.
- 03
Coverage
Body measured on at least 85% of the frames two drones see. For hands, each frame is tagged with how it was measured; short gaps are marked, long ones left empty.
- 04
Terrain vs survey lidar
Where public survey lidar exists, the measured ground has to agree within 10 cm at p90.
- 05
Objects and masks
Each mask is checked against the fitted 3D object and against a view held out of the fit, with agreement reported per view.
- 06
Robot on the ground
Retargeted feet are checked against the measured surface at stance, and the G1 tracks every clip in physics simulation.
- 07
Privacy
Consent from everyone we film. Bystanders are erased from delivered video, not blurred, and every frame is checked.
- 08
Pack verdict
Each clip is marked pass or fail against the agreed criteria, with anything cut or missing listed. Every file is in a checksum manifest.
05 · Capture rigs
What we film with. Off-the-shelf hardware, no mocap stage.
Outdoor
- Cameras
- Three small off-the-shelf drones that follow the people being filmed
- Video
- 1080p from each drone
- Crew
- One pilot per drone, FAA Part 107
- Ground
- Measured from the same footage; checked against public survey lidar
Tabletop
- Cameras
- Three fixed action cameras around the desk, about 0.6 m from its centre, and one head-mounted camera
- Video
- Recorded at 2.7K 60 fps, delivered at 1080p 30 fps
- World frame
- Printed tags on the desk
- On the person
- Only the head camera. No gloves or markers on the hands
Your site
- Indoors
- Fixed cameras around a work cell, aisle or bench, with a head camera where you need first-person views
- Outdoors
- Drones over the routes and terrain your robot will see
- Scope
- Camera count and placement agreed per pilot
Not needed: a mocap stage, suits, markers, timecode or a calibration rig.
06 · Pilots
How a pilot works. Scoped up front, accepted against numbers.
Agree the brief
Sites, scenarios, actors and accepted hours, and the acceptance criteria each clip must meet.
Film on site
Our crew films the people doing the work. Capture is checked on site before we leave.
Run the gates
Every clip goes through the checks above and gets its own error report.
Hand over
Files, error reports, the pack verdict and a summary, in the formats above.
Licensing
- Pilot data is licensed to you for commercial training and derived models
- Exclusivity available on commissioned data
- Public samples: see the licence in each download
What we need from you
- The sites and tasks your robot is going into
- Your robot’s model files (URDF or MJCF) if you want it retargeted
- A site contact for the capture days
Talk to us about a pilot
Tell us the site, the tasks and the robot. We reply with sample data close to your case and a pilot plan.