Robot-learning teams get human motion from six places: teleoperation, handheld grippers, first-person headcam video, mocap studios or suits, single-camera 3D estimates from ordinary video, and multi-view markerless capture. Headcam video has the volume, mocap has the accuracy, and only multi-view capture measures the whole body in the place where the work happens with an error bar on every clip.
01Why human motion
Robot learning in 2026 scales with hours of demonstrations of how to act. The State of AI Report 2026 lists three main sources for that data: teleoperation, handheld grippers and first-person human video. Humanoid teams increasingly lean on people rather than robots: in VioLA (arXiv 2610.12435), 93.2% of the 140.6 million training frames are first-person human recordings, and the policy follows locomotion instructions on a real Unitree G1 without task-specific fine-tuning.
02The sources compared
| Source | What it records | Where it works | Body accuracy |
|---|---|---|---|
| Teleoperation | Robot joint commands while a person drives the robot | Wherever the robot is | Exact for the robot; no human body |
| Handheld grippers (UMI-style) | Gripper pose and camera view | Anywhere a person can carry it | Hands and tool only |
| First-person headcam video | What the wearer sees; hands; body estimated from video | Homes, workplaces | Estimated, rarely measured |
| Mocap studio or suit | Markers or inertial sensors on the body | A studio, or a person wearing a suit | Millimetres in a studio |
| Single-camera 3D estimates | Body pose inferred from one video | Any existing video | Depth and scale ambiguous |
| Multi-view markerless capture | Whole body and hands triangulated from 2–5 cameras | The real site, indoors or outdoors | Measured per clip against a held-out camera |
03What is missing
Headcams see the hands well and the body poorly, and the 3D body is fitted to the video afterwards. The VioLA authors note that extending their approach to more video "depends on obtaining reliable estimates of that motion". Mocap measures the body precisely but only in a studio or on someone wearing a suit, so the motion of real work in real buildings, and of fast whole-body movement outdoors, is rarely recorded at all.
04What multi-view capture adds
With two to five synchronised cameras, every joint is seen from several sides, so depth comes from geometry instead of a guess. The error can be measured on every clip by leaving one camera out and checking the 3D against what it saw. In our tests a humanoid trained on a three-camera capture reached 12.9 cm keypoint error against 12.3 cm for one trained on a 27-camera mocap dome (evaluation-only data, simulation), while a single camera added 3–6 cm (details).
05Licensing
Several popular body models, datasets and pose estimators are licensed for research only. Data used to train a commercial robot needs a licence chain that allows it: consent from every person filmed, and models whose weights permit commercial use.
Questions
What is the best source of human motion data for humanoid robots?
It depends on the skill. First-person video gives volume for hand tasks; mocap gives studio-grade accuracy; multi-view markerless capture gives measured whole-body motion in the real environment, with an error bar per clip.
Can a humanoid learn from human video?
Yes. VioLA (2026) trains a Unitree G1 policy where 93.2% of training frames are human recordings, and it follows locomotion instructions on the real robot zero-shot.
Is first-person video enough for whole-body skills?
Usually not on its own: a headcam sees little of the wearer’s body, so whole-body motion is estimated rather than measured.