Without timecode, multi-camera clips are synced in two steps. First the audio tracks are cross-correlated: the same sounds appear in every clip, and the shift that best lines them up gives the offset to within a few milliseconds. Then the subject’s motion refines it: at the right offset, joints seen by different cameras agree in 3D, and at the wrong one they don’t. On a public three-camera dataset, motion sync landed a median 1.2 ms from the cameras’ wired sync.
Film crews sync cameras with timecode, a shared clock fed to every camera, or with a clapperboard they can find in every shot. Phones and action cameras have neither. Their clips start whenever each person pressed record, and the time stamped in the file is only good to about a second.
01Why a few milliseconds matter
A sprinter at 9 m/s moves 9 cm in 10 ms, and a foot at the top of its swing moves twice that. If Cam 2’s clock is 10 ms ahead, its view of the foot is from a slightly different instant than Cam 1’s, and the triangulated foot lands centimetres from where it was. Across a whole clip that shows up as jitter in joint angles and phantom changes in speed.
Fig. 1 shows what a bad sync looks like. Press File timestamps: Cam 2 is 280 ms off, a normal amount for phones started by hand, and the orange body trails the white one by most of a stride. Press Sync from the motion and watch them converge.
02Step 1: sound
If the cameras recorded audio, the same sounds are in every clip: footsteps, a ball strike, a whistle, a shout. Slide one audio track against the other and at the right shift the waveforms line up. The standard way to find that shift is cross-correlation, and the version that copes best with echoes and different microphones is called GCC-PHAT. It flattens the loudness of every frequency so that only the timing counts, which turns a broad hump into a sharp spike.
Two audio waveforms from two phones and their cross-correlation. With footsteps alone, several shifts match almost equally well; adding a clap gives one sharp peak at the true offset.
Two things can go wrong, and the figure has both.
Repetition. Footsteps come every 360 ms or so, and each one sounds like the last. Shift one track by a whole stride and it still matches well. Switch to Footsteps only and the best peak is barely taller than its neighbours. A single sharp, one-off sound fixes it, which is why film crews clap. A clap, a whistle or a ball strike at the start of the take is worth the second it costs.
Sound is slow. It travels about 343 m/s, so it takes 2.9 ms to cover a metre. A phone 10 m further from the clap hears it 29 ms later and the audio sync is off by exactly that. You can’t see it in the audio, because the audio is doing its job. It takes a different kind of evidence to fix.
03Step 2: the athlete’s motion
That evidence is the athlete. Every camera sees the same body, and a body can only be in one place at once. Take a guess at Cam 2’s offset, triangulate the joints from Cams 1 and 3, project them into Cam 2 and measure how far they land from what Cam 2 saw. At the right offset the miss is a few pixels. At the wrong one it grows quickly, because the body has moved on.
The curve in Fig. 1 is that miss at every shift from −400 to +400 ms. It has one clear valley, and the search is two passes: whole frames first to find the valley, then fractions of a frame to find its floor. Light and sound play no part, so the sound-travel error from step 1 disappears.
04How well it works
| Test | Result |
|---|---|
| Three fixed cameras, public dataset, 35 clips of 4 s or more, compared with the dataset’s wired sync | 1.2 ms median error, 2.4 ms at the 90th percentile |
| Two iPhones whose file clocks were up to 2.1 s apart | Recovered in 16 of 16 clips |
| Fig. 1, one clip, joints at 25 fps, searched in 10 ms steps | Lowest miss within 10 ms of wired sync |
One frame at 50 fps lasts 20 ms, so motion sync lands well inside a single frame. That is what lets triangulation treat the three cameras as if they had been wired together.
05Doing it yourself
- For editing, most editors (Premiere Pro, DaVinci Resolve, Final Cut) can sync clips by their audio waveform. That gets you within a frame, which is fine for cutting between angles.
- For measurement, you need the motion step as well. Our pipeline runs both steps automatically on clips as they come off the cameras.
- Either way, set every camera to the same frame rate, keep the audio on, and clap once in view of all of them at the start.
06When it struggles
- Very short clips, under about three seconds, give the motion step little to work with.
- Two cameras only: with no third view to cross-check, errors are larger.
- Repetitive motion like steady jogging or cycling can look alike at several offsets. A clap or any sharp sound at the start fixes it.
- Variable frame rate. Some phones drop frames in low light, so the clock drifts within a clip. Good light and a fixed 60 fps avoid it.
Questions
Can I sync phone videos without a clapperboard?
Yes. Cross-correlating the audio tracks aligns them to within a few milliseconds, and the subject’s movement refines the result when three cameras see the same person. A clap still helps when the only sounds are repetitive, like footsteps.
How accurate is motion-based camera sync?
On a public three-camera dataset, 1.2 ms median error against wired hardware sync across 35 clips of four seconds or more. That is well inside one frame at 50 fps.
What is GCC-PHAT?
Generalized cross-correlation with phase transform. It lines up two audio signals by weighting every frequency equally, so only timing counts. The result is a sharp peak at the true delay, even with echoes or different microphones.
Why is audio sync not enough for 3D analysis?
Sound takes about 2.9 ms to travel a metre, so cameras at different distances from the sound hear it at different times. Motion sync uses the athlete’s body instead of sound and removes that error.