Handheld phone video is mostly the camera rotating, so the movement between neighbouring frames, which a global structure-from-motion solver needs to place each camera, is tiny and noisy. GLOMAP’s global positioning starts from random positions and, with its default robust-loss scale of 0.1, treats most of those weak constraints as outliers, so the random start decides the path: about 3 runs in 10 placed a stretch of frames at the wrong scale. Raising the loss scale to 0.5 gave 10 of 10 good solves on the same clip (5 of 5 on a drone clip) at the same speed, and our replays went from 6 to 10 minutes to about 3.
01What we run
Autelica turns phone videos of an athletic moment into measured 3D motion and a textured replay. With one moving camera, the replay needs the camera’s path, which we solve from the background with structure from motion: SIFT features on 143 keyframes of a 14-second clip, matched within a window of up to 32 keyframes, then GLOMAP (global mapping in COLMAP 4.2) to place every camera at once. A check then keeps the longest stretch of the path that is smooth in position and rotation; we report the share of keyframes on it as “consistent”.
02The symptom
On the founder’s own 14-second handheld clip, identical runs gave 0.94, 0.32, 0.34 and 0.65 of keyframes on a consistent path. Below 0.6 the pipeline fell back to a denser, slower solve, so the 3D replay took 6 to 10 minutes instead of 3. In production that day, 6 of 9 first attempts failed.
03What it wasn’t
| Suspect | Test | Result |
|---|---|---|
| The mapper | COLMAP incremental instead of global, 3 runs | Still 1 in 3 broken, 2 to 3× slower |
| The lens guess | Focal length × 0.8, 1.0, 1.25, 3 runs each | All three fail about 1 in 3 |
| Too few features | Inliers between neighbouring keyframes | 1,000 to 2,000 per pair across the clip |
| The seed | Fixed random seed | Seed 0 gave 0.97 and then 0.34: the variation is from multithreading |
Every raw model registered all 142 keyframes. The difference was the shape of the path: in the broken runs, a stretch of frames (around frames 75 to 120) sat at about 5× the step size of its neighbours, so the consistency check kept only the largest smooth piece.
04The cause
COLMAP classifies each verified image pair. Most neighbouring pairs in this clip came out as planar or panoramic: the camera mostly rotated. Rotation alone says nothing about where the camera moved, so the translation directions between neighbours, which global positioning relies on, are small and noisy. GLOMAP initialises camera positions and points at random and minimises a robust loss. At the default scale of 0.1, much of that weak-but-real evidence sits in the loss’s flat outlier regime, so it barely pulls on the solution, and the random start decides where a weakly-constrained stretch ends up.
05The fix
| Global positioning loss scale | Handheld clip | Drone clip |
|---|---|---|
| 0.1 (default) | 3 of 10 broken (≈0.33) | 0.61 to 0.93 |
| 0.5 | 0 of 10 broken (0.67 to 0.99) | 0.90 to 0.99 |
| 1.0 | 1 of 5 broken | not tested |
Mapping time didn’t change (about 25 s on a workstation). Raising the minimum triangulation angle to 4°, which drops low-parallax points, made it worse: 3 of 3 broken. We also re-run only the mapping (features cached) if a solve still lands below the bar, and moved feature extraction and matching to a CUDA build of pycolmap (30 s down to 5 s in production). In production the next three runs on the same clip solved first time, and the 3D replay was ready 2.8 to 3.4 minutes after upload.
06Takeaways
- Run it more than once. A multithreaded solver that is fine most of the time hides a failure rate you only see by repeating identical runs.
- Check the pair types. If most of your pairs are planar or panoramic, translation is weakly observed, and global positioning needs help.
- Robust-loss scale is a modelling choice. The default suits typical photo collections; rotation-dominated video wanted a wider inlier band.
Questions
Why does structure from motion fail on handheld video?
Handheld video is dominated by rotation, so the translation between neighbouring frames is small and poorly observed; global solvers that place all cameras at once from those translations can converge to a wrong path.
What fixed GLOMAP on rotation-dominated video for us?
Raising the global positioning robust-loss scale from 0.1 to 0.5: 10 of 10 good solves on a handheld clip and 5 of 5 on a drone clip, at the same speed.
Does a fixed random seed make GLOMAP deterministic?
Not in our runs: with multithreading, seed 0 produced both a good and a broken path on the same input.