// 2026.06.20

Video to Gaussian Splats — Turn a Clip into a Walkable Scene

10:00 JST

For a videographer or photographer, the most direct route from footage to an XR scene is Gaussian splatting. The photo-to-splat and phone-scan pipelines covered earlier start from stills or a dedicated scanner; this one starts from a video clip, which most creatives already shoot. A slow orbit around a subject is, frame for frame, the same multi-view input a photogrammetry rig produces — and modern tooling turns it into a splat that loads straight into the WebXR viewers the rest of the camp uses.

▸ Postshot · Nerfstudio + gsplat · Brush (cross-platform) · SuperSplat editor/viewer

Two regimes: static scene vs. moving subject

The fork that catches everyone: is the subject still, or moving?

  • Static scene (3DGS) — the camera moves, the world holds still. A walk-around of a statue, a room, a product. This is the solved, weekend-friendly case: ordinary 3D Gaussian splatting, the same as a photo set.
  • Moving subject (4DGS) — a person talking, water, anything that changes between frames. This needs 4D Gaussian splatting — splats with a time dimension — and it is still research-grade: 4DGaussians, Deformable-3DGS, and Luma-style volumetric capture. Heavier to train, fussier to view. Treat it as a stretch goal.

For a 2.5-day build, shoot for the static case: pick a subject that does not move and orbit it.

Capture for splatting

The training cares about parallax and coverage, not cinematography:

  • Move, don’t zoom. Physically circle the subject; keep focal length fixed.
  • Cover every angle, including a high and low pass. Gaps become holes in the splat.
  • Even, diffuse light; avoid changing exposure mid-shot — bakes inconsistencies in.
  • Slow and steady beats fast — motion blur is the enemy. 30–60 seconds at a walking pace is plenty.

Extract frames

Splat trainers take images. ffmpeg samples the clip — three to six frames per second is a good starting density:

ffmpeg -i orbit.mov -vf "fps=4,scale=1600:-1" frames/%04d.jpg

Too many near-identical frames slows training without adding detail; too few starves the structure-from-motion step. 150–300 sharp, well-spread frames is a healthy target.

Train the splat

Three accessible options, in rough order of least setup:

  • Postshot (Windows, free) — drag in the video or the frame folder and it runs structure-from-motion plus 3DGS training end to end, exporting .ply. The lowest-friction path.
  • Nerfstudio with the gsplat backend — ns-process-data video --data orbit.mov runs COLMAP for camera poses, then ns-train splatfacto trains. Cross-platform, scriptable, CUDA-friendly.
  • Brush — a wgpu trainer that runs on macOS, Windows, Linux, and even in a browser; no CUDA required, which suits a mixed-laptop team.

All three export a standard Gaussian-splat .ply; convert to the compact .splat/.spz form with SuperSplat for faster loading.

View it in WebXR

This is where the splat meets a headset. The same single-file WebXR engines the camp already uses load splats directly:

  • Babylon.js has first-party support — await SceneLoader.ImportMeshAsync(null, "", "scene.splat", scene) drops the splat into the scene used by the Babylon WebXR starter.
  • Three.js loads them via GaussianSplats3D (mkkellogg) in the three.js starter.
  • PlayCanvas / SuperSplat publish a viewer to a URL with no code at all — open it in the Quest, PICO, or Vision Pro browser.

A static splat is a fixed scene, not interactive geometry — but standing inside your own capture on a Quest 3 is a strong demo on its own, and pairs naturally with the in-headset art tools for annotation.

Caveats

  • Static subjects only, unless you commit to the 4DGS rabbit hole — moving people ghost badly in plain 3DGS.
  • VRAM is the limiter for training; a cloud GPU or Brush’s lighter footprint is the fallback on modest laptops.
  • Splats are heavy on the wire — decimate in SuperSplat before shipping to a standalone headset; aim for well under a million splats for comfortable WebXR frame rates.
  • Reflective and transparent surfaces fool the solver — glass, chrome, and water produce floaters.

Questions? Reach the team via the Contact page.

// BACK TO NEWS