// 2026.06.14

Make Art Inside the Headset — and Get It Into Your Build

10:00 JST

The capture pipelines covered here — photo-to-splat, spatial video, 360° scenes — all start from the real world. Making art inside the headset is the inverse: the artist authors 3D geometry from nothing, hands tracked in space, and the result is a real mesh to bring into a build. For a hackathon the deciding question is not which tool feels best to paint with — it is which tools export engine-ready geometry, for free, and how to keep that geometry from breaking on the way into a scene. That second half is where most teams lose an afternoon.

▸ Open Brush · Open Brush Toolkit (Unity) · three-icosa (WebXR)

The free, open centerpiece: Open Brush

Open Brush is the community successor to Google’s Tilt Brush, maintained by the Icosa Foundation, free and open-source (Apache-2.0), and actively released through 2026. It runs natively on Quest (no PC) and on PCVR/SteamVR, and — the reason it anchors this whole piece — it exports the full set of engine formats: GLB (binary glTF 2.0), FBX, OBJ, and USD, plus raw per-stroke JSON and the LATK stroke-interchange format. There is no native Vision Pro build yet, but its output reaches Vision Pro through conversion (below).

Two companion projects make the export land correctly, because brush strokes are not ordinary meshes:

  • Open Brush Toolkit — a Unity SDK (UnitySDK v24.x, Unity 2019.4+). Copy an exported GLB into a Unity project and it reassigns the correct brush materials and shaders on import.
  • three-icosa — an npm package that implements the GOOGLE_tilt_brush_material glTF extension for three.js / WebXR. Register it with a GLTFLoader, point it at a brushes directory, and Open Brush strokes render with their real shaders instead of flat polygons.

That pairing — a free open painter plus extension-aware loaders for Unity and the web — is what makes Open Brush the safe default.

The rest of the free toolkit, by what it does

Pick by output, not vibe. Everything below is realistically usable for free at a weekend, and every entry produces real geometry unless noted:

  • Low-poly modeling — Open Blocks (Icosa Foundation, open-source, Quest native). The Google Blocks successor; exports OBJ.
  • Pro 3D design / CAD-ish — Gravity Sketch (Quest native + PCVR). Its free tier is genuine and permanent (reworked February 2026); in-headset export covers OBJ, FBX, and glTF/GLB, with IGES (NURBS/CAD handoff) reserved for paid tiers. Export from the web inspector without re-entering VR.
  • AI text-to-3D, in VR — Masterpiece X (Quest native, free app + 250 starter credits). Prompt a rigged, animated model into existence; exports GLB.
  • Illustration + animation — Quill (free, PCVR — needs a Windows PC and a tethered headset). Exports USD, FBX, and Alembic; note there is no glTF, and its frame-by-frame animation only survives as Alembic (see recipes).
  • Architecture / spatial design — Arkio (Quest 3 MR native + PCVR). Free Starter tier exports glTF/GLB and OBJ.
  • Sculpting (“digital clay”) — Adobe Substance 3D Modeler (VR + desktop). Best-in-class USD/glTF/FBX/OBJ export, but no free tier — only a 30-day trial, feasible for the weekend if you plan around it.

A note on what to skip for a pipeline: the 2D VR painting apps (Vermillion, Painting VR, Kingspray) are a pleasure to use but output flat images, not meshes. They earn their place only if you specifically want a painted texture.

The export discipline — where it actually bites

Getting a file out is easy. Getting it to look right in the engine is the real work, because painted and sculpted assets lean on features that silently fail in default pipelines. The rules that save the afternoon:

  • GLB is the universal key; USDZ is the Vision Pro key. Anything that exports GLB drops into three.js, Babylon.js, Godot, A-Frame, and <model-viewer>. Anything bound for native Vision Pro must become USDZ.
  • Use KHR_materials_unlit. Painted art carries its color in the strokes; under a metallic/rough PBR shader that color washes out or disappears. Unlit makes the painted color the final look. Supported across three.js, Babylon, Godot, and <model-viewer>.
  • Set materials double-sided. Brush strokes are often single-plane — without doubleSided=true (uncheck Backface Culling in Blender) they vanish at grazing angles.
  • Avoid Draco compression on painted assets — it can corrupt the vertex colors the look depends on. Compress textures with KTX2 instead.
  • Brush shaders need an extension-aware loader. Without three-icosa (web) or the Open Brush Toolkit (Unity), animated/custom brushes lose their appearance on a plain glTF import. Particle and transparent brushes are lossy even then — keep test sketches to simpler brushes, or keep the JSON/LATK to rebuild geometry yourself.
  • Godot has a known vertex-color bug (4.3–4.4.x): vertex colors can import black in the Compatibility renderer. Enable “Use Vertex Colors as Albedo” and prefer the Forward+ or Mobile renderer.

For the glTF↔USDZ conversion itself, two free paths are reliable: Blender (imports glTF, exports USD/USDZ natively — the dependable route) and Google’s usd_from_gltf (archived read-only since June 2024, but still converts GLB→USDZ with KHR_materials_unlit support).

Per-target recipes

  • Quest 3 native app: Open Brush or Masterpiece X → GLB → import directly with the Meta Spatial SDK (it reads GLB/OBJ/FBX with no conversion).
  • WebXR demo: Open Brush → GLB → three.js + three-icosa (or A-Frame’s gltf-model). Apply unlit + double-sided, skip Draco. This is the most portable target — it runs on Quest 3 and Vision Pro Safari alike, and it is where the Babylon.js and A-Frame crowd here already lives.
  • Native Vision Pro: any tool → GLB → Blender or usd_from_gltf → USDZ → Quick Look, Reality Composer Pro, or the Safari <model> element. See the RealityKit + ARKit starter for the scene side.
  • Quill animation → Unreal/Unity: export Alembic (.abc) → Unreal Geometry Cache (or Unity’s Alembic package), and wire the Alembic vertex color into an unlit emissive material to keep the hand-painted look. (FBX cannot carry Quill’s per-frame deformation — Alembic is mandatory.)

Vision Pro’s input shift — and why it matters for art

Vision Pro defaults to hand-and-eye input, which is poor for precise drawing — but that changed with visionOS 26 (2025), which added support for 6DoF spatial accessories. The creative one is the Logitech Muse stylus ($129.95, October 2025): a spatially tracked pen with a pressure tip, a force-sensitive side button, and haptics, purpose-built for drawing and sculpting. Sony’s PSVR2 Sense controllers ($249.95, November 2025) are also supported, aimed more at games. By visionOS 26.2 even Apple’s own Freeform and Notes accept Muse input, and third-party creation apps followed.

So controller- and stylus-based art on Vision Pro now genuinely exists, though the app ecosystem is still young, and the headline VR painters (Open Brush, Gravity Sketch) have no native visionOS release yet. For now, the native-AVP art path runs through small USDZ-exporting apps (Spatial Drawing, VoxScan — the latter also does glTF/OBJ), or through the cross-platform tools plus a conversion step.

Sharing, and the one device that doesn’t fit

For hosting and reuse, the Icosa Foundation also runs Icosa Gallery — an open-source, self-hostable replacement for the shuttered Google Poly, with a public API for fetching models. Open Brush can upload a sketch straight to it from inside the app. For free CC0/CC-BY assets to build against, poly.pizza has an API too.

The exception is Snap Spectacles: art made in a Lens stays as locked Lens content (Snap Cloud JSON), with no standard mesh export to glTF or USDZ. Lens Studio happily imports glTF/FBX/OBJ, so Spectacles is a fine place to show art authored elsewhere — but it is a create-and-display target, not a source for the asset pipeline.

Caveats

  • Free does not mean frictionless. The tools are free; the export breakage (vertex colors, unlit, double-sided, Draco, the Godot bug) is the real cost. Budget time to get one asset looking right before producing many.
  • Brush fidelity is lossy. Simple brushes survive a glTF round-trip; particle and animated brushes do not, even with the toolkits. Design around it.
  • Vision Pro native art is early. Controller/stylus support is new (visionOS 26+) and the big painters aren’t ported — plan on a conversion step, and confirm any small app’s price and export before relying on it.
  • Check the live details. Free-tier limits, app prices, and store listings shift; verify before the event. Open Brush and Open Blocks, being open-source, are the most stable bets.

Questions? Reach the team via the Contact page.

// BACK TO NEWS