Aura AR Color Changer

Aura AR Color Changer

The Brief

For a big tuner car show, Aura Vinyl wanted a way for visitors to see what their car would look like in any of Aura’s 600+ colors live and in person. The idea: just point your phone at your car, and the wrap color changes instantly.

The Challenge

Color replacement in AR isn’t new but doing it believably is. Every car has reflective paint, varying gloss, and real-world lighting that changes minute by minute. Our goal was to build a system that could recognize each panel, preserve reflections, and apply a new finish that actually looked like paint not a flat overlay.

The Idea

We built a TensorFlow-based segmentation model fine-tuned on this exact car (an Audi TT RS) and its color, Ultra Gloss Fluro Yellow. We used hundreds of labeled reference frames shot from multiple angles and trained the model to isolate paint from trim, glass, and wheels with pixel-level precision.

To account for lighting, we ran a one-hour fine-tuning session on-site the morning of the event, capturing live exposure and tone data under that day’s sunlight and using transfer learning to adapt reflectivity curves in real time

An early test in segmentation

The Build

The final paint segmentation

Here’s how it worked under the hood:

  1. Segmentation: A custom TensorFlow JS model ran directly in the browser using WebGPU, identifying this exact paint color with ~95% confidence in real-time.
  2. Color Mapping: The paint regions were temporarily desaturated (to a pseudo-grayscale albedo pass) and recolored using a LUT derived from Aura’s PBR material library.
  3. Reflection Preservation: We layered the result with a luminance-based blend to preserve highlights and environmental reflections from the live camera feed.
  4. Finishing Pass: The final composite was tone-matched and rendered with a perceptual contrast curve, giving the illusion of true metallic paint reacting to the real world.

Everything ran client-side — no app download, no cloud GPU. Just TensorFlow.js and WebGPU.

The Outcome

The result was an AR demo that felt like witchcraft. Visitors walked up, pointed their phones, and watched their car “repaint” itself live.e.

While this prototype was built specifically for the show, it hinted at something much bigger: a future where car customization, ecommerce, and AR blend seamlessly. No configurator, no imagination gap — just instant visualization.