Aura Underground: One Frame

One frame of Aura Underground taken apart: light path traced in a browser, live global illumination and why the race bakes it anyway, car reflections borrowed from the frame before, a phone build that fakes what it can't afford, and the Rust engine underneath.

Aura Underground: One Frame

Launch Web Experience

One frozen frame at golden hour, built up one layer at a time
One frozen moment, driving into the sun, built a layer at a time

The first case study was about the game: the cars, the films, the multiplayer. This one is about a single frame: what goes into the picture, the textures and buffers behind each effect, how the light bounces, and the engine underneath, which is written in Rust.

Above, one frozen moment is built up a layer at a time. Here's the same build at speed, the car racing into a low sun while every layer switches on.

Golden hour, into the sun: every layer switches on while the car races

Every screenshot and clip here is the game's own output from a real race, and every texture was read straight off the GPU mid-race. The diagrams are drawings, and the breakdown figures only add labels and outlines.

One frame, taken apart

The race is seeded and the physics runs at a fixed 120 steps a second, so I can stop it on any tick and get the same moment back on every load. That lets me switch the renderer's layers on one at a time over one frozen frame. This one is level one, the airport circuit, at golden hour: an Audi R8 in satin Electric Rose Gold, 39 seconds in, on the long straight with the sun 9° up and dead ahead.

While the GPU draws, every material also records what its pixel is made of in a set of screen-sized images called a G-buffer: its colour with no light on it, which way it faces, how rough and how reflective it is, and how far away. Motion vectors record how far each pixel moved since the last frame. The lights, the wet road, motion blur and anti-aliasing all read these instead of drawing the scene again.

The G-buffer targets of one moving frame, and the light before and after the grade
One frame pulled apart. The car is black in the motion view because it moves with the camera

The picture itself is drawn in HDR, half-float colour where a light can go far past white. In this frame the sun reaches 34 times white, and stays there until the grade at the very end.

Albedo. Everything starts as surface colour with no light: the rose gold paint, the asphalt, the green of the trees. Only the sky is lit.

Albedo only: the R8, the road and the trees flat and unlit under a lit sky
Albedo: surface colour, no light

Baked light. Multiply the albedo by the lightmap and the scene is lit. That light was path traced offline for this level at this time of day, so the barriers go warm against the sun and the road comes out flat and grey. The cars aren't in the bake.

The same frame lit by the golden-hour lightmap
+ Baked light, path traced ahead of time

Wet road. After the bake, this is the biggest single change. The damp patches turn to mirror and take the bright sky, the sun's glare runs down the road beside the car, and the dry patches stay dark. Rain streaks pull bright light down into the road, but only on pixels the G-buffer marks as more than 30 % reflective, and only light brighter than white smears.

Wet patches mirroring the sky, the sun's glare running down the road
+ Wet road

Reflections and live lights come next. On the road they're small by day: on an open circuit the wet road already shows most of what there is to reflect, and the live lights add only a little glow under the car. At night they do much more.

The car's reflections were the open question. We obviously can't capture a cube map of every scene for every car, so how do we fake that? My answer was to have the car reflect the sky and a matcap made from the previous frame.

Every frame, the game shrinks the finished picture into a small HDR copy, 340 × 192 pixels, plus a blurred 85 × 48 version. On the next frame the car looks its reflection up in that copy by the way each panel faces. A gloss car reads the sharp copy, and a satin one like this R8 mostly reads the blur. Only a band around the horizon ever reaches the body: the skyline, the sun and the far end of the track. That band is the warm glow across the R8's tail.

The copy as the GPU holds it, and the car with reflections only
The copy, read back from the GPU mid-race, and the same frame on the car. Only the blue band ever reaches the car
Left, the copy as the car drives into the sun. Right, the car one frame later, wearing it

To see which part lands where, we swapped the copy for a colour pattern for one frame and let the game draw the car as usual. The left flank shows the left of the picture and the right flank the right. The tail faces the camera, so it shows the middle, the road and the sun dead ahead. How far a panel tilts picks the row. The roof and rear glass show blue instead, because the copy holds the street, not the sky overhead: from about 37° up the shader starts swapping in the baked sky, and by about 67° it's all sky.

A colour pattern in the copy, and where each colour lands on the car
Which part of the copy lands where: columns by the way a panel faces, rows by how far it tilts, sky from about 37° up

It replaced a live 256-pixel probe that read low-res and flickery on the body and the rear glass.

Sun shadows. With the camera facing the sun, this step is a big one. The R8's long shadow is cut out of the baked sunlight and runs straight back at the camera, never taking away more than 94 % of the light under it. The filter sizes its blur from the sun's real disc, about half a degree across, so the shadow softens the further it falls from the car.

The R8's long shadow running back toward the camera
+ Sun shadows: the shadow coming back at the lens is how you know we're facing the sun

That shadow comes from four shadow maps, called cascades, drawn from the sun's point of view. Each covers more ground than the last, 17, 40, 86 and 228 metres across at 2048 pixels square, so near shadows stay sharp and far ones stay cheap. The track's shadows are already in the lightmap, so the cascades hold nothing but the cars.

The four sun cascades as depth maps, holding only the cars
The sun's four shadow maps. Each holds only the cars, seen head-on from the low sun

Motion blur. A frozen frame has no motion, so for the last steps the race runs one real frame up to the same tick. In the motion vectors the R8 reads almost black, because it moves with the camera, while the road and the trees streaming past are bright.

Motion vectors: the car dark, the world around it in colour
Motion vectors: the car barely moves against the camera

The blur first finds the fastest motion in each 128-pixel tile of the screen, then gathers along each pixel's motion with a 1/75 s shutter: 32 samples on MAX, 48 on spinning wheels.

Road and barriers streaking past a sharp car
+ Motion blur

TAA, bloom and the grade. Temporal anti-aliasing jitters the camera through 14 positions and blends each pixel with where it was in earlier frames, which settles the catch fence's shimmer and the car's edges. Then come bloom, built from six targets down to an eighth of the screen, and the grade: an ACES tone curve, split toning, a vignette and a little grain.

TAA's current frame alone, before any history is blended in
One frame without TAA's history: the fence and the car's edges are jagged
The finished frame, driving into the sun
+ TAA, bloom and the grade: the finished frame

The same stack with the camera orbiting the frozen moment, then the engine's own debug views.

The golden-hour stack, orbiting the frozen frame
The engine's sixteen debug views, each with its own label

On the same lap in the morning the sun is 15° up, just above the frame, and the wet road breaks it into a band of glitter.

Morning on the same straight: the stack with the sun just above the frame

Night, and the lamp posts

At night the lamps and the car lights do the work. This is level one's floodlit start straight, five seconds in, the pack bunched ahead. The floodlights' pools are already in the night bake.

The night lightmap on the start straight: pools on the barriers, the lamp heads still dim
+ Baked light: the floodlight pools are in the bake

Because their light on the level is baked, the live lamps only light the cars and the shine on the wet road, with the 40 nearest switched on. Each light is drawn as a shape (a sphere, a box or a spot cone), and every visible light of one shape goes out in a single instanced draw, one call that stamps out all the copies.

The live lamps on: the towers light up down the straight
+ Street lamps and glow: now the lamps light the cars too

Headlights, tail lights and underglow are live lights of the same kind. The player's two headlights are shadowed spotlights that reach 78 metres.

Headlights on the cars ahead, red behind the R8
+ Headlights, tail lights and underglow
The deferred lights alone on a plain grey surface
The live lights on their own: headlights on the cars ahead, a lamp pool on the barrier, red behind the R8

Night shadows come from the moon through the same cascades, plus up to six shadowed spotlights that hop to whichever lamps are nearest you.

The finished night frame on level one
The finished frame: the floodlights and the headlit cars flare

Baking the light

My rule for this game was: anything that's not moving, if we can bake it, let's bake it. Light, static shadows, the sky, visibility and reflection probes are all worked out ahead of time, so the live GPU time goes to what moves: the cars, the reflections and the wet road.

The baker is a path tracer written for WebGPU, running in headless Chrome. A path tracer follows rays of light as they bounce around a scene, which is how film renderers get soft, believable light. One command bakes a level through eleven cached stages. It unwraps the level into flat charts (10,470 on level one's road) at 8 texels a metre, a texel being one pixel of the lightmap. Then it traces up to 256 paths a texel with three bounces and a shadow ray to every lamp, denoises without crossing a chart edge, and stitches the seams.

Diagram of the bake, from level file to one byte a texel
Baking the light, in a browser
A lightmap converging: 1 to 256 samples a texel, then the denoiser, then colour

The charts are packed onto 4096-pixel sheets, and in the race the road just multiplies its colour by the sheet. All the floodlights' light and shadow costs one texture read.

The lightmap atlas, and where one chart of it lands on the track
The orange piece is the 60 metres of track under the car: a flat strip on the sheet, the road in the game

Level one's night atlas is 7,190,697 texels and took 351 seconds, and all ten levels are baked at all four times. Here's the night bake from above, with the cars hidden.

The night lightmap alone, from above, on grey
The night bake from above: every pool, and the barriers' shadows cut into them, was path traced
The same raised camera, the finished frame
The same camera, the finished frame

Reflection probes are baked too: a virtual camera every 25 metres along the track, 1.2 metres up, captured the whole sphere around it, folded flat into a square with pre-blurred copies for rougher surfaces. Level one has 72. Each frame the game picks the ten nearest along the road, leaning toward what's ahead, and every pixel blends the two closest. The probes are projected through boxes that hug the road edges, so the reflections line up with the barriers.

A reflection probe unfolded, and which probe each pixel reads
One probe: sky in the middle, road in the corners, with pre-blurred copies. On the right, each colour is the probe a pixel reads

Because the race is deterministic, the same tick can be shot at all four times of day, each with its own bake and sky.

Same race, same instant: morning, afternoon, golden hour and night

Light that bounces

Global illumination, GI for short, is light that bounces. The sun lights the road, and the road throws some of that light back up onto the walls, the underside of a bridge and the bottom of a car. Without it, anything the sun and the sky can't see goes black, and that's where the flat, video-gamey look comes from. GI is most of what makes a frame look real, and it's what those three bounces in the bake are for.

Level one is an open circuit with no bridges, so for this section I moved to level nine at golden hour, with the R8 under a wide overpass.

Under the overpass with GI off and on, GI's own light, and the nested boxes
One switch apart. With GI off the underside of the deck gets no light at all. With GI on, sunlight off the road and the slope beyond the bridge bounces back under it

The engine can also do GI live. It builds a rough copy of the world around the camera out of ¾-metre blocks, lights the blocks with the sun and the sky, and lets the light bounce between them. Anything standing in the world picks up its share, the cars included: with GI on, the R8's body goes from 44 to 71 on a 0 to 255 scale. The blocks sit in four nested boxes that ride with the camera, each one 48 × 24 × 48 blocks and twice the size of the last, so detail stays near the car and the far light stays rough. Each frame the engine rebuilds one slice of one box and relights two, which keeps up with a car at full speed. Parts of it are ported from kansei, an open-source project.

Under the overpass: GI off, GI on, then GI's own light on plain grey

Baked or live

The engine can do both, and every race level ships baked; live GI covers levels and hours without a bake. I asked at the time: "Is there any reason why we wouldn't use the bake for desktop too? Again it's a static map." I also asked to "leave a debug toggle to keep it live and then we'll bake later." That toggle is why the live frames here exist.

The bake costs nothing a frame, because the light is a texture: 256 paths per 12.5 cm patch, three bounces and exact shadows, traced once. In a full race on level five, live GI cost 21.6 ms a frame. Live GI follows any time of day and anything that changes, which a bake can't. But it's an approximation. Its blocks are ¾ of a metre and its probes don't check for walls, so some of the sunny road's light gets in under the deck, and in the finished frame the underside comes out about four times brighter than in the bake. The bake is the reference, and it's the same on every frame, where live GI is rebuilt a slice at a time.

Baked and live on the same moment, the bounce light alone, and why the race bakes
The same moment both ways. The bake is traced once and read from a texture; live GI follows any change but lets more light under the bridge

Light probes for the cars

The cars move, so they can't be in the bake. Instead the bake leaves a strip of small light probes along the whole lap: one every 5 metres, three across the road 7 metres apart, at two heights, 1 and 4 metres up. Each probe records the light arriving from every direction at its spot, and under the overpass the strip drops to 27 % of the open road's light, almost two stops darker.

The probe strip drawn by the engine under the overpass, and its light along the road
A probe every 5 metres, three across and two heights. The tops go dark where the deck hides the sky. Pink ones sit inside a wall and are switched off

They're there to give each car the light of the spot it's driving through, but on this build they don't reach the cars' shading yet, so by day a car keeps the open sky's flat light. I'd noticed it in testing: "during the day ... the car is like one solid color even when the ground is ... darker." Wiring the strip in is an engine fix still to do. Where live GI is on, the cars already get bounce light per pixel.

Sky occlusion

On a sunny street a lot of the light comes from the blue sky, not the sun. Under a bridge most of that sky is hidden, so its light should go too. Sky occlusion works out how much sky each surface can see. It draws the level around the camera into a height map, from above and below, and checks 24 directions from points a metre apart. Under the deck the road falls to under a tenth of its open-sky light, and the car loses most of its sky light too. The cars are left out as blockers so they don't shade the road under themselves; their shadows do that. On baked levels the lightmap already holds all of this, so sky occlusion runs only where a level is lit live.

The sky each surface can see, and the sky's light without and with sky occlusion
How much sky each surface sees. With it on, the road under the deck falls to under a tenth of its open-sky light

The sky

The sky is its own module: single-scattering air, volumetric clouds, a sun placed by the NOAA solar calculation and a hand-tuned twilight palette.

Live, it costs about 5 ms a frame, so every tier ships a baked dome for each race time, 4096 × 2048 on desktop, wrapped round the camera. The race times are morning at 7:00 with the sun 15° up, afternoon at 14:30, golden hour at 17:24 with the sun at 9°, and night. There's no noon, because a sun straight overhead leaves no shadow under the cars.

The four baked sky domes, golden hour on top with the race frame outlined
Golden hour's dome, with the part the race frame sees outlined. Below it, morning, afternoon and night

Night comes on with the sun: between 3° above the horizon and 6° below it, the lamps, the city glow, the windows and the neon fade up together. The stars are a hash. The sky is cut into tiny direction cells and about 0.12 % of them light up. Stars are smaller than a texel of the dome, so the bake averages 4 × 4 directions for every texel.

A whole day of the live sky

The hours in between only exist live.

Pink clouds over an orange horizon
17:48, sunset
Stars over the circuit, floodlights on
21:00, night

The engine underneath

All of this sits on Aura's 3D engine, and the core of the engine is Rust. The reason is portability. One Rust core does the engine's work (transforms, the scene hierarchy, culling, sorting, the draw list) on every platform, compiled to WebAssembly in the browser and to a native library everywhere else.

What keeps it cheap is that JavaScript objects are views over Rust's memory. The core owns every node's position, rotation and matrices in one block of shared memory, 270 bytes a node. When game code reads a car's position or a wheel's matrix, it's reading those bytes in place.

Diagram: JavaScript reading the Rust core's shared memory
JavaScript reads Rust's memory directly

Each frame the core culls and sorts the scene into a display list, the list of things to draw. A race frame is about 1,350 draws over 14 to 16 views: the camera, each shadow map, each light's map. JavaScript calls into the core a few times per render pass, and nothing crosses per draw.

When the core replaced the old JavaScript engine it had to match it to the last bit across 58 recorded scenes. Getting there meant porting V8's own sine and cosine, because Apple's maths library differs from V8 in the last bit for 4 to 18 % of calls.

Shaders work the same way. Every material is written once in Aura's GLSL, and a Rust assembler and the naga compiler translate it to WGSL for WebGPU and GLSL ES 3.00 for WebGL2. Every shader the game uses is generated at deploy, 191 of them packed into 4.3 MB, so a first visit translates nothing and a phone downloads only the 27 its tier needs.

Diagram: one material translated for WebGPU and WebGL2
One shader, every browser: one material, translated for WebGPU and WebGL2

The static level is GPU-driven, and baked visibility for every 25 metres of road cuts its draws by 76 to 85 % on HIGH. Each car is merged into one draw that still spins and steers its wheels: every vertex knows its wheel corner, and a small float texture holds a transform for each corner. That took 18 cars from 513 draws per view to 170.

Physics in Rust

Anvil, my physics engine, is Rust too: a rewrite of OIMO compiled to WebAssembly. It steps at a fixed 120 Hz and gives the same result for the same seed, which is what brings every picture in this piece back to the same tick.

Render it, bake it, fake it

On desktop I wanted to go full triple-A, and the pipeline works the way a AAA game's does. Draw at full quality where it's affordable, bake and cache everything that doesn't move, and on phones fake the rest so it still looks close.

Diagram: what desktop draws live, what's baked, and what a phone fakes
Render it, bake it, fake it

Desktop draws on top of the bake and keeps the expensive parts live: deferred lights over the G-buffer, headlights with their own shadow maps, four sun cascades and six lamp shadows, the wet streaks and TAA. The bake is shared by every tier, and it goes further than light. Even the shape of a low beam is baked, as a 256-pixel drawing of the flat cut-off, the kick-up on the right and the hot spot below.

A phone keeps that baked work and fakes what's left. Its headlights are two simple spotlights shaped by the low-beam drawing, one texture read a pixel, in place of the deferred light pass and the headlights' 4096 × 4096 shadow atlas. Its wet road mirrors a 680 × 384 copy of the last frame, with the bright parts smeared down for the lamp streaks, where desktop reads the baked probes and runs its own wet-streak pass. The car's reflections use the same last-frame copy as on desktop.

MAX and MOBILE on one moment, the low-beam drawing and the last-frame wet road
The same frame made for a phone: headlights from a drawing of a beam, a wet road that mirrors the last frame

The rest is the desktop's bake at lower resolution: a 2048 × 1024 sky dome that downloads in 297 KB, and a lightmap at 4 texels a metre, made by averaging the desktop bake. Shadows come from one map that follows your car and one drawn for the whole track when the level loads, and the other cars get a soft blob underneath. There's no G-buffer and there are no deferred lights. FXAA smooths the edges in one pass instead of TAA.

The phone's smaller sky and lightmap, its two shadow maps, and what isn't there
Smaller bakes, cheaper shadows: the same 13° of sky and 64 metres of lightmap on each tier, a shadow map for your car and one for the track

The frame, again

Every picture here came from a seeded race that comes back to the same tick and the engine's own switches. The light bounces three times, but it was traced once, in a browser, ahead of time, which leaves the live frame free to spend its time on what moves. Here's the whole film.

One Frame: the film

Play Aura Underground