Splats to polygons: how we rebuilt a real house for a baby to wreck

Big Baby Energy is set in a real house: an actual family home, captured room by room and rebuilt so a baby can climb the furniture, lick the radiators and throw the toaster in the bath. (The Monitor confiscates the toaster. There are limits.)

The first version of that house was made of Gaussian splats. The one you play today is made of polygons. Both are in the game. This is how we got from one to the other.

A house in eighteen photographs

It starts with a Giraffe360 capture, the 360° tour estate agents use: eighteen 8K panoramas with known camera positions, a floor plan accurate to the centimetre and 27 stills.

Each panorama went into World Labs’ Marble, which turns a single 360 into a Gaussian-splat world: a cloud of soft, coloured blobs that adds up to a photograph you can walk into. We registered every world into one house frame by matching its walls to the floor plan, rendered them with Spark inside three.js, and the game’s first playable build was set entirely inside it.

From standing height, it is astonishing. From 40 centimetres off the carpet, it is a different story.

Why splats weren’t enough

A baby’s-eye camera is the cruellest camera in games. It lives on the floor, up against table legs, sofa fronts and skirting boards, which is exactly where a splat world is weakest.

• Floaters and smears. Each world is rebuilt from one viewpoint, so whatever that camera couldn’t see clearly gets smeared. Our landing had a permanent ghostly curtain in the stairwell.
• Holes. At baby height, wherever no camera had looked, the floor had dark gaps.
• Nothing is solid. Splats have no surfaces. Every sofa you could climb in the splat build was an invisible box we placed by hand.
• Nothing is an object. You can’t pick up a splat, so every toy was a separate model sitting in a photograph.
• Nothing is lit. Game lights don’t touch splats. When Quin shone a torch at the floor to tease the cat, nothing happened, so we drew a pink laser dot instead.

We tried patching it: real photos projected onto plain walls behind the splats, Marble’s textured meshes along the floor, a floater filter. Side by side, the patched house looked worse than the raw splats, and re-rolling the worlds with new seeds gave near-identical results. So we switched the patches off and did it properly.

Exact, not “inspired by”

The brief fitted on one line: every wall, door, skirting board, radiator, cushion and mug where it really is, at its real size, in its real colours. And because “looks about right” isn’t a measurement, we would prove it by rendering the polygon house from the exact spot each 360 was taken and scoring it against the real photo. Gemini went through every panorama and drew up the shopping list: 671 objects across 12 areas.

Five teams, one house

Five teams built in parallel in Blender, all writing into the same house coordinates under the same rules. On the top floor: one on the shell (walls, floors, ceilings, doors, windows and joinery), one on the landing and staircase, one on the nursery, one on the main bedroom and en-suite, and one on the second bedroom and bathroom. That floor came together overnight: 207 pieces.

The next day the same teams were re-scoped. The shell team took the ground floor, then the outside of the house; the rest split the hallway and utility, the kitchen-diner, the living room, and the garden, street and sky. A lead integrated, verified and relayed notes between them.

Measuring a room with a photograph

A 360 is a measuring instrument, if you know exactly where the camera was. Ceiling heights were read off one-centimetre rulers laid over crops of the wall–ceiling line: 2.35 to 2.415 m upstairs, and no two rooms agree. The cot has 13 slats a side at a 102.5 mm pitch, and the model matches the photo slat for slat. Marble earned its keep here too: its textured meshes, registered into the house, told us where everything stood and how big it was.

Then there were the cameras. “Close” shows up as a doubled kettle when two photos paint the same worktop, so we bundle-adjusted the capture poses: project each room’s edges, corners and door frames into its panorama and solve for the position, heading and tilt that line them up. One kitchen camera moved 9 cm and turned 3°; one garden camera was 5° out. Real houses don’t hold still, either: the airing-cupboard door is open in two photos and shut in a third.

Painting with the real photos

For every surface we render depth from each capture position, work out which cameras genuinely see each texel, and project the real 360 onto it, blended by viewing angle and distance, with exposure matched between captures. The texture is, quite literally, the photograph wrapped back onto the geometry. It already holds the room’s real light, so these materials are drawn unlit. Surfaces no camera saw, like the underside of an armchair, are lit in Cycles under the room’s own 360; faces only glimpsed edge-on, like the tops of skirting boards, are repainted as clean paint.

Hard-surface things (the cot, wardrobes, kitchen units, radiators, doors, even light switches) are modelled procedurally from measurements. Softer things were cut out of the photos with SAM 3.1, cleaned into studio references with Nano Banana Pro and put through a bake-off between Meshy 7.1 and Hunyuan 3D 3.1 Pro. Image models drift on half-hidden objects, though: our armchair came back as a slipper chair with brass castors, and the laundry hamper as a picnic trunk. One prompt said “black handle” from memory, and a kettle with a steel handle duly grew a black one. Early on the first night our main generation budget ran dry, so apart from a few hero objects, everything after that was procedural geometry plus the photo bake. The rooms got better.

Checking our homework

The verification harness renders five views from every capture pose, puts the real photo, the polygon house and the old splat world side by side, and scores each with SSIM, a structural-similarity score where 1.0 means identical. On the top floor the splats scored 0.25 to 0.55 against their own photographs; the finished polygon rooms score 0.64 to 0.95. In the garden, scores went from 0.13–0.50 to 0.60–0.89. One view of the downstairs loo scores 0.99.

The harness needed checking too. ImageMagick reports one minus SSIM, so our first scores were upside down. The photos weren’t tagged as sRGB, so they were colour-encoded twice. And Spark rebuilds its splat buffers asynchronously, so a polygon shot taken straight after a splat shot still had ghostly splats in it.

We also learned when to stop chasing the number. On the last night, scores at the capture poses dipped by a hundredth or two as teams swapped photo-painted objects for properly modelled ones: the play gym, the brass, the gold bed frame. From a baby’s eye height they are far better, and that’s where the game is played. Carpet is the classic case. Projected from a camera 1.2 m up, it’s a beige blur, so a high-passed tile of the real carpet is multiplied into the floor and faded out beyond 1.35 m from the camera. The capture views stay true to the photo; the baby-cam gets crisp loop pile.

Making it run in a browser

The whole house came to 504 texture maps, about 10.3 GB of GPU memory uncompressed. Our test machine swapped itself into the ground, and boot time went from 18 seconds to 178. A KTX2 release of every model (Basis UASTC, capped at 2K indoors and 1K outside) brought it down to 25 seconds, with indoor SSIM within ±0.003 of the originals.

Physics runs on Rapier against a walk mesh built from the polygon house itself: 158,000 triangles from 86 structural pieces, with the stairs kept as a smooth ramp so bum-sliding stays dignified. It found bugs no screenshot would. A 2.5 cm overlap at the top of the stairwell, invisible from every angle, became a kerb Quin couldn’t climb, so the bum-slide order never fired. And the real fridge turned out to be 2.12 m tall, which meant the game had been keeping the biscuit tin 20 cm up in thin air. Now a scripted run plays all 13 orders, the launch and the credits in the polygon house.

Why we kept the splats

The splat house is beautiful. Soft, glowing, a little impressionistic, it looks the way home feels when you’re very small and very full of sugar. So it stayed.

Once Quin has seen the Biscuit Moon, press R (or right on the D-pad) for a SUGAR RUSH. The splat worlds stream in behind a shimmer, then the polygon house melts away in a sphere spreading out from the baby as its splat twin fades in. Both share one house frame, so nothing jumps. Colours saturate and wobble, the field of view breathes, the music slides into a woozy lo-fi dream, gravity roughly halves, and the Monitor notes that the subject’s perception of reality has been… updated. Forty seconds later comes the comedown.

What we learned

• Splats are for looking, polygons are for touching. Anything a player can reach needs to be a surface.
• Don’t patch a splat world. Replace it.
• The photo is the best texture you’ll ever get, and the best test. Project it, then score against it.
• Test the tester. Every one of our early harness bugs made good rooms look bad.
• Design for the lowest camera. A baby sees the underside of everything. Now, so does our house.

The house is waiting, and it has never been tidier. Go and fix that.