Building Lola's house as real 3D rooms

Lola is the lead in The Life of Lola, my own series made with AI video. Her house only existed as a few pictures, and the pictures didn't agree with each other. I wanted rooms the camera could move through, with the furniture in the same place in every shot. This page is the whole attempt: what I tried, why, what went wrong and what fixed it.

Experiment

1. A made-up bedroom, from one picture

The first 3D room, made by a world model from one picture of a made-up bedroom: a bed against the left wall, shelves, string lights, a window at dusk and a desk
One picture in, a whole room out. It looks the best, and the world model invented most of it.

2. Lola's kitchen, draft, from 27 views

The draft 3D build of Lola's kitchen from the main view: sage green cabinets, an orange tiled splashback, a wooden island with two stools, green pendant lamps and an arch with stairs beyond it
Grown from one trusted picture. It's the older, longer kitchen, and it still has a window that shouldn't be there.

3. Lola's bedroom, draft, from 11 pictures

The draft 3D build of Lola's bedroom: a bed with a lilac duvet, a window at dusk, shelves, fairy lights and pinned photos, with fine scratchy lines on the duvet and curtains
Built from a room made to the floor plan. Right from the 11 camera spots, scratchy from anywhere else.

4. Lola's bedroom, refined, from the same 11 pictures

The refined 3D build of the same bedroom from the same spot: the photos, lamps and shelves are sharper, and the window is darker
2 million blobs and a 31 MB file. It's heavy for an older computer or a phone.

Walk around the rooms. These are the four 3D rooms this work produced, in the order I made them. Press play on any of them, then drag to look around. To move, drag the dot in the circle at the bottom left, or click the room and use the W, A, S and D keys. The rest of the page explains why there are four, and what's wrong with each one.

The problem: a house made of pictures that disagree

Every background in the films is a picture an image model made. That works for one shot. It breaks when the camera has to show the same room from another side, because the next picture is a new guess. The fridge moves. A door changes colour.

My own reference pictures had the same fault. My picture of Lola's bedroom shows the room from four sides, and the four sides don't fit together. Three of them use up all four walls, which leaves no wall for the wardrobe and the vanity in the fourth. The door is navy in one panel, wood in another and white in a third.

A world model can turn pictures into a 3D room, but only if the pictures agree. When they don't, it builds both versions on top of each other, and you get doubled, ghosted furniture. So the real job was to get one layout that every picture agrees with.

First try: one picture in, a whole room out

Before Lola's house, I tested the simplest route on a bedroom I'd made up. I gave a world model one picture and it made a whole room I could walk around. It's room 1 above, and it's still the best-looking of the four, because the world model paints every wall and corner itself.

The problem is that it invents everything the picture doesn't show. Turning around, I found a tall bookcase, a dresser with a screen on it and doors I never asked for. The desk lamp became a floating glow and the rug changed pattern.

I also couldn't change it afterwards. Adding a sentence asking for a desk didn't add a desk, and it changed other things. Cutting the bed out left a hole, because the model had never made the floor under it. For a film set, a room I can't control is the wrong kind of room, however good it looks.

The kitchen: growing a room from one picture I trusted

Lola's kitchen had two pictures that agreed well enough. They still disagreed in small ways, so I chose one as the truth and used the other only as a written description of the far wall.

Then I asked a world model for 11 more views of the same kitchen. The half of the room that my picture shows came back the same in every view. The half it had to invent came back different every time. Through the arch there were stairs in some views and a dining room in others. Extra windows appeared past the fridge. I kept five of the 11 views.

I tried to fix the invented half with words, and then by masking the wrong part and asking for it to be repainted. Neither worked. Twice I asked for a framed painting to be removed, and twice it came back as a small square clock.

The two pictures the kitchen grew from

The first is the one I treated as the truth. The second was only used as a description of the wall behind the camera.

Lola's kitchen in warm evening light: sage green cabinets, an orange tiled splashback, a wooden island with two stools, two green pendant lamps, and an arch on the left with stairs beyond it
The main view, towards the stove. The arch on the left leads to the hallway.
The same kitchen from the other side: the sink under a window, the fridge, the island, a doorway to a dining room with a table, a round clock, a shelf with hooks and a white door
The opposite view, with the doorway to the dining room.

What fixed the kitchen

The thing I learned is that the world model holds on to whatever a picture shows, and only loosely follows what I tell it in words. So I stopped describing the wall and started showing it.

The loop that worked has three steps. The world model makes a view with the right shape. An image model edits that one view exactly, with the view itself as the base. Then the edited picture goes back in as a reference, and every new view copies it.

The same rule explained a fault I couldn't shift. A second window kept turning up past the fridge. I traced it back to one of the five views I'd kept, which had that window in it. Every new view was faithfully copying my own mistake. Fixing that one source picture fixed all the views made after it.

Some of the corrections were mine to get wrong. I said the front door should be visible through the arch, and that was wrong too: through the arch you see plain hallway wall, and the stairs are off to one side. I asked for a bench beside the fridge, and later took it out again.

The second draft of the 3D kitchen is room 2 above. It was built from 27 views that agreed, after four more were left out. It was made before the last two fixes, so it still has the extra window.

A crash, and a cheaper answer

While I was looking at the first 3D kitchen, my computer crashed. It's a 2020 laptop with built-in graphics driving a 4K monitor, and the viewer was redrawing the room at full size on every frame. The crash report named the graphics driver.

That changed how everything after it was done. The viewer now draws at a smaller size and only while you're moving. Every check of a 3D room since then was done on the processor, which is slow but can't crash the graphics.

It also made me ask what the 3D room was for. For a film background, I don't need to walk around. I need a sharp picture from a chosen spot. Asking the world model for stills directly gave sharper pictures than the 3D room, with no load on my computer. The 3D room is only worth it when the camera has to roam freely.

Why I stopped and drew a floor plan

Every correction I'd made to the kitchen was about how rooms connect: where the doorway goes, what's through the arch, which way the stairs rise. Pictures couldn't settle those questions, because each picture only shows one side. So I asked for an architect's plan of the whole house, to get it accurate before making anything else.

The kitchen was the one room with real numbers. The 3D draft let me measure it: 3.9 m by 5.9 m. It's drawn with solid walls. Every other room is dashed, because I laid it out myself.

After the first few versions, the plan became a short program that draws the picture, with an AI agent making each change. That way a correction is a change to a number, and the drawing can't drift. The house ended up 6.1 m wide and 15.5 m deep, on two floors.

The floor plan of Lola's house on two floors. Downstairs: a hallway, kitchen, pantry, dining room, laundry, bathroom and living room. Upstairs: a landing, three bedrooms, an office and a bathroom, with Lola's bedroom at the back
The floor plan, version 48. Solid walls were measured from pictures. Dashed walls are my design.

48 versions, and what each mistake taught me

Most of the 48 versions fixed something you only notice when you imagine living in the house. The downstairs shower had a 0.6 m gap to get into it, between the basin and the toilet. The fix was to make the bathroom door open outwards, which freed the floor. In the dining room, nobody could get past the table, so it moved into the bay window.

Some rooms are where they are because of other rooms. The living room is at the back of the house, not in the front room with the bay window. The kitchen's doorway has to open into the dining room, and the arch has to sit at the foot of the stairs. A living room at the front would have pushed the stairs about 10 m from the front door.

I changed my mind more than once. I asked for more furniture in the hallway four times, and then asked for most of it to go. I approved Lola's bedroom, and 20 versions later said her bed was on the wrong wall. Guessing what I meant took several rounds each time. One direct question, with the choices named, got a clear answer in one go. I use that for every unclear correction now.

The last change broke something. The walk from the front door to the stairs was 7.9 m, so I shortened the kitchen from 5.9 m to 5.0 m and got it down to 7.0 m. That means the 3D kitchen above is the old, longer kitchen. It no longer matches the plan, and it has to be built again. I've left it on the page because it's part of how I got here.

The bedroom: shape first, then paint

With a plan, I could build a room from the layout and not from a picture. The first version was a grey-box model: plain blocks for the walls, the bed, the wardrobe and the vanity, at the plan's measurements. A world model was given the depth of that model, so it knew where everything was, and words for the look.

Two things went wrong straight away. When my words mentioned a door, it painted a door in every view, including views where the real door is behind the camera. The fix was to leave doors out of the words and let the shape place them. And “late afternoon” made the whole room dark, so the lighting had to be spelled out in plainer words.

Five tries at the look, from one camera spot

The layout held from the first try. The look took five. The first five are the same view from the bedroom door, in the order I rejected them. The sixth is the one I approved, from a better camera spot.

The bedroom from the door as the world model painted it: dark navy walls, a mirror with bulbs, a bed with a red runner, a white door and a bright window, in a realistic style
1. The world model's own painting. Right layout, but it looks like a realistic render, and it added a door.
The same view restyled: warmer light, curtains, a patterned red rug and richer wood, still realistic
2. The style described in words. Still too realistic, and it changed the bedspread without being asked.
The same view in a bright cartoon style: bold blue walls, a big red rug, rounded furniture and a glowing window
3. Asking for an animated-film look in words. Now it's too much of a cartoon.
The same view in a dark, moody style close to the first reference picture: navy walls, walnut furniture and a small red rug
4. Sending my reference picture itself, not a description. The style is right at last.
The same view in a new mauve and lilac style: lavender walls, fairy lights, pinned photos, shelves with a trailing plant, a lilac rug and a dusk sky
5. A new reference with the colours I wanted. Better, but the picture is mostly empty wall.
Lola's bedroom from a lower, closer camera: the bed and the window both in view, lilac duvet, glowing lamps, fairy lights and a pink and purple dusk sky
6. The same style from a better camera spot. This is the one I approved.

What the five tries showed

Words failed twice at describing a style. “Too realistic” and “too cartoon” were both honest results of what I'd written. Sending the reference picture itself worked on the first go. It's the same lesson as the kitchen: show it, don't describe it.

The fifth try had the right style and still looked wrong, and the cause wasn't the style. It was the camera. A wide view from standing height in a corner shows mostly empty wall. I moved the camera lower and closer, to about 1.3 m high with a narrower lens, so the bed and the window were both in frame. The same style suddenly worked.

The next problem: eleven pictures that are each a new guess

One good picture isn't a room. A 3D room needs many overlapping views, and each painted view is a separate request to the image model. So the details drifted. The wardrobe had two doors in one view and three in another. Fairy lights turned up on more than one wall.

The image model also fills blank space. In the first eight views, three got a vanity on a wall that should be bare, and one got a whole second bed. What fixed it was telling it what isn't in the picture: “no window in this picture, the window is behind the camera.” A note like that fixed all three views.

Small things had to become real shapes before they'd be painted in the right place. A rug modelled 1.5 cm thick was ignored. At 4 cm, it was painted every time.

So the room was built properly

At this point I was frustrated that no model got the room right the first time. The answer was to stop asking models to imagine the room at all. An AI agent built Lola's bedroom in a 3D modelling program, from the plan's measurements, with every visible thing modelled: the fairy lights, the pinned photos, the books, the plant and the cat's bed.

That solved the drift completely. All 11 views now come from one model, so the bed is the same bed from every side. It had to run within my computer's limits: on the processor only, one picture at a time, about three minutes each.

What it didn't solve was the look. A plain render is exact and flat. So each render got one pass from the image model, with the render underneath to hold the layout and my approved picture as the style.

Shape, then built, then painted

The same camera spot at three stages: the grey-box depth, the plain render of the built room, and the painted version.

A grey depth picture of the bedroom made of plain blocks: a bed, bedside tables and a window recess, lighter where things are nearer
The grey-box model, as depth. Lighter means nearer.
A plain 3D render of the bedroom: flat mauve walls, a bed with a lilac duvet, a window with a dusk sky, shelves and fairy lights, all smooth and simple
The built room, rendered plain. Everything is in the right place, and it looks flat.
The same view painted by an image model: warm lamp glow, a fluffy duvet, woven rug, wood grain and a pink and purple dusk sky, with the furniture in the same places
The painted version. Same layout, and now it looks like Lola's world.

Checking the pictures

The image model still added things, even with a note on every picture saying what to leave out. In four of the 11, it put in things that aren't in the room: a rug on bare floor, extra shelves, lights and photos, and a mirror with the wrong reflection.

I didn't want to check 11 pictures by eye every round, so the checking was handed to AI agents. One agent compared each painted picture with its plain render, and a second agent had to confirm each fault before it counted. One reported fault was thrown out that way: a “missing bed” that was simply below the edge of that view.

The four pictures with invented things were painted again with a stricter note. After that, the layout matched in all 11. A few small extra props, such as candles, are still there.

A sheet of the 11 painted views of the bedroom, each from a different spot: towards the window, the bed head wall, the door, the vanity and the wardrobe
The 11 painted views the 3D bedroom was built from.

Draft against refined

A world model turned the 11 pictures into a 3D room. First it worked out where each camera stood. Because the room was built, I knew the real answer, and its guess was within about 20 cm on average. That told me the painted pictures still agreed with each other.

There was a trap in the settings. The slow, refined build is the default and costs ten times as much as the draft, and the option to ask for a draft isn't in the tool's list of settings. I had to ask for it by name. It worked, and the draft is room 3 above.

The draft looked like etchings to me. From the 11 camera spots it's right. From anywhere else there are fine scratchy lines, because the room is made of thin stretched blobs and you're seeing them from the side. Moving the page's camera stops onto the original 11 spots fixed two views out of four. The corner by the door and the wardrobe stayed smeary, because few of the pictures show it up close.

So I paid for the refined build, room 4. It has 2 million blobs, four times the draft, and most of the scratches are gone. I tried cutting it down to a lighter file for the web, to a half and to a quarter. Both left holes and speckles. It needs all 2 million, which makes it a 31 MB file.

Where it ended up

The plan was to use the 3D bedroom as the opening of my home page. Next to the first one-picture room, the refined room still looked blocky to me. I found two reasons. The tool takes each picture at 1280 by 720 at most, so the room can't be sharper than a 720p video. And 11 pictures is few. Any spot seen by only one or two of them comes out rough.

It was also too heavy. A 31 MB room is a lot to ask of a phone, and of my own computer. So the home page became a much lighter gallery, and the rooms moved here, where someone who wants to walk around can choose to.

What it showed me

Show it, don't describe it. Every time words failed, a picture worked: the kitchen's far wall, the style of the bedroom, the extra window. These models copy what they see far more faithfully than they follow what they're told.

Fix the layout once, in something exact. A plan and a built room cost more at the start than asking for another picture. After that, the furniture stopped moving, and every later fault was about the paint, not the room.

Tell the model what isn't there. Left alone, it fills every blank wall. And check its work with something that can't be charmed by a nice picture: the plain render, compared one by one.

There's a trade-off I haven't solved. The room the world model invented from one picture is still the prettiest, because it paints every corner itself. The rooms that keep my layout show every gap in what I gave them.

Next question

Can a room that keeps my layout look as good as one the world model invents? The next test is the kitchen, built from the plan the same way as the bedroom, with far more pictures of the weak corners.

More experiments

Stereo video

Turning a flat AI film into 3D

I turned scenes from The Life of Lola into 3D by estimating depth and redrawing each eye. Then I measured the same method on a separate test shot, where true stereo could be rendered from a 3D layout of Lola's bedroom.

Lola Studio

Catching mistakes before they're filmed

The Life of Lola is my own series, made with AI video, and every bad take had to be paid for and made again. I started building Lola Studio to help me direct episode 1, and I logged every take along the way.

Liquid controls

Controls that melt into droplets

I wanted to try a different style of controls for Lola Studio, so I made the controls for directing a scene from a gooey material. The menu melts open into droplets and the slider handle wobbles when you let go.