Toyota Simulated a Million Fake Humans in a City. The Real Use Case Isn't Urban Planning.

Have you ever thought you could run your own simulations without needing a supercomputer?

Toyota ran up to 1 million fake residents through a simulated city. Not scripted extras following a route, agents with a dominant need, their own beliefs, a plan that can get interrupted mid-way. The paper comes from Woven by Toyota, presented at EMNLP 2025, and the authors check their results against real 2021 Japanese census data. On daily time use, it lines up.

What stopped me wasn't the urban planning part. It was the experimental section. Pull out the needs module and the realism score drops more than if you pull the entire planning system. Pull out beliefs, same story. Pull out the recursive planning alone and almost nothing moves. That says something specific about what makes an agent believable. Not how good the plan is. Whether it has an internal state that can break that plan.

An architecture like that, at the scale of a whole city, is an urban research tool. At the scale of a business decision, a price to test, a feature to validate before you write a line of code, it's a completely different tool. I built a small version of it.

A Million Fake Humans, One Real Japanese City

CitySim is the name of the system, built by Woven by Toyota, the same team behind Woven City, the actual physical experimental town Toyota is building at the foot of Mount Fuji. The simulation runs up to 1 million LLM-driven agents at once, and it does it in 0.183 seconds per simulation step at that scale, which is the detail that made me sit up. That's not a toy demo with 50 characters wandering a grid. That's population-scale behavior modeling running fast enough you could actually iterate on it.

Each agent gets a persona (age, job, income bracket, household), a set of beliefs about the city, and a need that dominates its decisions at any given moment: hunger, energy, social contact, whatever ranks highest right now. The agents don't follow a fixed script. They plan, then get interrupted, then replan, the way actual humans do when the fridge is empty right before the meeting they were about to leave for.

The authors didn't just claim it feels realistic. They checked it against Japan's actual 2021 census data on how residents spend their day, hour by hour: work, commute, leisure, sleep. The macro pattern lines up close enough that the paper treats it as validation, not just a nice chart to show.

The Part That Actually Makes an Agent Believable

Most agent builders start from the same assumption, myself included until I read this paper: the smarter the planning, the more believable the agent. Give it a better model, a longer context window, a sharper prompt describing who it's supposed to be, and the simulation gets more human.

The ablation study says otherwise. The researchers rebuilt CitySim 4 separate times, each time stripping 1 component: the persona, the belief module, the need module, the recursive planner. Then they scored how much realism dropped compared to the full system.

Losing the need module hurt the most. Losing beliefs came next. Losing the persona description came after that. Losing the recursive planner, the part that decides what to do next and adjusts the plan, barely moved the needle at all.

I read that twice because it goes against how most of us build these things. We spend our prompting effort describing who the agent is (persona) and what it should do next (planning), and treat the internal state, the thing that can actually interrupt a plan mid-execution, as an afterthought if we bother with it at all. Turns out that's backward. A city resident with a great itinerary but no hunger, no social need, no belief that might turn out wrong, doesn't behave like a person. It behaves like a checklist with legs.

That's the founding idea worth stealing for anything smaller than a city: structure before model. A lightweight setup with a real belief, a dominant need, and a persona will out-react a much bigger model that's just told to "respond as a typical customer of type X." I tested the lazy version first, out of curiosity. It reads like a customer testimonial written by marketing. Psychology got there decades before agent builders did, and I went deeper on why psychology already solved AI agent memory if you want the longer version of that argument.

Why a City Simulator Is a Business Tool

Nothing in the CitySim paper talks about pricing pages or churn. I want to be upfront about that, this next part is my extrapolation, not something the authors measured or claimed.

But look at what the architecture actually is once you strip the "city" framing off it: a population of agents, each with a belief about the world, a need that's currently winning, and a decision to make. Swap "which subway line do I take home" for "do I pay $15 a month for this feature" and the mechanics don't change. You still need a belief (what does this persona think the product does), a dominant need (saving time, looking competent to a boss, not getting fired for picking the wrong tool), and a decision that follows from the collision of the two.

Most AI-generated customer feedback I've seen from builders reads flat, the persona is the only lever anyone pulls. "You are a busy startup founder, react to this landing page" gives you a founder-flavored yes-man. Give that same founder a belief that turns out to be wrong (they think your tool needs a credit card to try) and a need that's currently dominant (they're 20 minutes from a demo call and panicking), and the reaction changes shape entirely. It stops being a customer impression and starts being a decision under pressure, which is closer to what real buying actually is.

I Built a Small Version of This

I scaled it down from a city to a single decision, tested against a handful of synthetic personas built the same way: a belief, a dominant need, a decision to render.

Each persona gets seeded with 3 things before it ever "sees" your pitch, feature, or price. A starting belief about your product or category (accurate or wrong, that's the point). A dominant need, ranked, so the simulator knows what's currently steering the persona's attention. And a decision frame, the actual choice you're asking it to make.

Think of it less like a survey and more like a boss fight: the belief is the shield, the dominant need is the actual health bar you're trying to drain.

The output isn't a paragraph of polite feedback. It's a rendered decision plus the reasoning behind it, and I visualize the whole batch on a small isometric city grid, a building per persona, colored roads running from a central parcel (your product) out to each building based on how strongly that persona's decision leaned yes or no. It's a gimmick, sure, but a gimmick that makes 12 conflicting reactions scannable in a single glance instead of 12 paragraphs you have to read end to end.

Random aside: the isometric grid renderer took longer to get right than the actual simulation logic. Some SVG road-routing bug ate an entire Tuesday afternoon, the kind of bug that makes you question every life choice that led to a laptop and a deadline. Anyway.

I didn't build this to replace user interviews. I built it because I ship fast and most of my bad decisions happen before I've talked to a single real person, back when the idea is still cheap to kill. If you're the kind of builder who ships first and validates never, which the Vibe Coding, For Real method spends a good chunk of pages trying to fix, this sits earlier in the pipeline than that. It's a filter, not a verdict.

10 Decisions You Could Test This Way

All of these are framed as things you could run through a setup like this before committing engineering time to them, not things I've personally validated with it. Some I have tried, most I haven't gotten to yet.

• Pricing: does a persona with a "this should be free" belief react differently to $15 than $49, or does the belief dominate regardless of the number
• A new feature's first impression, before it's built, just described
• A landing page headline, tested against personas with different starting beliefs about what your category even does
• Roadmap prioritization, letting competing dominant needs (speed vs. control vs. price) argue with each other
• Churn messaging, simulating a persona whose dominant need just shifted (they got promoted, they got laid off, their team doubled)
• A support macro, checking whether it resolves the belief or just resolves the ticket
• An ad's core message, against a persona who already has a wrong belief about your competitor
• Idea validation, with the built-in guardrail that a synthetic panel skews toward whatever's already popular (more on that below)
• Onboarding flow, watching where a persona's dominant need (get value fast) collides with your actual first steps
• A head-to-head against a competitor, letting 2 persona batches argue about the same decision from different starting beliefs

The common thread: none of these need a real user yet. They need a cheap way to find out which idea is obviously dead before you spend a weekend building it anyway.

The Starter Prompt

This is the skeleton I actually use. Copy it, swap the bracketed parts, no XP required, and change a single trait between 2 personas if you want to expose a bias instead of just collecting agreement.

You are simulating a single persona for a business decision test.

PERSONA: [age, role, income bracket, one defining trait]
BELIEF: [what this persona currently believes about the product,
  category, or problem, accurate or not]
DOMINANT NEED: [the one need currently steering this persona's
  attention, ranked above all others right now]
DECISION: [the specific choice being presented]

Respond in character. State the decision first, then the
reasoning. Do not soften the decision to be polite. If the
belief and the need conflict, say which one wins and why.

The single-trait swap is the whole trick. Run the same decision through 2 personas that are identical except for a single belief (say, "thinks AI tools are a fad" vs. "already pays for 3 AI tools"), and the gap between what they decide tells you more about your pitch than either decision alone. The full version of this lives inside the prompt contracts method I use to ship, same idea of forcing structure instead of trusting a loose prompt to hold.

The Bias the Authors Themselves Flagged

The paper's own authors ran a reality check on Shibuya, comparing what their simulated agents believed about popular points of interest against actual Google Maps data. The agents systematically over-valued places that were already well-known and under-valued the niche ones. Not a small skew, a documented, measured one, the kind that shows up when you check synthetic output against ground truth instead of trusting it because it sounds coherent. It's the agent-simulation version of that Stack Overflow answer everyone upvotes because it's popular, not because it's right.

That matters more once you're using this for anything shaped like idea validation instead of pricing or messaging. A pricing test tells you a number people react to. An idea validation test tells you whether an idea deserves to exist at all, and that's exactly where a popularity-skewed panel will quietly nudge you toward whatever's already been done, because that's what's dense in the training data these models pull from.

Maybe I think the honest way to use a tool like this is as a pre-filter, not a panel. It's good at telling you an idea is dead. It's worse at confirming a good one, since "genuinely good and unfamiliar" and "just unfamiliar" look identical to a system trained on what's already popular. The paper's own authors flagged this on their own data, so I'm inclined to trust their caution over my optimism.

---

The paper documents its own blind spot, and it's the same blind spot every synthetic panel has, whether it comes from Toyota or from a scrappy solo build like mine. Agents skew toward the familiar. A panel of fake humans isn't more objective than a panel of badly recruited real ones, it just fails in a direction that's easier to ignore because a model produced it instead of a person you can push back on.

I use mine to kill bad ideas before I write a single line of code, not to convince myself a good idea actually is one. 😅

Simulate before you build. Doubt the result before you trust it.

Sources

• Bougie, A. & Watanabe, K., "CitySim: Modeling Urban Behaviors and City Dynamics with Large-Scale LLM-Driven Agent Simulation," Woven by Toyota, EMNLP 2025 Industry Track
• "Why AI in User Research Isn't Replacing Real People (Yet)," UserTesting, with Mario Callegaro

This post may contain affiliate links. If you click them, I might earn a small commission (costs you nothing, and helps me keep shipping quality articles every day for your reading pleasure).

---

Toyota's million-agent city sim reveals what actually makes an agent believable: not better planning, but internal state that can break the plan. If you're building agents for business decisions (price tests, feature validation), the Demo vs Product Checklist in the kit shows you the production guardrails that same internal complexity needs.

→ Get the welcome kit