ConvApparel: Why Your LLM-Based User Simulator Is Probably Lying to You

I’ve been around long enough to remember when user simulators for conversational AI were just glorified finite state machines. You’d script a few paths, pray the user didn’t deviate, and call it a day. Those days are gone. Now everyone’s rushing to use LLMs as user simulators, and on paper it sounds great — give an LLM a persona, tell it to act like a frustrated customer, and boom, you have infinite training data.

But here’s the thing: LLMs are trained to be helpful, polite, and encyclopedic. They’re terrible at being human. Google Research’s new paper on ConvApparel puts a number on exactly how bad the gap is, and it’s bigger than I expected.

The realism gap is real

The core problem is what the authors call the “realism gap.” An LLM playing a user will almost never lose patience, forget a constraint it mentioned three turns ago, or suddenly change its mind based on a mood swing. Real humans do all of that. If you train your conversational agent exclusively against these sanitized simulators, you’re building a system that works great in the lab and falls apart in production.

Think of it like flight simulators. The best ones throw in engine failures, crosswinds, and the occasional bird strike. A simulator that only runs sunny-day scenarios is worse than useless — it gives pilots false confidence. Same thing here. If your user simulator never gets annoyed, never contradicts itself, and never says “actually, forget what I said earlier,” your agent will never learn to handle those situations.

ConvApparel’s dual-agent trick

The ConvApparel team did something smart to capture the full range of human behavior. They set up a dual-agent data collection protocol where real human participants were randomly routed to either a helpful “Good” agent or an intentionally unhelpful “Bad” agent. This isn’t just about collecting positive examples — they wanted the full spectrum from satisfaction to genuine frustration. The result is a dataset that captures how humans actually behave when things go wrong.

And let me tell you, the bad agent conversations are where the gold is. People don’t just politely ask for help when the agent is being obtuse. They get sarcastic. They repeat themselves. They give up mid-conversation. They say things like “you already asked me that three times.” These are the behaviors your simulator needs to replicate.

Counterfactual validation: the killer feature

The paper introduces a concept called counterfactual validation that I think is the most important contribution here. The problem with most simulator evaluation is that you test how well the simulator reproduces behavior from its training data. But that’s circular. A simulator that just memorizes training examples is useless for testing new agent policies.

Counterfactual validation asks: how would a simulated user react to an agent that behaves completely differently from anything in the training data? Specifically, they test simulators against the “Bad” agent — a frustrating, unhelpful system that looks nothing like the helpful agents the simulator learned from. If the simulator can plausibly adapt to that out-of-distribution scenario, you know it’s actually learned something about human behavior rather than just pattern-matching.

This is harder than it sounds. Most simulators I’ve seen would just keep being polite and helpful even when the agent is being a jerk. That’s not human. Real users escalate. They leave. They complain. ConvApparel’s framework measures whether simulators do the same.

What this means for practitioners

If you’re building a conversational agent and using an LLM-based simulator for testing, here’s what I’d take from this paper:

First, your simulator is probably too nice. Run some stress tests with deliberately bad agent behavior and see how your simulator reacts. If it stays polite and constructive, you have a problem.

Second, don’t just look at surface-level metrics. The paper uses a three-pillar validation strategy: population-level statistics (do the aggregate behaviors match?), human-likeness scoring (do individual turns sound human?), and counterfactual validation (can the simulator handle novel situations?). Most teams only do the first one, if that.

Third, the ConvApparel dataset itself is now available, and it’s focused on conversational recommender systems. That’s a smart choice — CRSs are one of the most promising applications for this tech, and they’re also one of the hardest to get right because they require multi-turn reasoning, preference tracking, and handling contradictory user feedback.

I have one mild criticism: the paper focuses heavily on evaluation but doesn’t give as much guidance on how to actually fix the realism gap once you’ve measured it. Is it a prompting problem? A fine-tuning problem? A data augmentation problem? The answer is probably “all of the above,” but I’d like to see more concrete recipes for improvement.

Still, this is solid work. The field has been treating LLM-based simulators as a solved problem for too long. ConvApparel reminds us that we’re still in the early days, and that measuring the gap is the first step to closing it.