Microsoft Copilot: a brand new AI-driven gaming experience

The model this came from

Muse is a world model: instead of running a game engine, it learns from recorded human gameplay and predicts what should happen next, generating each frame in response to the controller inputs you give it. Nothing on screen is rendered from game assets. It is all produced by the model. Microsoft Research published it in Nature in 2025, and I’ve written about that work on the Muse case study.

This project is what happened next: taking that model out of the lab and putting it in front of the public.

The problem

The research was published, but it wasn’t something anyone could play.

The model behind the Nature paper generated about one image a second. That is fast enough to study a model and far too slow to put in front of the public. A world model you have to wait for isn’t a world model you can inhabit; it’s a slideshow.

So the first question wasn’t how do we make it faster. That was the research team’s problem, and they solved it. Running alongside it was a product question: what, out of this research, could stand up in front of a global consumer audience, and how would we know if it had worked?

What I did

Six weeks is not long enough to explore options, so the scoping had to be right early. I owned that initial scoping: what we could productize out of the research, what we would count as success, and how we would describe it.

Success wasn’t a feature list. It was whether we could serve it. Every frame in this experience is generated by a model in real time, which makes it extraordinarily expensive to run. The criterion I set was infrastructure: could we stand up compute that would support people playing a generated world simultaneously, anywhere in the world. Nothing that failed that test was a product, however good it looked in a design.

That meant stripping down what I had first designed. My first pass did something only a model can do: letting people insert assets into the game, so you could see you were playing something generative rather than a copy. It didn’t survive. We were training on a new dataset and an unfamiliar kind of game at the same time, so the front-end design had to take a backseat to that technical exploration. And every feature I wanted was more inference to serve, on the exact axis where we were most constrained.

So we reprioritized. The differentiating features came out, and the bar became whether the game itself could be replicated and served at scale. A narrower promise, but one we could hold in six weeks.

Which is why the narrative mattered more than it sounds. With the features that would have signalled this is not a normal game cut, the framing had to carry it. This is a generative model, not a port of a game. Deciding in advance what we would and would not claim was as much a product decision as anything technical. Get it wrong and you ship a broken copy of Quake II instead of a genuinely new thing.

What shipped

Six weeks after the research release, an AI rendition of Quake II gameplay went live in Copilot Labs: real-time, playable with a keyboard or controller, generated frame by frame in response to your inputs. In its first week it served 220K+ successful sessions.

That number is the answer to the criterion we set. It isn’t a popularity figure. It means the compute infrastructure and the model inference behind it held up under real, worldwide demand, which was the thing we had scoped the entire experience around.

The Microsoft Research write-up tells the technical story properly: how the team went from roughly one frame a second to real time, and trained on a week of gameplay instead of seven years. It was a large joint effort across Game Intelligence, Xbox Gaming AI and the Xbox Certification Team.

Playing the model, not the game

The honest framing became the product. The launch says it plainly: this is playing the model, not playing the game. It’s an approximation learned from data, not a reproduction of Quake II. The limitations were published rather than buried: fuzzy enemies, unreliable combat, a model that forgets anything out of sight for more than about a second, and one part of one level rather than a whole game.

Several of those read as bugs and behave like features. Because the model forgets what it can’t see, you can despawn an enemy by looking at the floor, or teleport by looking at the sky and back down. You aren’t exploiting the game. You’re exploiting the model, which is the most direct demonstration of what a world model actually is that we could have put in front of people.