Microsoft Xbox Unveils Ambitious AI Model Designed for ‘Gameplay Ideation’

Yesterday, Microsoft Xbox unveiled a new project called Muse. This “generative AI model” is designed to help with what they describe as “gameplay ideation.” If you’re scratching your head wondering what that actually means, Microsoft explains it as the ability to generate “game visuals, controller actions, or both.” However, don’t expect it to take over the game development process anytime soon.

While its capabilities may sound impressive on paper, the practical applications of Muse are still quite limited. The model underwent extensive training on H100 GPUs, a process that took roughly a million training updates just to stretch one second of real gameplay into an additional nine seconds of simulated, engine-accurate play. The data used for training primarily came from existing multiplayer gameplay sessions.

Instead of running this on a standard PC setup, Microsoft had to utilize a hefty cluster of 100 Nvidia H100 GPUs. This setup is incredibly costly and consumes a significant amount of power, all to produce a mere nine extra seconds of gameplay at a humble resolution of 300×180 pixels.

One of Muse’s intriguing demonstrations involved creating duplicates of existing props and enemies within a game environment, maintaining their functional behavior. One can’t help but wonder, though, if the extensive expenditure on hardware and power could have been better spent utilizing traditional development tools to achieve similar goals.

The technology does manage to maintain object permanence and replicate original gameplay behavior, which is neat. However, when stacked against the already efficient video game development processes out there, it feels a bit like overkill.

While Muse could potentially evolve to achieve more compelling results in the future, right now, it sits alongside a host of other projects attempting to simulate entire games through AI. Despite its engine accuracy and maintenance of object permanence, it’s such an inefficient way to develop, test, or play a video game that it’s puzzling why anyone would choose to use this method after examining all the involved documentation.