Game AI already had the problem of too tough opponents two decades ago. Nobody (OK, outside Rainbow Six players) is going to play a game where AI demolishes any player within (milli)seconds.
Skill or mastery is not what LLMs offer in-game AI though, it's creativity, originality, and agency. Players don't primarily prefer to play against other human players because they're an "easier" opponent, but rather because they're dynamic, creatively proactive and reactive, and endlessly original.
While I agree that some players like playing against other players, I can say at least for me, I've never preferred that ever in a video game. Real players can be mean, or over powered, or over skilled, or use cheats, etc... and they can be annoyingly competitive.
On the other hand, I've loved co-op games. Maybe (it's a big maybe) I'd enjoy an AI driven NPC co-op character. It assumes they can get them good enough to be fun to play with, do what you want, etc. Player: "Let's go to the castle", NPC: "See you layer I'm going to the bar". You either follow the NPC or play without it.
I think that's a play-style preference, and honestly one I share almost completely; I very much enjoy PvE over PvP experiences 9 times out of 10.
But again, that's where LLMs can thrive! Currently if I want to play anything co-operatively I have to either convince my partner or friends to pick up the game and care about it, or hope that the existing community of players is such that I'm not spending what can be a very uncomfortable time trying to relate to kids/teens or toxic adults. If all the NPCs were driven by LLMs whereby I could modulate their play-style, personality, drive/ambition I could be looking at genuinely stimulating, creative, and original "co-op" experiences from bots.