This is the final post in our four-part series on the future of character AI. Like part 1 it was co-written with Werner Schirmer. Follow me on Twitter if you're interested in tweets about games, AI and psychology, and to stay tuned to future posts. See here for posts #1, #2 and #3.
Video games have been around for more than 60 years, and they've certainly become a mature art form. Some have amazing graphics, some can boast awesome game design, story or sound. A majority of today's high-impact games (whether defined in terms of revenue, award nominations or some other criterion) also feature non-player character (NPCs). Can we say confidently that the NPCs in these games are 'great' in the same sense as their art or design are often undeniably great? The answer depends on whether we're talking about antagonistic or collaborative NPCs.
- Antagonistic NPCs (broadly defined) only need to function well in an extremely narrow type of interaction with the player, usually combat or competition. Defeating them is often a matter of seconds or minutes, and the interaction follows precise and well-understood rules about physical engagement (aiming and shooting, throwing, melee moves etc). These rules have been figured out by game AI developers years ago, and an enormous amount of dedication and skill have gone into best-in-class combat systems, stealth systems and so on.
- Collaborative NPCs include virtual pets, side-kicks, companions, passers-by, villagers or merchants, and they are an entirely different matter. In real human societies across time and space, collaborative interaction outweighs confrontation and violence by orders of magnitude. The same is true of written fiction or cinema, with some notable exceptions (such as war or superhero movies). It should be reasonable to expect collaborative interaction to play an equally important role in video games, and to receive a corresponding amount of developer attention and business focus. But that's not the case. We at Virtual Beings have talked to hundreds of experts, gamers and even non-gamers about game AI over the years. Not a single one of them has ever told us that today's 'friendly' NPCs are fine as they are.
Take three recent examples from very different genres. In the cute world of Animal Crossing: New Horizons, NPCs interact with the player in simplistic, highly scripted ways. The looped idle animations (see below) make it obvious that this simplicity is intentional. Believably interactive NPCs are just not part of the game's value proposition. (See post #2 for more on believability.)
|Two NPCs and one player in Animal Crossing: New Horizons (2020).||Haldor the trader showing a looped animation in Valheim (2021).||An NPC in Cyberpunk 2077 (2020).|
For our second example, Valheim only seems to feature a single collaborative NPC, at least at the time of writing: Haldor the trader. Like in Animal Crossing, Haldor displays little more than basic looped animations, and so it's clear that the developers didn't prioritize making interactions with him believable. But unlike in our first example, there is a noticeable dissonance between the believable (if stylized) graphics and the mechanistic behaviors of the NPC.
As a final example, the dissonance in Cyberpunk 2077 is even larger because the realistic graphics and the open-world nature of the game create high expectations for believable interactions with the game's many collaborative NPCs. These are then thwarted in the most basic ways (lack of eye contact, basic responsiveness etc), which even lead to a significant backlash from players when the game was released.
Of course, you could say that Animal Crossing does just fine without believable NPCs, Valheim is an edge case and Cyberpunk 2077 a disappointment. But the problem remains. The seemingly intentional choice of "wholesome" games such as Animal Crossing to use cute and simplistic NPC behaviors shouldn't distract us. It's not as if Nintendo - or anyone else - has figured out how to create believable collaborative NPCs, and then chosen not to implement them.
This is in stark contrast to all other base technologies used by today's video games, notably graphics and sound. In fact, nobody today quite knows how to create believable collaborative NPCs. The goal of this post is to convince you that this is a problem. There are four reasons why collaborative NPCs need to become significantly better.
Reason 1: Players want better, friendlier NPCs
There is already a wealth of evidence showing that players want to have better, deeper and more empathic interactions with NPCs today (see post #3 for details & sources).
- They want them on Reddit. We compared positive and negative threads about NPCs on Reddit, and found that the negative ones were far more numerous. They also produced an order of magnitude more engagement, and this engagement broadly agreed with the negative assessments of the OPs.
- They want them on YouTube. The lack of product/promise-fit in today's NPCs, especially in AAA games, has resulted in rants and fail-compilations that attract billions of views in total.
- They want them on Twitter, where accounts such as Can You Pet The Dog or Wholesome Games celebrate games that offer collaborative NPCs, despite the fact that they have little to offer besides visual cuteness.
- Multiple academic studies show that players want them. Demand for games with better NPCs is large and unsatisfied, especially among players who aren't young males.
- Social neuroscience teaches us that the brain doesn't really know how to make sense of creatures that look and act like they're alive, but that also behave in ways that are impossible or abnormal for living organisms. Interacting with such creatures is 'uncanny' and therefore largely unpleasant.
Reason 2: Bad NPCs are a creative bottleneck
(Image: a beautiful world, largely free of NPCs, awaits the player in Everybody's Gone to the Rapture ).
Have you ever wondered why so much of the gameplay in many first-/third-person 3D games revolves around exploring space by walking, swimming, driving or flying around? It's not just that exploration is fun for players. From a technical point of view it's also far easier to realize than meaningful social interactions with NPCs with today's game engines.
Game development has always been the art of the (technically) possible. This prompts a question: What kinds of gaming experiences would be possible if we could already create NPCs with believable behavior and interactivity?
- Today's RPGs are a wonderful experience for fans (like us) who already enjoy the core experience of combat, world exploration, fetch quests and progression management. But that's an acquired taste. For everyone else, the lack of 'normal' (in the sense of friendly, rich, natural and sustained) social interactions in today's RPG worlds is deeply weird. We feel they could be so much more engaging for everyone if the NPCs were as great as the graphics.
- Narrative adventure games, too, could evolve to an entirely new level and perhaps even become truly interactive fiction if game AI were far more advanced. Right now such games tend to fall into two groups: those which are carefully crafted around an essentially NPC-less experience (Gone Home , What Remains of Edith Finch ) and those that let players interact with NPCs in highly constrained settings, using simple dialogue- and behavior trees (Life Is Strange [2015-2019], Detroit: Become Human ). We've played and loved both types of games, but we also think it's blatantly obvious that they're held back by current game AI.
- Virtual pet games with believable pets could focus on providing rich and authentic care experiences instead of kid-focused humor and endless minigames that are disconnected from the core gameplay. That is why we at Virtual Beings are currently creating a next-gen mobile pet game.
More examples could be given, but we hope we've made our point. Bad game AI is a constraint that game developers occasionally handle with bravura, just like early cineastes produced wonderful movies without sound, or medieval painters were able to produce beautiful paintings without an understanding of perspective. But that shouldn't fool us into thinking that the lack of these technologies was or is intrinsically desirable.
Reason 3: Believable NPCs will make gaming more inclusive
The last decade has made game developers more aware of the lack of diversity in their own industry. It's still overwhelmingly young, male, and changing at a glacial pace. While this fact is now at least widely talked about, it has created content biases that are deeply entrenched and apparently hard to get rid of. So many games on console and in the AAA space still feel like they were designed to please young men first, and everyone else second, with a heavy emphasis on conflict over collaboration.
At least part of this bias is due to technology. Game engines (Unreal, Unity, Godot, ...) have drastically lowered the barriers of entry when it comes to developing graphics, sound and gameplay. The same hasn't yet happened for game AI, and thus developers have to face the bitter fact that, say, a narrative game with believably interactive human NPCs is still science fiction.
According to a widely-cited study by Hartmann & Klimmt, this is especially unfortunate for older and more feminine audiences, who tend to prefer more collaborative, more social and more story-rich experiences than young males. This finding shouldn't come as a surprise, as it coheres with decades of media research. More inclusive art and game design won't be enough to address today's insufficient product/market-fit. Better technology needs to be part of the equation, technology that makes it easy for anyone to create believable NPCs and to give them a narrative direction that is only limited by the imagination of its authors. This is precisely the kind of tech that we have been working on for years, and we'll have more to share about it in future posts.
Reason 4: Believable NPCs are a necessary part of humanity's '3D future'
Finally, we need to mention that games and other 3D experiences aren't just becoming better-looking every year, they're also getting more interactive. This is most obvious with regards to the XR (VR & AR) revolution, which radically increases the richness of the interaction between player and 3D world. Head & eye tracking, hand & finger tracking, haptic feedback, binaural sound and other technologies allow players to change virtual worlds in ever more fine-grained ways.
Some of the richest corporations on the planet invest heavily into this 3D future, which will necessarily involve interactions with both other players and NPCs. Whether or not you believe tomorrow's reality will be 'synthetic', the sheer magnitude of these investments should make you think.
As we ourselves pointed out a few years ago, the more players can interact with inanimate virtual worlds in XR, the more the lack of believable behavior and interactivity in NPCs seems disturbing to them. This means that players who accept relatively un-interactive NPCs today (such as many fans of RPGs or adventure games) aren't likely to tolerate them tomorrow. If humanity's future is indeed filled with immersive 3D, then engaging NPCs must be a part of it. What needs to be done to create them? Stay tuned to this blog because we'll discuss this question in several upcoming posts.