One recent trend in games, particularly in the open-world genre, focuses on personalizing game experiences through adapting the game’s rules and values to a particular player. Monolith Production’s Middle-Earth: Shadow of Mordor famously did it in 2014 with its nemesis system, through which enemy Orcs remember encounters with players so that they can remark on them or show the scars of later on in the game. More subtly, Kojima Productions’ Metal Gear Solid V (2015) adapts enemy behavior and equipment to what the player does best – for example, hitting a lot of headshots will make the enemies wear helmets all over the world.

The Orc hierarchy menu in Middle-Earth: Shadow of Mordor, giving the player information on the status of the world an its inhabitants. It shows what Orcs have been encountered or what Orc could be your next target, all while keeping the personalities of Orcs alive and personal.

Both games give the experience to players that the game is somewhat listening to input, rather than merely responding to it based on a given set of rules. While this is executed on more of a thematic level, the idea of the mechanics listening and adapting to what a specific player does is interesting. Given current technological advancements, these types of systems could very well become an addition to a broader variety of games to make them resonate more with individual players.

Machine Learning and Observant Fighters

One popular way of implementing changing rules or adaptation in games is by the use of machine learning. While games traditionally employ the aforementioned static systemic responses, machine learning can break repetitiveness and system transparency by taking the player’s input to then adapt the AI behavior or values in the game world.

As mentioned and prototyped by Ricciardi and Thill (2008) [2], fighting games can particularly benefit from such AI implementations. In particular the modes in which players fight against CPU actors are often mastered when certain rules or patterns are known to a player. However, machine learning can give the CPU actor information about the player’s patterns to then strike with different attacks that the player is not expecting. In doing so, the difficulty can be rebalanced, even for players who have mastered the game in a traditional sense.

An Evolving Game about Evolution

Personally, I became interested in adapting systems and mechanics in order to build a game about evolution, both mechanically and thematically. In the past four months I have been tackling this with a few others as a graduation project to get a Bachelor’s degree in Game Architecture and Design, and the results have been very intriguing so far.

The goal was to build a small demo in which enemies would observe the player’s input and adapt their behavior accordingly. To underline this more, the game is all about evolution, set in a world in which enemy species are gathering food, sleep and do anything to survive. The player can be seen as a specific threat for them, which is why they have to closely observe her actions to react in a particular way. The project is now called Pine, and we are developing this further as a full game.

To give a small impression of what the game looks like, here’s a short trailer:

The game world and enemy species are solidly designed to react to the player in a fun and engaging way. However, rather than employing static strategies that are selected through static state machines, their moment-to-moment actions are decided by a dynamic state machine influenced by the player’s input. Players slash, roll, shoot and block their way up a mountain, on top of which a boss is waiting to be defeated. This boss is, more than just a physical challenge, a mental opposition as it tries to predict the player’s moves based on the patterns observed throughout the demo.

While the gameplay itself is fairly simple, I discovered a ton about adapting systems and the communication thereof. How do you show players what is adapting in what way? How do you let players play naturally and still let the system recognize their ‘style’?

I outline 5 big lessons I was able to take from my past 4 months as the designer of three iterations on the aforementioned demo. Obviously, lots of smaller design decisions were made in terms of combat, but in the following paragraph I tackle some of the more high-level design concepts that became apparent upon testing.

Lessons from prototyping an adapting game system

1. Players need a broad spectrum of options and mechanics.

It may sound trivial, but my first thought was that the adaptation will and should work with the smallest possible game structure. Our very first demo (after 5 weeks) included only slashing and rolling for the player, and that was initially the final plan for the combat. However, because of this, all player data we received came back similar: there were not enough options for players to even adhere to any style. If this is indeed the case, the system will not be able to recognize or distinguish any style either, let alone respond to it.

Consequently, we added a blocking and ranged mechanic, allowing players to use a broader range of actions that could be analyzed by the system in terms of frequency, timing, distance and accuracy. As a result, data was much more versatile across all playthroughs and species could adapt to a lot more aspects of the player input. In simpler words, there were more playstyles possible.

In future iterations we will look into adding items that influence the world and combat to broaden this spectrum even more. The style of a playthrough of games like Ratchet and Clank (Insomniac, 2016) is partly based on the weapon and item choice of players, and an adapting game system could keep track of this too to make enemies aware of the player’s toolset.

2. Playstyles are heavily influenced by certain design affordances.

Somewhat to my surprise, something interesting happened regarding the feedback on the control scheme in the Pine demos. Players seemed to want a layout tailored for mechanics they were mostly utilizing, as they usually do. However, it appeared that these button layouts were more than a preference – the game was discriminating certain playstyles because of them. Players who were inclined to block a lot while engaged found themselves searching for the block button more, as that is placed further away from the thumb (B) than, for example, the roll move (X).

To solve this, remappable controls or additional pre-defined control schemes could be added to foster any player’s preferences. Although this could result in players indirectly ‘choosing’ their playstyle, it does allow for natural play and the smallest cognitive load possible in terms of performing actions.

Lastly, although strictly not at all an affordance, something interesting happened with communication through level design. I was eager to test if observation as a mechanic would work well to convey the evolution – whether players were indeed more open to observing details when a pause or breathing moment was inserted. To test this, I put in two small points of experimentation in the level design. In the first level, the mountainfoot, I made players come in from below, meaning there is no option to observe anything before arriving at the living area for the species. In another instance, the last summit level, I let players come in from a higher point where species will not notice the player until they explicitly engage.

What was instantly noticeable is that, through the level design, players seemed to observe the species’ behavior much better in the second instance. I expected players to approach the first situation more carefully, but instead they mostly rushed into the fields and engaged immediately. This moment of observation can be useful in communicating adaptation, as there is a moment of safety and rest to carefully assess the situation and perhaps see the observable traits of the species before moving on.

3. The update point and rate are highly important values that should change in accordance to the length of the experience.

The neural network systems driving the species in Pine have a learning rate, defining how fast they ‘accept’ player moves they need to remember and how fast they need to forget them. This value forms the core balancing point for species’ evolution, as it tells them when to store information about the player and when to remove old information.

For smaller demos, we discovered this learning rate was actually too high. This causes species to learn and unlearn so fast that they appear dumb and easy, as they never ‘stick’ to an established playstyle long enough. The game needs time to ‘detect’ a playstyle and players need time to adhere to a playstyle.

Iteratively, the learning rate must be set so that players can notice a change in behavior every now and then (or constantly), although this is a value that is hard to quantify. Another suggestion is to not let the species update in realtime (per organism) like in our demo, but per group or physical stage instead, so that actual distinct changes can be given and seen. This gives control over the evolution and progression as a whole and might lower the cognitive load for players.

4. Procedural visualizations as a core feedback tool need to be properly prioritized based on player interpretation.

Pine utilizes a system of procedural visualizations (to a certain degree) to communicate and emphasize the specialization of evolution for organisms. For example, enemies that develop a higher health statistic in order to cope with a more aggressive player will show that by wearing more armor. Stronger enemies will start carrying more stone weapons instead of wood.

However, I found that such a visual equipment system was often secondary to more basic observable traits such as the scale and size of an organism. Not surprising in hindsight, a good lesson nonetheless. All observable traits need to be balanced, prioritized and tested to gather the desired effects – which can only be through lots of playtesting and asking players what their impressions of an organism is.

5. The game’s adapting systems should not be the core of the experience, but an enhancement.

From a design perspective, this turned out to be a highly effective and important paradigm. Many games in the past that utilize rather unstable systems of neural networks have succumbed to their unpredictability, where the game puts out results that are unwanted or unmanageable. A common critique [3] on neural network systems in games is also how they can be exploited by players, who can force AI actors to learn something the player could use later.

However, Pine uses the neural network as an addition to an existing, designed foundation. Species cannot evolve below a certain threshold and only specialize into certain directions. The neural network is used to constantly look at player input, to change enemy behavior and stats accordingly, not to teach enemies unprecedented moves or to give them skills procedurally.

The aforementioned recent games with adapting systems seem to take on a similar approach: adaptation can be used as a way of personalizing the game, rather than make it the core. Once we realized it should not be the end goal of the development team to have a fully evolving ecology that performs in ways we cannot predict, the game became more manageable while still keeping a solid layer of personalization that could really work out in a larger game.

Weak points

Obviously, Pine has only been in development for four months, and there is a lot of ground still to cover. So far it has been interesting seeing all these players take on enemies that observe them and change their own behavior in order to survive.

At this moment, the adaptation is not working equally well for all playstyles. We notice that an enemy’s range of observation should be large to cover all ground. To give an example of this: we did not let the enemies keep track of average blocking time throughout a playthrough, meaning that defensive players were never really countered with more block-breaking attacks overall. Perhaps a simpler, more elegant design with the balance of a game like Rock, Paper, Scissors could help start out an extremely solid game design for this.

One thing I have not yet tried out but should be vital to a system like this, as was pointed out to me by another designer, is some proper A|B-testing. Does the adaptation actually add anything? Because Pine relies on the evolution as a theme, it would add something anyway (given that the ecological hierarchy changes based on the player input) – but purely from a combat perspective, it would be interesting to observe if the adaptation is noticeable indeed. A placebo demo without any of the systems, but a certain progression hardcoded in, could be used for this.

Lastly, one must be careful that the AI does not become frustratingly smart in their pattern recognition. Players want a fair and interesting AI, rather than a punishingly smart one.

Conclusions

A crucial value to consider when designing adaptation in games seems to be time. Not only does the player need time to get to know the game and subconsciously develop a playstyle, the game will need time to recognize it. Moreover, some of these adapting systems seem to work better in a larger game in which players can really invest. Time, again, might be a constraint for a lot of teams dealing with these more experimental features.

Luckily, more and more games are using adaptation under the hood. If done well, the player does not notice it very consciously. The trick seems to be to let players play naturally, as they would go through any game, and make them feel like the game is actually listening to what they do.

For Pine, I think the adaptation will work better as we progress with the project – adding certain types of weapons and items could really enforce the idea of the playstyle, meaning enemies could react differently to a player using a hammer than to a player using a longsword. I cannot wait to design the intricate systems behind enemies keeping their distance because they observe that the player has a very long weapon – and honestly, it seems like a very natural step forward in combat- and game design in general.

You can play the Pine demo soon by signing up at http://twirlbound.com/pine/testing.php. We could use all feedback!

References and sources

[1] DualPixels (2014). Shadow of Mordor screenshot. Retrieved from http://dualpixels.com/wp-content/uploads/2014/10/Middle-earth%E2%84%A2_-Shadow-of-Mordor%E2%84%A2_20141007231820.jpg

[2] Ricciardi, A., Thill, P. (2008). Adaptive AI for Fighting Games. Stanford, CA: Stanford University.

[3] Rollings, A., Ernest, A. (2003). Andrew Rollings And Ernest Adams On Game Design. New York, NY: Pearson Education.

If you want to discuss any of this, have questions or just like to chat, feel free to message/follow/email me on Twitter (@Matthijz) or matthijs[at]twirlbound.com. Thanks for reading!