[How do you stop racing game AI seeming unfair, but heighten competition? Black Rock's Jimenez goes in-depth to reveal the company's AI tactics for the critically acclaimed Pure.]

Introduction

This article offers an alternative rubber band method to balance the AI behavior in racing games. It presents the concept in a chronological manner, demonstrating its evolution throughout the development of Pure, our recently released trick-based racing game.

Initially we will cover the three main systems behind the concept: skills, dynamic competition balancing and the "race script". We then move onto explaining its implementation in Pure and how we used the previously mentioned toolset to try and give the player the desired experience. Finally we will offer conclusions and suggest possible alternative uses for the system.

Rubber Band

When developing the AI in Pure, we wanted to create a system which always provided convincing, fair and interesting races for the player. We wanted to challenge the player by spreading out the field, while keeping rivals close but not punishing them too much for making mistakes. Races in video games need to be managed well to make them exciting; otherwise the player will almost always stay ahead or fall behind the pack and stay there.

Rubber banding is a system that tries to maintain the tension and thrill of the race by keeping the AI characters around the player. It does so by reducing (drastically) the velocity, cornering skills, obstacle avoidance, etc., of the AI characters in front of player, and increasing (just as drastically) the skills of the ones behind.

Usually rubber band methods rely predominately on speed changes, and so are often criticized because it's obvious when AI riders are going superhumanly fast or brain dead slow.

This method is very effective, as it keeps players surrounded for the whole race. It has an important downside: it's not fair, and that unfairness is easy to spot. It can easily break the illusion of fairness in the race. No matter how well a player does during the first 75% of the race, everything is decided by how they perform at the end. A single mistake in the last section can cost the player the whole race.

On the other hand, no matter how many mistakes the player makes at the beginning, there is still a chance of winning the race. The result: players can get frustrated and feel the competition is not fair. Given all this we rejected using rubber band.

Skills

When we originally started the project we wanted to base the performance of the AI mainly on the concept of skill. Every AI character's performance was originally based exclusively on a unique set of skills. Different aspects of the behavior of the AI will do better or worse depending on the associated skill for each ability.

For instance, the "tricks performance" skill governs how well the character performs the tricks, and how often he fails them would be dictated by what we called the "jump effectiveness" skill.

The skills are represented as a real number within the range [0..1], where 0 is the worst the character can perform in the associated category and 1 is the best.

Besides performance, skills can also be used to represent personality. For instance, the character's aggressiveness (controlling how much a character will try to take you off the track) or the probability for him to oversteer/understeer the corners. Thus, you can have skills that won't noticeably modify the performance of the AI character but nonetheless change its behavior. In this article, we will only discuss the skills that affect performance.

Dynamic Competition Balancing

We initially thought the game difficulty could be balanced by determining a skill range for the AI. For instance, we thought a range of [0.4..0.6] would be fine for normal difficulty, having the best character of the race 0.6 in all his skills and the worst character 0.4. This approach presented some problems:

We soon realized this was far too inflexible and put us even further from achieving a fair but challenging race. Therefore we decided to call these values "initial skills" and have another set that would be the actual skills to be applied, which would be based on the initial ones.

These new sets of "actual skills" (we will refer to them simply as "skills" from now on) were calculated by adding an offset (either positive or negative) to the initial skills. This offset depends on the state of the race. We called this system of modifying the skills dynamically during the race "Dynamic Competition Balancing" (we will refer to it as DCB in this article).

You could argue that this first iteration of the DCB was merely a rubber band method with some variations. We used a different name anyway because of the bad reputation of the phrase "rubber banding". We tried to (internally) avoid the prejudices associated with that term.

We initially based the DCB values on the leaderboard position of the character. Every AI character was aiming to be in the position of their index -- that is, player 0 (human player) gets the first position, player 1 the second, etc.

This system alleviated the original problems a bit, but was far from solving them. It was still too unresponsive and it didn't work; the AI characters finished in a random order. Most importantly, the human player was still racing alone a lot of the time.

We also wanted the characters to form three groups that would be distributed over the race so we decided to add a "group" concept to the DCB. That meant that we had "group leaders" that were the ones trying to finish in a fixed position in the leaderboard and the rest of their group was trying to beat the next group's leader.

We ended up having a way too complex system for the DCB that didn't work so well either. The groups usually weren't compact and more often than not, the leader and the group were distributed randomly in the leaderboard.

We could have gone on with this concept and maybe we could have achieved acceptable results. Instead we came up with a different and innovative concept (that I will explain later on) and that worked really well.

Skills... Again!

There were some issues with the skills that appeared when we implemented the DCB.

The first thing that came up was what to do when the skill goes out of the range [0..1]. We defined the skill to be applied must be in the range [0..1]. We realized the initial skill sometimes ended up falling outside that range, after applying the DCB modifier to the initial value.

To solve this, we just clamped it to the valid range, because values outside make no sense. We really couldn't have the AI performing at -20% or at 110%; if we could make them perform better or worse we simply reassign both ends of the range to match the better and worse performance we could (and wanted to) allow.

We found out that an important part of the problem we had balancing the AI and trying to keep them close to the player had to do with the skills themselves and the way they were implemented. We ended up identifying properties the skills had to have to be suitable for the job:

Race Script

The AI still wasn't performing as well as you would expect for a next-gen AAA racing game, not to mention how well we wanted it to perform. It wasn't challenging, it wasn't fun, it wasn't respecting the groups we tried to keep together, and you still had this lonely racing sensation during most of the race.

Definition

We had a meeting to brainstorm around this subject, and that is where the "race script" concept came up. It was proposed that the designers wrote a "script" for how they wanted the race to develop. It had to cover all possible scenarios, such as when the player's skill is greater than the best AI, when it is lower than the worst AI, and at specific points somewhere in between.

Races don't always follow the ideal script, in other words, as they depend on the player's performance. The script forced the designers to think about that fact and how the game should behave on those occasions. The script is only a guideline, then, not a strict, linear script to be followed.

An example of what this script would look like could be:

How could a typical race scenario really play out, based on the player's skill level, without a race script?

Using this, the programmer can focus on the standard case: the ideal situation. In the previous example, this would be the first point.

Implementation

We then had to find a way to implement the script. We came up with a method that worked very well in our case: we decided to have every AI character aiming for a point a number of meters ahead or behind the human player. If aiming ahead, this value will be positive; if behind, it will be negative.

Then, each AI character's skills were set up to change dynamically depending on the AI character's distance to their assigned point, not in relation to the player (although this target point depends directly on the player's position).

Thus, an AI character could be shifting up its skills to try to get to its target point, even though it is still ahead of the human player. The same could happen the other way around: the AI character lowering his skills to get to the point even though he is already behind the player.

The previous image shows how the skills shift according to the distance to the target point. The skill increase is proportional to the distance to the target point. There's a maximum distance (150 meters in the figure) after which the skill increase gets limited (both positively and negatively).

Since we are aiming to a point relative to a human player, we realized we could dynamically modify this point during the race. This is a key concept, as it allowed us far more control over the behavior of the AI.

Since now we are aiming at a point in the track relative to the player we can modify this point to simulate the ideal script we had written. The best way to illustrate this is using an example:

The previous image shows how a race would typically evolve. The white dot is the player, while the blue dots are the back group, the green ones the middle group, the red ones the advanced group and the orange ones the close group.

There are a number of subtle extra mechanisms used to try to adapt to the script, but the one described above is key to the whole system. Every AI character may be set up to follow a different progression in relation to the human player, but they are also constrained by their skill limits. They will only increase or decrease their initial skills so much, so if this isn't enough to get to their position they will fail, performing better or worse than "expected".

It's important to note that we don't allow the AI to increase or decrease their ability as much as they may want, but instead limit their improvement to a range determined by the difficulty level. This helps to bring on the illusion of fairness to the game.

We really wanted the player to feel they have earned their place in the race. If they have a very bad race, they will see themselves punished for that (not getting to the first group at the end of the race). They will also be rewarded (by adding some distance to the next AI character) if they have a great race.

Extra Mechanisms used in Pure

As mentioned previously, we developed a few extra mechanisms to have the AI perform as expected:

Conclusions

The algorithm presented here offers a solution to "configure" or "script" your race in a fixed, predefined way, but allows a dynamic evolution of the race that tries to match the human player's performance -- but in a subtle way, and with bounds, so that it doesn't break the sensation of fairness.

We hope that the system explained here, which is the underlying AI behaviour in Pure, was never obvious to the player and kept the rewarding sensation from working their way through the race positions.

We think this method fits pretty well for our game since the genre is arcade racing. We wanted to present the player with exciting races where they are almost always surrounded by other guys and where they could easily progress, gaining positions, and overtaking opponents.

We wanted the top positions accessible but they had to fight for those positions at the end of the race, so that it's as thrilling and as exciting as possible. So we found this method fits our Hollywood-like goals very well.

We also think the algorithms described here may be used in any other kind of race game (simulators) and should be easy to implement.

We also think it could be used for other kind of games like FPS, RTSs, RPGs, etc. For instance, in a shooter you may want to have the player reaching certain strategic points with a certain amount of energy (assuming you don't have auto-regeneration).

If the player is over that energy, then you know your AI guys must press harder, but if the player is below that zone your guys should leave him some room to recover.

Then you can put some bounds to that, so that your guys never stop fighting and never have a lucky 100% headshot rate, and you set these bounds based on the difficulty level.

On easy levels they shouldn't press very hard, no matter what the human player's energy and the strategic points should have a higher reference energy values. It should be the opposite on the hardest difficulty level.

In short, this methodology presents a set of tools or values (the skills) that determine the way the AI characters behave. Then it uses those tools to try to match the player's experience to a predefined script, dynamically modifying the values of the skills based on the player's performance.

Credits

I would like to credit the people that worked shoulder by shoulder with me developing the Pure AI and whose ideas you may find in this article. It was thanks to working together and a lot of iteration and sharing our ideas that we ended up with the successful AI system we finally had.

The article presented here is the result of one year of hard work iterating over the initial ideas and doing lots of tests to improve and evolve the existing systems. The focus tests that we had were especially important, and helped us realize what the problems behind the AI were.

I would like to thank, in no particular order, Jason Avent, Michael Benfield, Jim Callin, Iain Gilfeather, and Tom Williams for their input as well as all of the people who helped us blind-test the product throughout development, without whom we couldn't have achieved the quality level we have.