Daily news, dev blogs, and stories from Game Developer straight to your inbox
Sponsored By
Indie AI Programming: From behaviour trees to utility AI
A modern strategic game demands a complex and unique enemy AI. This blog article provides examples of monster archetypes, their implementation and explains why we moved from behaviour trees to utility AI.
8 Min Read
Welcome to an insight into the AI development on Rise of the Elders: Cthulhu, a tactical RPG inspired by the work of H.P. Lovecraft. The game puts you in control over a group of Investigators who are facing the difficult task of preventing the awakening of one of Lovecraft’s most terrible creatures: the ancient alien god, Cthulhu.
MONSTER ARCHETYPES
One of our main goals for enemy encounters is to come up with original AI behaviours. Basic enemies will simply focus on killing your investigators. Others will focus on draining the Sanity of your investigators to slowly drive them mad. Finally, some creatures will be so powerful that you will mainly run away from them unless you’re prepared to cast blasphemous spells and sacrifice your sanity.
The Creeper has the ability to Leap over obstacles and will target your investigators with the least amount of health points. They are the worst nightmare of your spell casters as they can quickly progress through the map and leap over your first line of defence.
The Deep One is a “tank” enemy that is difficult to kill. This creature feeds on your investigator’s sanity and will focus on driving them mad. It has a terrifying Roar ability that will make investigators in neighbouring cells lose Sanity. The Deep One will try to move to the centre of your team to affect as many investigators as possible with its Area of Effect attack.
The Maniac is a specific type of cultist that will attempt to split your team. He will target your investigator with the least amount of Sanity, capture him with its chains and drag him away from the rest of your investigators. The captured investigator will slowly lose sanity while he is dragged away from his squad.
The Priestess is another type of cultist who specializes in the dark arts of the Mythos. He will cast various creatures whilst staying in cover from your investigators line of sight. They are easy to kill but difficult to reach.
UNDER THE HOOD
(Warning, here is where we start getting technical, this might appeal to your inner geek.)
So how does our AI work? We use a Utility AI System similar to XCOM.
But before we go into that, let’s talk about behaviour trees.
Historically, game developers have been using Behaviour Trees to model AI.
To keep it (very) simple, a behaviour tree is made of two types of nodes:
Composite: Nodes with one or more children
Sequence Composite: A node that executes all it’s Children in order
Selector Composite: A node that executes the first successful Child
Leaf: Action nodes (Actions that your AI will execute)
It will probably make more sense with an example. The following tree describes a behaviour to “walk through a Door”:
You start at the root node which is a sequence node, so you will execute all it’s children:
First you start with “Walk to the Door”
The next node is a Selector, so you will try to execute the first successful child
Try simply “opening the door”
If you can’t open it, try to “unlock” it and then “open it”
If you can’t unlock it, simply “smash the door” down
At this point you can “Walk through the door”
Finally “close the door” (assuming you have not smashed it…)
Behaviour trees have been around for a while and some games (notably the Halo series) have managed to create interesting AI behaviours with them. That being said, behaviour trees have two major drawbacks:
They are case based so easier to predict by the player
They are difficult to scale because you’ll end up having huge trees to model complex behaviours.
In recent years, Utility AI has been gaining traction in the Industry. XCOM : Enemy Unknown was one of the first AAA games to leverage this technique.
Utility AI Systems work as such:
You need to start by identifying the Actions of your AI. In a shooter, those actions could be “reloading”, “shooting a target” or “moving to cover”.
For those actions, you need to create a list of Scorers. Scorers will determine if an action is good in a specific situation. For the “Reload” action you could have those potential scorers
Ammo available: if you have one ammo left in your clip, reloading is probably a good idea. If your clip is full, then there is really no need to reload.
Behind Cover: Are you standing behind a cover? Because it’s probably safer to reload behind cover.
Distance to closest enemy: If you’re in close-combat with an enemy, reloading might not be your best option. On the other hand if you’re quite far from an enemy, then it might make more sense to reload.
Finally, you need to define an Evaluation Function that will use the Scorers to determine the value of an action in a given situation. For the reload action you could imagine the following functions:
Ammo_Available_Score() = (1 – Bullets_Left / Clip_Size) * 100
Behind_Cover_Score() = if(Is_Behind_Cover) 30 else -5
Distance_To_Enemy_Score() = The distance between you and the closest enemy
So, if you have 3 bullets left in a clip of 10, you are NOT standing behind cover but you are 30 units away from the enemy (for the sake of simplicity imagine that 1 world unit is 1 meter):
Evaluate(Reload) = (1 – 3 / 10) * 100 - 5 + 30
Evaluate(Reload) = 70 - 5 + 30
Evaluate(Reload) = 95
Note that in this particular example, the numbers of bullets left in your clip is the main driver behind the value of the Reload action. But we could tweak the function by increasing the importance of being in cover for example.
The major advantages of the Utility AI are:
Easier to model complexity: A Evaluation functions can capture many cases at once (e.g. clip < 33 % full or 33% full < clip < 66% full or clip > 66% full)
Easier to scale: To add an action, you simply have to add scorers and an evaluation function without modifying the existing actions
Less predictable: You can make the AI less predictable by randomly choosing from a list of actions whose score is above a certain threshold
AI IN RISE OF THE ELDERS
We initially started using behaviour trees in Rise of the elders. However, as the complexity of behaviours increased, we decided to transition to a utility AI system as we believed it would be easier to manage in the long run.
Let’s come back to our Deep One monster. In Rise Of The Elders, every action has an AP cost (Action Point). Moving to an adjacent cell cost 1 AP. Some spells or abilities can have a higher AP cost. By design, the Deep One has 3 AP per turn, and the Roar ability uses up 2 AP. This means the monster can only move by one cell and still be able to use the ability:
The roar ability will affect every investigator in a 2 cells radius around the Deep One. A good score for that ability is the number of investigators caught in the Area of Effect. Here is a potential evaluation function:
Evaluate(Roar) = Number_of_affected_Investigators * 25 - AP_Cost_To_reach_Cell * 10
Generally speaking, we try to have the evaluation function return something between 0 and 100. 0 being a no-go and 100 being an absolute yes (but this is an arbitrary decision).
So the AI will Search all cells it can reach in one move (all adjacent accessible cells) and determine how many investigators it could affect by triggering its ability from those cells. If the Deep One moves one cell to the left, it will hit all the investigators with its roar (since they would be in a 2 cell radius) so the Evaluation function would be:
Evaluate(Roar) = 4 * 25 - 10 = 90
If the monster was to trigger the ability from it’s current position, the function would be
Evaluate(Roar) = 2 * 25 - 10 * 0 = 40
If the monster was to trigger the ability from the cell to its right, the function would be
Evaluate(Roar) = 0 * 25 - 10 * 1 = -10
With this evaluation function, the Deep One will try to maximize the number of investigators in its Area of Field. But in practice you might want to add more scorers. For example you might decide that you’d rather target one Investigator that has only a handful of sanity points left, rather than 3 investigators that have all their Sanity. In that particular scenario, you’re not maximising the sanity loss of the investigators but focus targets that are at the brink of madness.
By tuning the evaluation functions of the monster actions, we hope to achieve an increasingly interesting AI and a highly challenging experience for our players.
If you enjoyed this article and want to learn more about our project, make sure to visit our website. We're always happy to receive questions and feedback, so feel free to reach out to us!
Read more about:
BlogsAbout the Author(s)
You May Also Like
