AI and Games is a crowdfunded YouTube series that explores research and applications of artificial intelligence in video games. You can support this work by visiting my Patreon page.
In collaboration with Ubisoft and Massive Entertainment, I present three blogs exploring the AI behind Tom Clancy's The Division 2, including excerpts from my interview with the Lead AI Programmer of the franchise, Philip Dunstan.
Part 2 of this series, which explores open-world and systemic design, can be found here.
Philip Dunstan: "One of our mantras was creating a combat puzzle for the player. And so a lot of resources and time is required from the NPC team but also from the level and game design teams to create this scalable combat puzzle, that is fun from level 1 through to 30 and continues to be fun as the players are trying to optimise their gear."
2019's Tom Clancy's The Division 2 brings the war to Washington D.C. as the United States is still caught in the grip of the Green Poison outbreak six months after the fight to reclaim New York City. Massive Entertainment returned with a larger and more ambitious title, with a variety of new gameplay features that reshaped how player interact with both friendly and enemy AI characters within the game.
In this first of a three-part series in which I interview Division 2's Lead AI Programmer Philip Dunstan, I take a look at the lessons learned from shipping and maintaining the first Division. What the teams goals were as they moved into production on the sequel and the changes laced throughout the games AI systems to build a fun and nuanced combat puzzle: keeping the player engaged from the very first encounter all the way to endgame.
The Lessons Learned
Tom Clancy's The Division - like many a live service game - was an exercise in growth as the project not only made it to store shelves in March of 2016, but also was maintained and updated for a period just shy of two years post-launch with the West Side Pier update 1.8 released in December 2017.
The core AI behaviour of The Division is built through a proprietary behaviour tree system - a topic I've previously covered in blogs on Gamasutra - with significant effort put into tools to help the design and programming teams build behaviours and address inconsistencies and bugs within them.
Philip Dunstan: "When we started to look at what we were going to do for the Division 2, we took quite a while to look back at what had worked and what hadn't worked on Division 1. What we liked and what we thought needed changing. From an AI specific point of view, that meant our core AI architecture, some of the core systems like how we control which NPCs are shooting at a time, how NPC works, NPC animation and sort of the basic locomotion system. A lot of those systems were working actually really well or with some small incremental improvements would be just perfect for the Division 2. Also the other thing we really liked from the Division 1 was that we had created a set of tools which we thought worked really well. Division 1 of course shipped with Snowdrop Engine and we own and build a lot our tools ourselves. Division was the driving game behind the Snowdrop engine. So we had created a set of tools in the engine for the way we edit and debug behaviour trees, the way we show debug drawing on the screen and historical debugging. Those systems we really liked, it gave a lot of power to our content creators - our game and level designers. And it meant as programmers we didn't have to be involved in a lot of the work that they wanted to create."
But while there's always a need for change, there's also a chance for growth and improvement. A new title - even a sequel in a franchise - is an opportunity for systems to be refreshed and iterated upon. Addressing bugs in the toolchain and re-engineering existing tools and systems to be more stable and reliable, with the navigation tools in particular an area that the team focussed upon. But the biggest changes that players would notice was in the design and structure of the enemy archetypes and factions that players would face in DC. This was a focus not just for the main campaign but more critically during endgame, where the majority of players would spend the bulk of their time.
Philip Dunstan: "We knew we wanted to solve this problem of replayability in endgame. And the endgame was one of our big design pillars for the Division 2. To make a game that is more satisfying to play at endgame. Making sure all of our NPC factions were viable at endgame, that they are all fun to play against. It didn't matter whether you were playing against the Black Tusk or the Hyenas, they would all be fun to play against. As part of that, we had to increase the faction diversity to have more differentation between the factions. So that throughout the game you felt the difference when playing one faction instead of another. So when playing endgame and you play a mission that had been invaded by the Black Tusk, then that was a very different experience than if you were playing a Hyena mission for example. And finally we wanted to make some targeted changes to some areas of the game to increase the immersion. One of the things about the Division is that we are a 'grounded in the modern world' type thing, we're not a sci-fi shooter, and I think that's what attracts players to our game. They feel like they're playing in New York or they feel like they're playing in Washington. We wanted to double-down on that by creating a gameplay experience where they felt immersed in the world and that meant having NPCs that look they're immersed in the world."
To address the challenge in diversifying the enemy factions, the team focussed on three key areas:
- Improving Core Behaviour and Design: reducing their time-to-kill, but also their reaction to player behaviour and reinforcing the core combat puzzle each enemy should present.
- Amplifying Differences Between Factions, such that they appear more distinct.
- Distinguishing Archetypes in Each Faction, providing novelty and fresh gameplay.
This would help satisfy the teams goals of not just diversity, but ensuring challenging gameplay during endgame. So let's walk through each of these in-turn and explore the big changes happening under the hood...
Improving Core Behaviour
One of the biggest tasks the AI team set themselves was to amplify the players impact on AI behaviour. Naturally in a given like the Division you interact with enemy AI primarily by shooting them. But players need to feel rewarded when their persistence and skill enable them to clear a room of enemies. Communicating an AI's decisions is incredibly important for maintaining immersion for the player and its equally important in the heat of battle as it is during more quieter moment.
In the first Division, the enemies response to being shot was not as rewarding for the player as it could be. Division 2 embellishes the stagger animations for when a character is under fire, which help give your attacks meaning, but more critically increase the rate of falter animations interrupt the AI's behaviour, given you an opportunity to re-evaluate the situation. But having the AI character reacting to being shot is one thing, it's giving them a more natural reaction that enriches the experience. Hence they focus more on dodging and ducking enemy fire, forcing players to keep track of their movements, not just shooting fish in a barrel, but constantly re-evaluating target prioritisation and maintaining and edge of uncertainty to the fight.
The last major change to the core systems, was the Time-to-Kill (TTK), the period of concentrated gunfire players need to maintain to take an enemy down. Given the pseudo-realistic setting of The Division, there is a need to balance this alongside the design of each character. The TTK of the Division 2 is - on average - much lower than the original, but the team still wanted to provide big meaty enemies that would require greater resilience and a bit of co-ordination to overcome.
Philip Dunstan: "It was reducing the time-to-kill for NPCs so you didn't feel like you had to unload a full clip into a NPC to kill it. And the ones that took longer we used the destructible armour system to give a plausibility for them requiring to be shot at for extended lengths of time."
But there were still many an additional change both aimed at improving the enemy immersion both within combat and idly in the world. Weak points based on equipment enemies had on hand was pushed further, while a new barking system reinforces the intensity of combat as characters not only shout among one another, but also at the player.
The Division 2 has three main enemy factions player face off against: the Hyenas, the Outcasts and the True Sons. As players enter Washington D.C. the Hyenas, Outcasts and True Sons have taken up refuge in varying parts of the city and maintain numerous strongholds, forcing you to play against each faction throughout the campaign. However, once the player reached level 30 and defeated the True Sons at the Capitol Building, the fourth and final faction emerges - the Black Tusk. They become the games main end-game opposition, re-capturing strongholds and establishing their authority across the map, with many of the story mission locations now providing remixed variations that now pit you against them.
Ensuring each faction was not only distinct, but still proved valuable throughout endgame was a top priority for the development team. In fact, much of this was driven by the lessons learned when building the post-launch content for the first Division:
Philip Dunstan: "That was a problem that we hadn't foreseen when we launched the Division. We had created this campaign that showed us the different factions as they get introduced. We ramped up from the Riker and Cleaner up to Last Man Battalion and as you saw that progression of the campaign, there was a progression of difficulty and complexity of the NPCs. When we got to endgame (in the Division 1) we found that the most enjoyable to play with was the Last Man Battalion: the Riker's and Cleaners - at least initially - were not able to match the high-level player in the endgame. So from the very early part of our post-launch maintenance, we were looking at what we need to do to bring those factions up to a higher level. And so all through that post launch period we were trying to bring the factions up so that they were as equally viable in endgame as the Last Man Battalion. And it was a shame because were the Cleaners were actually one of the more interesting factions to play against. And yet they weren't as good to play against in the endgame. But over time and culminating in that Resistance [1.8 west-side pier] release we were able to at least bring them up to a close enough level that they were viable together and you could balance them all together."
So with this in mind, the AI design team designed the movement, positioning and combat parameters of faction against four traits: Aggression, Organisation, Training and Tech. Each of these traits has numerous permutations and it allows for each faction to be unique in its overall design. But still prove valuable for use in end-game.
Aggressive factions such as the Hyenas and Outcasts are quick to reposition and advance towards the player, they prioritise circling the player at close range, but also have longer uninterrupted shooting patterns. Meanwhile the True Sons and Black Tusk are defensive factions: they prefer to stay at an optimal distance, use short controlled bursts of gunfire and relocate less frequently. This results in more chaotic and scrappy fights with aggressive factions, while defensive factions force a more drawn out gun battle and spread themselves out to stay at range of the player.
Next up, the True Sons and Black Tusk are organised factions - meaning combat barks between AI reflect the hierarchy and order of the unit - plus their animations show a lot of non-verbal communication. Meanwhile the Hyenas and Outcasts are unorganised - they don't communicate as effectively or really consider what other characters are doing.
The third faction design trait reflects the level of tactical and combat training a the group has been given. Training reflects the overall combat performance of a given faction, including their use of cover fire, flanking maneuvers, time taken for winding up attacks such as lobbing grenades, weapon accuracy and the animation style. The animations are really important give it communicates how quickly enemies can transition from one action to another, but the animation itself reinforces their training and effectiveness in achieving a given action. The training of a given faction is broken into three groups.
- The Hyenas are the only untrained faction - which suits given they're the first ones you come across in DC. They don't use cover fire, rarely flank, have slow wind-up and poor accuracy.
- Meanwhile the Outcasts and True Sons are Trained Factions: they fire from cover, flank more frequently, and have improved wind-up times and accuracy. Plus their animations imply they have a stronger graps of what they're doing.
- This leaves the Black Tusk as the only Highly Trained faction, with much more aggressive flanking tactics, faster wind-up times and even better gunfire accuracy.
The last trait of a given faction is their use of tech, the use of melee weapons, standard guns, improvised explosives and gadgets all the way to robotic tanks. This is the one trait where the Hyenas aren't on the lowest setting. Outcasts run on the lowest tech trait: given they use melee weapons and homemade explosives - which largely fits their tribalistic nature. Meanwhile the True Sons are medium tech: given their focus on traditional weaponry. The Hyenas are high-tech: they carry shields, use remote controlled bombs and even carry grenade launchers. Lastly the Black Tusk has a 'very high' tech trait, with the use of the drones and mini-tanks.
So with all of these traits in place, each faction provides a different experience. The focus isn't to make them clearly easier or harder to fight against - though it's clear that the likes of the Outcasts or Hyenas might be a little easier to get on top of versus the Black Tusk. But the traits provide a range of unique characteristics to each faction, providing diversity in enemy strategy without effectively making them too weak to be interesting.
Now the more seasoned Division 2 players might have listened to that list part about Faction traits and noticed an inconsistency: at the start of the game, the Hyenas attacking the White House don't fit the faction traits I just described. They fight more defensively, seem a little more organised and have a modicum of training. You're right, they don't. It also isn't the only time that a faction doesn't necessarily line-up with their selected traits.
That's because they're one of numerous 'hidden factions': special versions of a given faction that are used in very specific situations. Hidden factions are custom built by designers to provide a hidden tutorial in the games combat. Tutorial factions are designed to teach you core gameplay mechanics. Meanwhile the hidden versions of each faction are used not only the first couple of times you meet them in the open world, but also in several of the opening missions in the game. Once the opening hours of the game are completed, these hidden factions are removed from the game and you play against the true versions of each faction.
Philip Dunstan: "In fact a lot of the gameplay for the Division and Division 2 is the player up against big groups of NPCs. And each of them is underpowered compared to the player, but combined they provide a good combat puzzle."
In the my earlier blog posts on The Division, I talked about how each faction is comprised of numerous archetypes: specific enemy designs that have their own preferences or behaviours. These force the player to constantly evaluate target prioritisation and minimising their exposure to attack as different enemy types change the flow of combat.
Now the archetypes have largely stayed consistent between games, but with some new additions based on new tech:
- and Special for named bosses.
But for Division 2 an effort was made to try and make each version of the same archetype play and behave slightly differently in each faction: providing both unique weaponry and more distinct AI behaviours. If you want to consider the Rusher: which runs directly at the player and aims to attack, the same archetype is quite distinct for both the Hyenas and Outcasts: with the former using riot shields to block incoming fire, while the latter is a suicide bomber whose gas tank can be shot to eliminate them if you're fast enough. Meanwhile the sniper behaves differently for the Outcasts compared to the True Sons. The True Sons sniper is a more traditional version of that archetype: it uses a marksman rifle, has a long wind up but deals heavy impact damage. However the Outcast sniper fires explosive arrows from a bow with a more pronounced wind up time that also has an area of effect on impact. As players we recognise the archetype, but on a faction-by-faction basis we are now adapting our strategies accordingly.
Philip Dunstan: "...we're creating a PVE game, I mean the Dark Zone is PvP but it still has some PvE elements in it. We're creating a PvE game, a looter shooter where the player needs to experience an increase in power and potency in the world but they need to be able to express that by feeling like they're getting better at shooting NPCs, so we need to create this scalable challenge for the player to play against."
Achieving diversity and novelty in the AI for games such as the Division can be a real challenge, there needs to be a level of consistency such that players can improve their skills. However, your enemy AI needs to keep up with the player as their mastery of the game increases, providing new challenges, forcing you into tight spots to fight your way out of and helping maintain that balance of tension and enjoyment. The Division 2 is an impressive feat in that the programming and designs teams took the lessons learned from the first game and went back to the drawing board to design new enemy AI characters such that they would avoid making the same mistakes twice.
Plus with the release of the Warlords for New York expansion - which has players return to the island of Manhattan - we can now see these principles applied to some of the original factions, with the Rikers and Cleaners making their return to the franchise.
But building the enemy factions is but one facet of how AI is applied in The Division 2. Looking beyond the need to shoot and kill, the world of Washington D.C. is one of rebirth: a painful process as the city rebuilds itself after months of conflict. How do you create a more dynamic city full of patrolling civilians, of convoys moving between checkpoints and balancing it all without it turning into a chaotic mess? I tell you all about that next time.
Special thanks to Ubisoft for the opportunity to work with them on this project. And of course to my patrons who crowdfund the AI and Games series.