AI and Games is a crowdfunded YouTube series that explores research and applications of artificial intelligence in video games.  You can support this work by visiting my Patreon page.

In collaboration with Ubisoft and Massive Entertainment, I present three blogs exploring the AI behind Tom Clancy's The Division 2, including excerpts from my interview with the Lead AI Programmer of the franchise, Philip Dunstan.

Part 1 of this series, where we discuss enemy AI design can be found here.

Meanwhile part 2, which explores open-world and systemic design can be found here.

Building a live-service game such as Tom Clancy's The Division, comes with all sorts of challenges.  Ensuring the game is stable for players in a variety of online connections, handling the different ways players move through the world and exploring interactive systems and gameplay challenges, but more critically, checking that the game plays as expected so that players aren't getting frustrated because world events don't trigger or missions don't register as complete.  While this is most certainly a challenge for quality assurance and testing teams, as the scale and complexity of these games increase, the workloads of QA teams explode in scale.  And that's where artificial intelligence can help not just create in-game functionality, but change how the game is being developed.

In this final blog on the AI of The Division 2, I'm going to take a look at the secret AI players that playtest Tom Clancy's The Division with insights from Philip Dunstan: the lead AI programmer at developers Massive Entertainment.  I'll be looking at the custom AI bots that are deployed to assess specific parts of the game and how the first games post-launch DLC changed how the game would be tested moving into Division 2.

The Testing Frameworks

Tom Clancy's The Division has not one, but two types of bots that are used to help test the games of the franchise: the server bots and the client bots.  The Server bots - as the name might suggest - run natively on the server and don't interface with the game like a player would.  As I'll explain in a minute, these bots behave completely different from real players and are designed to stress-test the limits of the Division servers.  Meanwhile the client bots run as if they're playing a build of the game client-side.  They assume control of the game instead of the player, adhering to all the same rules as a regular player, to a point that all the in-game systems genuinely believe that this player character is being controlled by a human.  They don't have any special modifications that allow them to manipulate or cheat the game and are built to run on any platform, be it PC, Xbox or Playstation.  Their job is to test how the game actually works for players: testing the main story missions, wandering the open world and gathering all sorts of performance stats along the way that help give the developers a stronger understanding of how the game will perform for players when they log into  Washington.

The demand for these types of tools is ever increasing.  As in-game worlds in live-service games continue to increase the number of potential bugs explodes exponentially.  Even if you consider both Division games, it's not just the map of Washington DC being larger than Manhattan, but each update to both games not only introduces new content - which might have bugs in it - but it also can change or impact a lot of the existing content in the game - meaning even more bugs because you broke something that was already working.  This is only made worse by the reality live service games need to be updated fairly frequently to maintain player engagement and these updates need to work, so the word of mouth continues to be strong.  This is a problem that exceeds the capabilities of human testers: as more content is being built and existing content modified, not only does quality control need to be maintained on all the new content, but everything else that already exists in the game.  This is thousands upon thousands of play hours and is increasingly difficult to balance.  And sometimes, the requirements of testing exceed the number of available staff who can even sit down and play the game...

Server Bots

As mentioned in a previous blog, the original Tom Clancy's The Division runs with what is known as a Server Bot: it's an AI player that logs into a Division server and plays around in the game.  This is being used to test whether or not the games systems are operated as expected.  As Philip explained, while the develop team really benefit from this, the actual AI they built was really simple and, well... it cheats a lot.

Client Bots

While the server bots were conceived from the very beginning, the client bots are a different story altogether and emerged from an interesting problem during the development of the first Division.  But not at launch, rather with the second DLC update for the game The Underground.

The Underground opens up a whole new game mode in the Division accessible from the basement of your base of operations: the James A Farley post office building across the street from Madison Square Garden.  In the underground players would complete procedurally generated missions comprised of different enemy factions hiding out in the tunnels underneath New York.  And this is where it introduces a new problem: unlike the rest of the Division, if a mission is going to be procedurally generated how do you test each possible permutation to know it's going to work?

The Client bots were headed up by one Massive's partner studios working on the Division: Ubisoft Reflections based in the UK.  As mentioned earlier, the team opted for the more challenging task of creating an entirely new system.  The AI players are not based on the existing AI archetypes, instead it's a custom built AI layer that directly controls the players inputs.  This helps keep all the development of these tools isolated from main enemy AI but as mentioned, it means every system in the game still believes that a human player is playing the game.  The system was subsequently interfaced into the debug console and tools, allowing for a variety of game actions to bypass the controller layer and be processed by the player character.  This means that just like a human, the actions it's trying to execute only work if the current game state permits it.

One of the first priorities for the bots was to the test navigation mesh coverage.  Navigation meshes are the data structure baked into a level that tells AI characters where in the world they can move around.  Without a working nav mesh, no friendly or enemy AI would be able to walk around the map.  Hence if any of it is broken this needs to be identified immediately for designers and programmers to test.  In addition, follow bots were built that allowed for AI players to follow human ones, once again checking how AI characters might be able to use the navigation mesh to move through complex environments and combat arenas.  Plus simple combat behaviours that - while they didn't pay attention to their health or avoiding any hazards - would eliminate targets simply by turning the in-game camera towards an enemies head and then pulling the trigger.

But in-time this scaled up from the more low-level tests of movement space and simple combat, to being able to take on an entire mission.  This requires a lot more complexity and interfacing with the separate mission system built into The Division's codebase given it needs to know what the objectives are at any point in time and naturally these shift throughout a given story mission.  This requires a more nuanced process, whereby the bots kill all the enemies in an area, follow the path to the objective marker, destroying specific objects if expected to and triggering any and all interactions it finds within a radius of itself.  

With the client bots successfully implemented, they could not only test hundreds of permutations of the Underground missions over a weekend, but they could also run tests overnight on all the main story missions from the first game.

Transitioning this technology from Division 1 to Division 2 still required a lot of extra work, as well as a change in workflow for both testers and the level and gameplay designers.  The progression of missions in the Division 2 is not as linear as it was in the first game, hence the bots might become confused in certain parts of the mission as to how to proceed.  So throughout development on Division 2, invisible testing markup is laced throughout each mission of the game by the level designers.  It's completely invisible to players and has zero impact on our experience of the game, but the client bots can read this mark-up as directions for continuing their work.  Mission tests are ran nightly and help ensure errors don't creep through into production as they identify mission content failing or game systems not executing as expected.  There is also a separate error type for logging when the client bot wasn't working as expected and the developers strive to ensure all three of these categories have zero failures in them at all times.

In addition, the open-world testing is used by the QA and Tech Art teams on the game to identify areas of the world where there are performance overheads and they report specific bugs once they've looked at the data.  As the client bots visit all playable space within the world, they identify areas of the world they get stuck in and cannot return from as well as framerate drops that could be due to high poly counts or issues with textures or particle systems in the environment.Lastly, there is also the AI Gym: which is dedicated to testing both the bots functionality itself, but also core gameplay mechanics should changes be made.

There are still limitations to what these systems can do, largely in part due to the complexities of the game world that would seem intuitive to a player, but the client bot might need some extra hand holding.  And of course despite this big push into automation, there's still a lot of value gained from having people sit down and play the game as well.

Closing

As games continue to become larger and more complex, there is a real need to automate critical aspects of development.  Be it testing frameworks, batch processes of art pipelines, animation controllers, design tools and more, all of it serves the needs of the development team.  Allowing for programmers, artists and designers to focus their efforts on delivering the best game they can. Artificial Intelligence is slowly changing the way in which video games are built and is being applied in ways that players would never really think about.  The real achievement is by employing AI in these new and pragmatic ways, it helps keeps the problems that can emerge in game development manageable.  It keep projects on track and in budget.  But it's important that players understand the challenges of how games are made: be it the Division 2, other Ubisoft projects or the industry as a whole.

Special thanks to Ubisoft for the opportunity to work with them on this project.  And of course to my patrons who crowdfund the AI and Games series.