Column: 'Podcast Transcript: Brian Reynolds On 'How AI Enables Designers''

Gamasutra is now providing full transcripts of selected Gamasutra podcasts featuring interviews and lectures with key game industry figures, this time with a GDC 2004 lecture from Big Huge Games head Brian Reynolds, which looks at how the game design process and how the creation of AI overlaps in today's game business. The official panel abstract read: "Whether you're a programmer who's always wanted to work on the game design or a designer who thinks there might be something to this 'programming' thing, here's your chance to talk with someone who has worked both sides of the fence. We'll focus on AI and the ways in which AI development does (or should) overlap with the game design process, drawing case studies from the presenter's experiences as Lead Designer for Rise of Nations, Alpha Centauri, and Civilization II. We'll talk about why delaying AI development 'until the design docs are final' is a wasted opportunity, and how both AI and Design benefit from simultaneous prototyping. We'll explore not only the traditional use of AI to determine goals and strategy for computer players, but also the critical role of AI in supplying 'personality' to computer-controlled characters. Perhaps most importantly we'll talk about the sometimes-unexpected ways AI techniques can be invaluable in content generation.." A full, unexpurgated transcript of the lecture follows: "Brian Reynolds: I am going to talk about some of the specific different categories of AI. The different kinds of things one can use AI to do, and how they relate to the design process - how they can enhance the design process if they are developed side by side with the design. Then I am going to try to use as much of the time as possible to talk about some case studies of this process at work, and games that I have worked on over the years, and finally I'll end with the classic do's and don'ts AI techniques for designers. Since, in theory, I am talking to a lot of programmers, I am going to quickly talk about what the priorities are for designers, because I am not going to assume that necessarily the AI person and the design person on your game are the same person - you may be trying to form a cool team with a cool AI creator and a cool design creator. These are the things that your designer is going to be thinking about. Above all, he will be thinking about maximizing interactivity. Interactivity is the thing that is special about our entertainment medium as opposed to other entertainment mediums such as the cinema, such as novels, such as music CDs. These other forms of entertainment have their own strengths and weaknesses. Maybe the best way to tell a story is a novel. Maybe the best to deliver the full-on audio/visual experience for a linear story is the cinema. But our special strength is interactivity: the ability to not put a player outside of the story but put the player inside of the story, immersing the player in the story so he is not watching the movie, he is playing the movie. He is making the key choices for the key heroic character that then affects the outcome of what happens in the story. So that brings us to the next thing designers are thinking about: how to create interesting and important choices. You don't want to make unimportant choices because then people complain about micromanagement. You don't want to make uninteresting choices where the answer is always the same thing every time. You want the choices to be exciting and tense, and each choice that you make to pull at you, and you want your game to be a back-to-back series of interesting important choices. Designers are going to find ways to introduce asymmetry and tension into the game, because that is a tool for creating more excitement for the player. An example in the game of chess is if you've got on one side a rook, and on the other side a bishop and a knight. Your material count is pretty even, but the capabilities of the pieces on the opposing side with respect to each other are very different the kinds of maneuvers that the players can conduct. The kind of tactics that they can engage in are very different and that creates a lot of excitement. In our form of entertainment, in the strategy genre, the most important example of asymmetry would be what Starcraft did to the RTS genre. Before Starcraft, players tended to have fairly symmetric capabilities with respect to their opponents in the game. Starcraft introduced the concept of forcing the player to choose one of three asymmetrical races, which got the player to commit to a whole set of unique capabilities with respect to players that had choices for the other races. That led to a lot of opportunities for exciting interesting game play in that game. Designers are looking for replayability, the ability of the game to be interesting after you play it the first time. That can extend the lifespan of your game on your player's computer, and that is a good thing, so you want your game to have ways that the story can come out in different ways depending on different strategies you try, so there can be different endings to the story depending on the action you take with your character. Designers looking for ways to create compelling multiplayer experience and solo experience. These things pull against each other. For a multiplayer experience, players don't have a very long amount of time to spend on a particular multiplayer engagement. They have no more than an hour, and these days often no more than twenty minutes that they are free to get together with their friends and pop off a quick multi player engagement. You are looking for something that is very short and very sharp, whereas on the solo side you are looking for something players want to be entertained with probably for a longer period of time - even if that longer period of time is broken into some distinct intervals. They want to sit down with a solo game play for a while and then go away and come back, and when they come back they want the story to continue from where they left off, and not set them back to the beginning. That's another tension your designer is going to be dealing with. Now I'll cover some of the uses of AI in games, the tools we have to work with in enabling designers. First of all, services which are for sample path finding, complex physics, and basically all the things that underlie the computer world and make the world be able to work. Now those have a relationship to design that is important, and obviously if your path finding sucks, nobody is going to appreciate how cool your design was. But I am going to leave those aside and talk about some of these other areas where there is an even stronger relationship between AI design, and even more opportunities to maximize that relationship. There is the classic strategy and tactics for computer players. In strategy games, we want to have a so-called computer player take the other side of the game, and kind of push against you - compete with you - so that computer player needs to be able to come up with goals: decide what to produce, how to group its units into armies, how to use those armies effectively and having that stuff work well is the key to having a good experience. Then we get into diplomacy. When we get into stories with characters, then AI can be used to create the personality for those characters, and by personality I essentially mean the illusion of a human being on the other end of the line. Someone with the illusion that they have their own agendas their own motivations that they are acting on, and the complexity and unpredictability that you would expect from dealing with another human trying to create that illusion. Finally we can use AI to actually do design. We can use it to generate content for our game. We can use it to create new levels random maps, random scenarios that extend the replayabilty of our game. So where do design and AI overlap? First of all, there is the obvious one. Any time we add a new feature as designers, we have to teach the AI, or somebody has to teach the AI how to use that feature. It would be a big waste if we implemented a bunch of cool features and then the AI couldn't actually use those features and you played the game and nothing happened. That is one situation where you get very quickly to the players complaining and saying, 'Why doesn't it ever use spies?' or what have you. If you are ever going to create a feature, you have to have an AI for it, and that means you - to some extent - want to make sure you develop a feature set that lends itself to good AI. That's not to say that no feature should go in the game if it is hard to make AI for it, but at the same time if you implement nothing but very complicated features and you never really talk early to your AI guy, then don't be surprised if, toward the end, you find out 'well we don't actually have time to do all this AI, so there is not going to be AI for all these cool features we invented,' and everybody is going to complain, and it's going to be your fault. It's very important to make sure you have a good communication between AI and design about what the feature set is going to be, and how we are going to go about creating some AI for it. Now, what AI should be used for - the goal of the AI and design team is to drive the interactivity. The thing that the designers are aiming for is a lot of really high quality interactivity, and the AI can be used to create the pressure on the player - the competitive pressure. The AI provides enemies to compete with. It provides allies to cooperate with, and it provides partners to negotiate with and conduct diplomacy with, and the AI, if it operates well, can be used to give the player the opportunity or even the requirement to make choices that are then interesting and fun to make. The AI again can be used to generate content and increase our replayability. It could add to the asymmetrical landscape. Finally, a way that design and AI should overlap is in the prototyping process by which we create the initial versions of our game, by which I am saying AI and design should be prototypes together. Now you've heard other designers talk about how to do an iterative design, iterative prototyping process. I'm not going to spend too much time on this, but it's the way that many of us believe is the best way to create rich game play. It's also the best way to create rich and complex AI. We take our early key design ideas, and, very early in the process - weeks not months, we implement them into a simple prototype with some simple code, so that we can test and play our game with our key high risk features implemented - to see if they are fun, to see if they are interesting, to see what's strong, what's weak and how they all work together. That then generates feedback which we can put back into the design part of the loop and create this continuous iterative process, where each time we play the game we then go back and revise the prototype based on that feedback. Then we implement a better prototype, play it again and so on and so forth, until we gradually get toward our final product. Prototyping AI and design together is going to improve the results not only for the one, but for the other. It's going to improve the process for everyone. For the designer, it's going to mean you get better results from your prototype earlier, because first of all, you can play without multiplayer code working and you are going to have the benefit of actually having some enemy nations and some opposing characters starting to take shape, and since that is part of the story you are creating, the more parts of that you have in your prototype being tuned, the more effective your prototype is at making your game better. Now when you prototype AI, the classic failure state when you put AI in front of some players and they play it for a while, is the AI is too dumb because of something. It actually is pretty much inevitable that is going to be the feedback - the only question is how long it takes to get there, and what the reason is once you have gotten it. So, say, the AI is too dumb because it doesn't collect any resources. Or it collects the resources, but then it doesn't use them to build units, or it builds units but all the same kind of units. It builds a different kind of unit but then sends them one by one up against my castles. It doesn't send them one by one anymore, but it groups them all together and then they all commit suicide against my castle at the same time, instead of going around the flank where I am weak. These are some examples of 'the AI is too dumb,' because these are the hurdles that you have to overcome to tune your AI. The AI is going to have lots of moving parts. It's not going to be something you can sit down at a table, write a big long spec, create it from whole cloth and sit down and implement the spec and expect it to be fun. The first time you implement any AI it's going to fail, usually very quickly, and only by then going back and revising and trying again and overcoming that hurdle will you find out what the next hurdle is - and the next hurdle, and the next hurdle beyond that. The more you have a chance to bake your AI, the more iterations you can go through, the better the final product is going to be, and the longer it is going to be before a player says the AI is too dumb because something happens, and the more esoteric the reason is going to be when you get there. Creating AI and design prototypes together allows each to draw inspiration from the other, and the new features that you add to the game not only force you to create AI to implement them, they can often provide the basis for better AI. You can actually, in the best cases - and we will talk about this in some of the case studies - come up with AI, or rather come up with game features that make the AI play better, while also improving your game play. So, on to some quick tips for prototyping AI. First of all, like I said, you have to start in about week two of your development process - do it early do it often - but the second point is something I have seen over and over with programmers that are coming into the field of AI, or the field of AI for strategy games. This is something I experienced myself when I got into doing this: strategy AI is overwhelming, and it's not a defined field where it's easy to just get out a book and look up the next thing you need to do. It is very easy to say, "Oh my gosh, this is so big I don't know what to do," and the answer is you need to start by doing something. Teach it to play one turn of your game before you worry about teaching ten turns of the game. Teach it ten turns of the game before you worry about how it is gong to play 100. As you go about that, X = RND3 really is valid prototype AI for a game with three choices to be made, and if you can't think of anything else, start with that. Obviously with that you are going to get very quickly to 'the AI is too dumb because -' milestone, but then you will have something to push against, because you will have seen it in the context of the rest of your game play. Then you will have the ideas 'well the AI is too dumb' and 'what was it that the AI could have done better? What piece of information could the AI have taken into account to do a better job in making that decision?' Then you can go back and implement a rough version of that, and your prototype will play better. Don't worry about creating final code. The classic programmer fatal flaw is to worry too much about creating clean structured perfect code from the beginning, when what's really called for here is the ultimate in just hacking it together and getting it to run as fast as possible. Later on after you have done a lot of iteration, that's when you can gradually start to introduce the idea of cleaning up the mess and getting some cool algorithms designed. The first and most important thing is to get it going, get something happening so that you have something to push against both in the programming process and the design process. The other kind of fatal mistake you can make in creating AI is - let's say, OK, so you pass step 2, you say I'm going to do something. I'm going to create some production AI so my AI will know what to build, but then you get excited about that, you work on it a lot, you go through a lot iterations, but you leave behind army AI and diplomacy AI. They are sitting over here and you haven't done any work on them, so of course it is the same problem, where it's really intimidating to figure out where you even are going to start, because you've never done a diplomacy AI before. But, if you don't start, you won't ever get anything done, and if diplomacy is one of your cool features, you are going to be sad when you get to the end and it doesn't have any time to bake. So that's no good. So, my other piece of advice is to make sure that you not only take a stab of doing AI from the beginning, but you take a stab at doing some AI for some of your cool features very early in the process, so that all of these features then have time to get the advantage of getting these cool iterations of your prototype, and they all have a chance to bake and be tuned by the time you are done. My final piece of advice is to try to make sure the AI confronts the player with choices. Always think about the fact that what the designer really wants is to find ways to stimulate the player with interesting and important choices, and to find ways to have the AI enhance that. I would like to try to use the rest if the time to talk about case studies from games I have worked on over the years. Here are five cool games. The first one is in parentheses because I actually didn't work on that one. But that game kind of defined the state of the genre when I came to work in it, and so formed the starting point for my own thought process in developing these various games. Now this is four different games over a whole lot of years and three different companies but I think it illustrates how some of the thought processes went on in my head and in the heads of the people on the teams, and I think in some ways this illustrates what the iterative prototyping process can be like when it works well. So, in the area of diplomacy and strategy AI for nations, I would like to go in and look at this case study. When I got into doing strategy games, Civilization was out and was successful, so we were working on a game called Colonization, which was kind of a spin-off game. In both of those games, you have little cities and you would have little units you would move around, and there would be enemies and friends, and you can make peace and war. But what would happen is the AI would walk up to your city with some troops and it would say, 'Hi, would you like to sign a peace treaty?' and you would say, 'Yeah, you've got some troops. I will sign a peace treaty,' and then it wouldn't leave. It would actually fortify those troops right at the gates of your city. At the time I thought that was a really cool feature, but that was the first point of 'the AI is too dumb,' because players found this very unintuitive. They felt like if I make peace, then he ought to get out of my territory. What do you mean by territory? I don't know, but definitely not right next to my city, so we said, "OK, we will think about that." Conversely, the AI had no concept of you being in its territory. You could just wander your troops in amongst its cities. Check things out, get all ready, bring twenty tanks in. It wouldn't notice - it would say, 'Hey, I made a peace treaty; with him everything is fine.' [laughter] We tried an ad hoc rule called the 'zone of control' rule, which made it inconvenient to go about doing that, but it didn't make it impossible. It could still work - tanks back in, and then suddenly, when the moment is perfect, you could declare war and the AI doesn't have any cities any more, and doesn't get a turn. That was another 'AI is too dumb because -' situation. The AI didn't understand the concept of territory. It didn't detect threats very well when you approached its cities. Finally, the AI was not aware of your interactions with other computer nations. For example, if you had been repeatedly and gregariously treacherous to the Greeks - you make a treaty, you break a treaty, you make a treaty, you break a treaty and finally you wipe them out - over time, the Greeks would catch on and wouldn't deal with you much. Then, when you finally finished them off and you moved over to the Romans, the Romans would say, 'Oh cool, new guy, clean slate let's make friends.' So the process would repeat as you one by one knocked off each of your seven opponents. That obviously wasn't so good and it wasn't so logical. It seemed like, 'shouldn't like Alexander have shared some information with Caesar about this terrible thing that is going on?' These other guys should be talking to each other too, shouldn't they? Because this is all a community of nations and rulers. So when we were talking about Civilization II, and how to create that game we thought, 'let's make some adjustments based on the feedback we had gotten, and figure out how we could make some of these things better.' One of the points I want you to get as we talk through some of these, is not only did we make the design better, the game was deeper and more fun. More importantly, almost all the design changes themselves made the AI better, and the AI changes made the design better. The new things we were able to do with AI made the game features that we added seem much more cool and significant then they otherwise would have. What we added was a two square exclusion zone around the city. Now you can't walk right up to the city with someone you are at peace with, because there is a two square exclusion zone. If you sign a peace treaty and you have forces next to Rome, then it will say, 'hey, you are in my territory - get out,' and you have to get out. Likewise, if the AI has signed a peace treaty with you, he now has a way of detecting, 'well, OK, this belongs to him, I'd better not go there, because I don't want to make him mad - I'm supposed to be at peace with him.' That works pretty well - the AI can now avoid your cities a little bit, and make sure you don't get too near it. Now, this was effective only up to a point. Gaming the system trick, you know now to stay two squares out, so now you pile up a whole bunch of tanks two squares away, and then you rush in all at once and you're sad. Even more important, imagine the AI has a whole continent peaceful to itself. As long as you avoid being within two squares of one of his cities, you can unload your entire army on his continent and go driving around and he will say, 'oh, OK, that is fine, that is not my territory so everything is OK' - again, a counter-intuitive situation. Conversely, your AI ally - if you do not tessellate your cities and exclusions together perfectly - the AI who is supposedly your ally will wander a little settler down to the middle and help himself to a brand new city right in the middle of your island. Now he is using your production sites, and everything is sad. So, we've got some improvements here, but some flaws as well. We did also ensure to introduce the concept of reputation. This is the concept of the Greeks actually talking to the Romans about the terrible genocide that is going on in their part of the world so that if you, again, repeatedly and flagrantly abuse one nation, word will get out and you will find the Romans a little bit more prepared when you come to your supposed 'next victim.' By introducing this exclusion zone, we basically not only introduced a rule that kept the human player from doing something, we introduced a piece of information that the AI could look at and take into account. It knew now how to detect - at least to a certain extent - if units were to close, it knew how to keep its unit from getting too close, and it knew how to keep track of what has been going on with treacherous human players in the first thousand years of the game. The next game I was involved in was Alpha Centauri. This was also a turn-based game, and for Alpha Centauri we thought we would introduce a concept of national territory on a broader basis. We thought back to a long time ago, where in the very first Civilization there was a feature called "replay," where at the end of the game you could see a quick animation of everything that happened in all the many thousands of years of the game, and cities would rise and fall, and as they did it would look like national borders shifted. There was a very simple algorithm just for each square on the map. Which city is it nearest to? That's the color that we would make that square, and it made a cool little animation. For Alpha Centauri we thought, 'hey, let's use an algorithm like that, and make national territory show up all across the map in the game.' So now, based on where the cities are on the map, every square in the game was subject to being in his territory, my territory, their territory or sometimes neutral territory. This was, as it turned out, really effective, really cool, really successful, partially because it was a very visual representation. It suddenly made territory very intuitive to both players and the AI that they were playing against. It was almost like a way for the computer and the player to agree on whose territory it is. The AI does not cross into your territory when you are at peace, and it expects you to not cross into its territory at peace, because the AI can now tell whose territory a unit is in. It can tell if your unit is in his territory. It can tell if his unit is about to go into your territory, and it can avoid doing that. When the AI does those two things, it naturally has the effect of making the AI seem smarter. We did not actually add a lot of AI to make it do that, to make it seem smarter. What we did was carefully thought about the relationship of the AI and the feature, and introduced this feature that made the AI better. The AI has more and better information about how it can make the best decision. This had the unintended, but happy, consequence of making the AI better in other ways. Up until now, in the previous games, the way one of the key algorithms made diplomacy decisions had been, 'well' is the guy on the same continent with me? If he is, he is probably my rival and I should start attacking him. If he is over on some other continent, then maybe I should make peace with him and trade with him while I finish off my close-by enemy, and then I will go over there and get him later.' Now we have a new subtlety in that we can say, 'so, he is on my continent. If he is adjacent to me, if we share a common border, then he is close-by and my rival. If he is on my continent, but we do not share a common border, that probably means he is on the other side of my near-by rival. He is the enemy of my enemy, and so should be my friend, and I should actually seek that guy out and make an alliance with him.' That again is a very simple piece of information - it was very easy to collect that information - and suddenly the AI played a whole lot better, and it was all because we introduced this game feature that made that information available. The AI can tell whose territory units are in, it can avoid making hostile acts when it is suppose to be a peace, it can detect adjacent nations and threats, and it can all in all do diplomacy in a much better way. All right next game - Rise Of Nations. Rise of Nations is a real time strategy game, and when we sat down to say we want to do this, we knew we wanted to do a real time strategy game. There are a lot of cool real time strategy games out there, so what was our market niche going to be? What are we good at? What would we be good at doing in a game? Well, one thing we felt we were good at was game play innovation. We felt like we knew how to take new ideas and make them play well in an existing genre. So, we thought, we have been making these turn based games for years and years and years. What if we take some of the ideas that work in turn based games and try them out in real time games? So, we did that. We had a whole bunch of ideas, and some of them worked better than others, of course. One of them that worked really well was national borders. Real time games have a whole different set of design issues. First of all, there is a lot of time pressure. The clock is always running, and the player has to make decisions very quickly, so therefore the interface needs to be very intuitive. Your game effects need to be very graphical. They need to appear in the world in ways that the player can understand very quickly. National borders, as it turned out, were very visual. Right in the world, you could see the thick blue line that said, 'this is mine, that is yours, stay away.' Players immediately understood that and they were even more successful because it actually spoke to an issue that had actually been going on in real time strategy games up until then, which is the issue of, 'if there are no rules about where I can build a building, then somebody can sneak in and build a giant fortified gun thing right in the middle of my base, or they could sneak around and hide a barracks back behind my town.' Guys come in, and terrible things happen, and obviously there is a lot exciting game play - some think - in doing that, but there was also a lot of complaining about'this doesn't seem fair, this doesn't seem intuitive,' kind of reminiscent for me of some of the early comments we had about the guys fortifying right next to your city in Civilization. National borders were an easy way to instantly fix this problem. Now there is a big line - you can build on this side, they can build on that side, and then as the political game develops, territories can change hands, cities can rise and fall and the places where people can build can change. That worked really well, but where we went from there got really exciting because we discovered as we were testing that testers started to talk about this strategy called 'border pushing,' where they would - very quickly, at the beginning of the game - try to build a whole bunch of little tiny cities and ports and things and try to encompass as much territory on the map as possible, and then spend the rest of the game desperately trying to hold onto it in order to deny that territory, and the resources in that territory, to the other player. So this was exciting to us that this new strategy was starting to develop and starting to be effective. That, of course, led to more AI. We needed to create some AI to understand this. Obviously, the initial version of the AI fell down pretty hard when you tried to border push it, and we needed to make AI that could understand the border push directed against itself, and could actually play a border push game and try to do that to a player. As we moved forward, the borders inspired additional AI and design ideas - this was an interplay back and forth. We felt like we needed to have some features that made it really clear. We wanted to bring out the effect of national borders. National borders turned out to let us do, for the first time, an intuitive version of supply lines where we made this slow, vulnerable, and expensive unit called the supply wagon that had to go with you into enemy territory. When you were behind enemy lines you needed to have it around, otherwise you would succumb to attrition damage, and that underlined for the player the significance of being prepared before you cross the border. Now obviously the first AI versions of this would just send their troops wandering around everywhere, like AI had always done in all the ages past, and very quickly succumb to attrition and was seen and heard from no more. We needed to create some AI now to deal with the concept of attrition - to wait until there was a supply wagon, accumulate some units with it, put those together into an army, and then send the army to do something. Because we had to create that AI, it then inspired us to think of other things we could do with the army, and we created this abstract concept of an army which could then understand the difference between being an offensive army operating in enemy territory, a defensive army operating in friendly territory, and the AI could maybe even more importantly understand the concept of having multiple armies. It could keep track of when enough troops had been put into an army and was ready to go do this or that task. It could dispatch armies to particular tasks and goals that it set for itself and operate multiple armies at once, without getting confused about which unit was in which army. Again, even moving into a completely different genre - real time strategy - we found that there was a lot of room to take these cool game play ideas and work together between the game play feature and the AI to create a much more exciting and interesting experience all around. My next topic is personality. I talked briefly about what personality is - trying to create the illusion that there is a person on the other end of the line: that there are motivations, agendas and that there is some complexity and some unpredictability. The way to go about creating personality is to start with some very simple and dramatic polar categories: rush versus boom, guns versus butter, friendly versus antagonistic. Take these things and try to create some really black and white variables that are kind of either on or off, maybe a neutral position in the middle, but no more than one neutral position in the middle, because it's difficult for both players and computers to make distinctions on large linear slopes of gray scale. It is much easier for a player to distinguish between on and off, or left, right and center. It's also much easier for AI programmers to generate the effects that you want from those kinds of simple decisions. Even though those decisions seem simple, when you put them together with all the other variables - when you put rush/boom together with guns/butter together with friendly/enemy - you then get the complexity you are seeking in the interaction of these dials. As with design and all other parts of AI that have lots of little moving wheels, that means it is going to be very cantankerous and require a lot of tuning. So, again, start very early, iterate often and do a lot of prototyping work to do that. Now my case for personality is going to be Alpha Centauri. Specifically, on the relationship between the social engineering we did in that game and the AI for the computer diplomacy. Now since Alpha Centauri was a game about the near-future, we wanted to provide you with a prefab democracy and a prefab monarchy communism, each with different strengths and weaknesses. It was all kind of pre-designed - we wanted to let you sort of create your own government just like you could create your own units. We had you choose from several menus of social policies, and these policies then mixed together to create effects in different areas of the game world. Some of the effects were good, some were bad, and the net effect of all the policies you chose was effectively the government you created. That was all well and good as a game feature, but the thing that was really special about Alpha Centauri, I think, was the way that we then drew some lines between the social engineering government side, and the characters we were creating for the story, and the diplomacy AI we created for those characters. In the historical games we had created previously, everybody understood in a rough sort of way who Caesar sort of was, and who Napoleon sort of was, and they had some ideas about what would it be like to negotiate with Napoleon, and we could kind of play on those preconceived ideas to bring out the characters. It was easy to bring out a little whiff of personality from historical characters. It was a lot harder in the case of a fictional game, because nobody knew who these weird people were. We had to tell the story, and ideally we wanted the player to get to be involved in telling the story, and making the choices and kind of live the story of finding out who these characters were. What we did was we took these different leaders and we gave each of them an established agenda - not with respect to the policies you choose, but with respect to the effects and the world that the policies can generate. For example Lady Deirdre is the one that cares about the environment of the planet, so that's her agenda, and if the social policies you pick have the net result of being kind of a plus in the 'environment of planet' column, then she takes note of that and she likes that, and she comments on your policies and says, 'Hey, you've got pretty good policies. I actually am more favorably disposed to you.' She would be more favorably disposed to you, and more likely to treat you well - to form an alliance with you and to stay true to that alliance. Now, at the same time, if you did the other thing - if you made policies that were a net minus for the environment, then she would criticize you and suggest that maybe you might want to do something else, and when you persisted, she would be less and less favorably disposed towards you - less likely to stay in an alliance with you, less likely to make peace with you and more likely to stab you in the back when the first convenient moment arose. So now suddenly diplomacy is not simply black and white. It is not simply about the balance of power between nations. In the old days before this, there was simply, 'Is he bigger? Then I ought to buy him off. Is he smaller? I ought to attack him. Is he adjacent to me? I ought to attack him. Is he far away? I ought to make peace with him. Is he in first place? Well, I'd better attack him to keep him out of first place. Is he way behind? Well, then I should make him my toady to kind of help me along. Am I way behind? Maybe I should attach myself to one of the leaders, and try to get into his faction and get ahead that way.' Those were the decisions. That was the data we had available before for our AI to make the decisions. It made some pretty cool AI, but kind of one-dimensional. Now there is a second dimension of diplomatic AI that isn't just about who is strong and who is weak. It's about who conforms to whose agenda, and that created some really interesting, different, mixed-up combinations, where maybe the person that it made sense to ally with from a balance of power point of view was different from the social policy you wanted to pursue, and because we had different faction leaders, there were like seven or eight of these guys that all had different agendas. There was no way you could satisfy everyone, so now we had found a tool to introduce some asymmetry into the diplomatic game. We could guarantee that if you made these people happy, those people would be unhappy and create a really rich and complex environment for making choices that kind of tore at you between different things you wanted to do. That, I think, resulted in the strongest feature of Alpha Centauri - my favorite feature of that game - the diplomatic AI and its interaction with the social policies that you picked. I'm kind of running out of time, so I'm going to power through some of this content generation stuff. Content generation is, briefly, the use of AI to actually do design. I think it's a very useful tool and often neglected in game development. The things to remember with content generation is that what you are trying to do is create more game play. You are trying to get the AI to fulfill those tenets of what designers want to do. You need to create AI algorithms that can generate asymmetrical situations that are nonetheless balanced and fair. What you are trying to do is sort of fill up the world with fun toys, create a lot of variety for the player to encounter and create situations that force the player to commit themselves into different asymmetrical situations with respect to other characters and other players. What I end up with are AI and design: the do's and don'ts. Let's start with the don'ts. Don't wait until the design document is complete and the design is finalized before you start doing AI. You need to start AI at the very beginning of the process, and develop it alongside the design. Don't create either AI or design in the vacuum. Talk a lot, talk early, work on these things together. Figure out which features you can have and still be able to have good AI for them. Figure out which features will actually make the AI play better. Figure out how AI can bring out the best in the features that you have designed in the game. Don't be afraid to get started on AI. Get started early. Get something in. Do something, even if simple, then you have something to push against as you go down the process of development. Don't get bogged down in complexity. Don't work on solely your production AI to the exclusion of all the other kinds of AI. Make sure you start with something for every one of your key features, and give them all the benefit of all those iterations. Don't spend too much time up-front on algorithms and clean code. Let that follow naturally as you continue to iterate, and in the later iterations, once you've found where the fun is in your game, once you've found what the key tasks are for the AI, then you know where you are going and can work on the cool algorithms. Do start early in prototype. Do form a close relationship with AI and design. Do AI and AI design simultaneously. Do aim for crisp, simple game effects from both the features you want and the components of your AI. They are easier to test. They are easier to develop and they are easier for the player to recognize which means they are easier for player to appreciate. Do teach the AI to use all your newly designed features. Don't waste features on not having AI. Do use your AI to generate open-ended content for your game, new content, cool content. Finally, most importantly, use your AI as a tool for achieving the principle design goal, which is increasing the interactivity and the quality of the interactivity in the game. Thank you very much. [applause] Audience: The question is about agendas, to what extent do they make the AIs dumber? Brian: To what extent do having the agendas make the AIs seem dumber? Because it gave the AI more to have to think about, and it failed, I guess. And I would say that it really didn't. The overall effect of having the agendas as far as I could tell, and certainly according to the reaction we got from fans and players, seemed to be entirely positive that having the agendas made the AI seem smarter. The agendas seemed like part of the personality, a kind of a deep, sometimes religious commitment on the part of one of the characters to a particular thing, which then led them to make that decision. It didn't actually make the character seem dumber. Audience: When do you take the actions of the AIs and create different strategies? Do you, when you build a game, create the ability to play the game really fast, turn off all the graphics and sound and have it just jam, and then have the AI play each other? Brian: We do. Yeah, we actually have an automated system where you can have all the AIs play against each other. And I find that I've done that in almost every game I've developed that has been a strategy game, probably every that was a strategy game, and I've found that it actually is very effective both for testing AI and for kind of learning some of the implications of features that you can create. Audience: How did you know when you were done with the process? How do you know when runs right? Brian: You're never really done. [laughter] You're done when you run out of time. The idea is to start early in the process and iterate as many times as possible. It's always going to be possible to go on and on and on and make the AI better. What you have to do is use the time you have effectively to make at as good as possible. Now obviously, you should have an idea in your head of how much time you have left, how much time you have to begin with, and where you maybe ought to be at any particular point with respect to iterating on it. I mean, you do want to make sure that you kind of converge on something that's polished at the end. But there's really no limit to how much you can do. I would just encourage you to start as early as possible so that you get the maximum amount of tuning, because you're going to get much more bang for your buck out of the things that you design based on feedback from iterations, your responses to the feedback, than you are going to get from anything that you create from whole cloth out of your head on a piece of paper in the beginning in a spec. So the real value is in doing the iterations and the process as you go along. Audience: It seems that the point of this talk is that you should change your game design somewhat to reflect what the AI is doing and so forth. Would that put you in a position where the game - to fit the AI better - is going in a direction where you didn't originally want it to go? Brian: The question is, since my thesis is that sometimes you should change the design of your game to reflect where the AI is going, have I ever had an experience where a change I needed to make based on the AI led the game in a direction where I hadn't wanted to go before? I guess the first thing I was going to say is that there's never been a situation where the AI forced me to go somewhere I didn't want to go. AI issues, or more likely, inspirations based on cool emergent behaviors that the AI has started to produce, has inspired me to take the game in new directions I never expected to go, or to get to places I never expected to get to. That's absolutely the core theme of my little 'national borders over the years' case study, that never at the beginning did any of us conceive we would have gotten as far as we've gotten, and get so much mileage out of that. And particularly, it was because of the AI implications and the amazing synergy between that feature and the AI that we felt like we got as far as we did. Audience: I'm interested in the implementation of how you got to these ideas. It sounds to me like you created a bunch of procedural rules, like an expert system, to be able to get the AI to think in a strategic kind of way. Are there any other technologies that you used to implement these ideas? It seems to me that one of the primary concerns is flexibility and extendibility as you iterate and iterate and iterate. Brian: The question is, what are some of the different technologies that we use to create AI, and what are the things we're looking for in a technology to create AI? I think the key thing we look for in a technology is something that's very simple and quick to implement in the prototype. Later on, we will look for more complex algorithms, or we'll introduce an algorithm in a place once we understand where we're going. But the key at the beginning is to try to figure out where we're going. We go into the process knowing that we don't know where we're going, that goal one, and the reason for doing the prototype is to find out where we need to go. So yeah, the early stuff is a very basic expert systems stuff. In fact, I showed you the example where my expert system consists of x equals rnd(3). That's how you get started, because getting started is the key thing. The more complicated and more academic algorithms can come later once you find out where you're going. Audience: I was thinking, when you have talked about AI, you have talked mostly about how programmers implementing the AI for a game that somebody else has designed, how they have to interact with the designers of the game. Brian: Right. Audience: You have talked about designers in terms of the guys that create the features for the game. Do you conceive that the figure of the designer also designs the AI? Do you know what I'm... Brian: I understand what you're saying. I've talked about AI and designers as if they were separate people. And I've talked about designers as people that design the features of the game. But have I considered the idea of a designer that designs the AI for the game? The interesting comment for me is what you describe has been very close to my own experience of creating these games, because what I did on a lot of these games was work on both the AI and the design. So I was sort of the designer of the features and the designer of the AI. Some of these games took place back in the day when you could create a game with only one programmer on the game, and I was lucky enough to be the programmer. Later on, we evolved into a situation where we have a lot more people, a lot more designers, and a lot more people working on AI. I do certainly consider, properly speaking, the job of AI lead, of whoever's creating AI and in charge of AI, to be very much a creative job - a job bordering over on the design side of things, a kind of overlapping field. I think of design and programming as, properly speaking, very much overlapping fields. So your question leads me to where I already am. Yes, there's a lot of creativity in designing AI." [For Gamasutra readers interested in listening to the podcasts as they are recorded, you can subscribe to the Gamasutra podcasts by clicking this link for iTunes. You can manually subscribe to our feed in your favorite RSS reader that supports enclosures by using this URL: http://feeds.feedburner.com/GDCRadio. Transcripts of older and more recent podcasts will now be supplied weekly.]