This is meant to be an in-depth look at how the AI makes decisions in Age of Rivals, a strategy card game. This AI is considered to be pretty strong by players, so hopefully this will be useful to anyone approaching the challenge of building a competitive AI opponent for a strategy game.
To anyone unfamiliar with Age of Rivals, a quick video overview can be found at https://www.youtube.com/watch?v=S2WhRlh2tqI.
Briefly, Age of Rivals is a card drafting game in which 2 players compete to build the best ancient civilization. Players draft 16 cards to form their city over the course of 3 rounds and then select their best 8 cards for a final showdown round. Each round also consists of a War phase during which cards attempt to knock each other out and a Scoring phase used to earn the points that ultimately determine the winner. City cards each have various stats and a special ability that can influence the outcome of the game.
Goal of the System
The initial goal of the AI was to provide a good learning experience for new players, at which point players would transition into playing multiplayer exclusively. But as we realized that there was a significant number of players who wanted to play single player, I kept improving the AI to play at a higher level.
During the course of any AoR game, there are basically two kinds of decisions a player must make:
- Which card to draft out of a possible pool of 3 or 4 cards.
- Which cards to assign any opponent damage to.
To make the first decision (what to draft), the AI scores each card across 17 different categories. For each category, a function examines the card and the current state of the game and returns a score between 0 and 1. Then the AI weights each of these results, averages them into an overall score, and drafts the card with the best overall score. Weights for the categories range from 1 (not very important) to 20 (very important).
The 17 categories are:
- How much of a discount am I getting on this card?
- Taxes (negative factor)
- How much tax do I have to pay my opponent for this card?
- How affordable is this card given how much Gold I have left and how many cards I still have to buy this round (so I won’t end up with Waste).
- How many new resources will I get with respect to how many resources I and my opponent already have and which round we are in.
- How much income will this card generate with respect to how much income I and my opponent already have and which round we are in.
- How much immediate gold will this card give me this round?
- How much culture will this card give me with respect to which round we are in?
- To what extent does this card allow me to win additional conquests?
- To what extent will this card allow me to make a dent in my opponent’s current armor?
- To what extent will this card help me deal with my opponent’s upcoming attacks?
- Field Combos
- How well does this card combo with my other cards currently in the field this round?
- How well does this card counter my opponent’s cards currently in the field this round?
- City Combos
- How well does this card combo with other cards I have drafted for my city so far?
- How well does this card counter the cards my opponent has drafted for his/her city so far?
- Draft Combos
- How well does this card combo with the 4 cards my opponent might pass to me next turn?
- How well does this card counter the 4 cards my opponent might draft this turn?
- Field Counters (negative factor)
- Does my opponent have cards in the field right now that counter this card?
- City Counters (negative factor)
- Does my opponent have cards from previous drafts that counter this card?
- Draft Counters (negative factor)
- How well do the other 3 cards I must pass to my opponent counter this card?
- How well do the 4 cards my opponent might draft counter this card?
- Special Abilities (both positive and negative)
- Special card-specific weighting to help the AI understand when to use cards with abilities that are not covered by the default combos and counters logic.
The first 10 categories are fairly straightforward. The AI looks at the current state of the game (how much culture/armor/attack/economy each player has and what round it currently is) and tries to find the card that will best benefit it.
The last 7 categories attempt to handle Abilities, which are trickier. It is assumed that the cost of each card is fair and accurately reflects its value under ordinary circumstances, so the AI is basically looking for extraordinary circumstances to take advantage of. If it has a lot of Infantry cards, then Shieldwall is a high value card since that ability represents a combo opportunity with Infantry cards.
Categories 11-16 (combos and counters) handle most card abilities, and for a long time that seemed okay enough. But it eventually became obvious that for some cards it’s harder to describe what a combo or counter is, and so category 17 was added to provide custom logic for those cards. Examples include Plaguebearer, Master Thief and The Oracle. These are cards that you should only buy under very specific circumstances, and if the AI buys one when it shouldn’t, it can ruin the illusion of intelligence completely. About 25 cards have such custom logic.
The first version of the AI probably only had about half of these categories. Whenever I embarked upon a series of AI improvements, I would simply play a game and wait for the AI to do something that seemed obviously sub-optimal, and then I would analyze its logic to figure out why it made that choice. This usually resulted in a re-write of a category, the addition of a new one, or the adjusting of weights.
Combat damage assignment follows a similar pattern, but with only 10 categories. The AI randomly assigns damage and then calculates the total value of the cards left over. It does this 1000 times and then picks the permutation that resulted in the highest total value. I probably could have done this in a more rigorous way, but in practice it seemed to work pretty well so I never improved it.
The 10 combat categories are:
- What is the scoring value of the leftover card?
- What is the overall value of the leftover card, as defined by its base cost?
- How much armor does the card have leftover after damage has been assigned?
- Waste/Ruins (negative factor)
- Is this card Waste or Ruins? (Such cards should have been sacrificed typically)
- Buffing Culture
- Is this card currently buffing the culture value of other cards?
- Is this card currently contributing resources or income?
- Post-War Ability
- Does this card have a post-war or end-of-round ability that will trigger?
- District Combos (same as above)
- Deck Combos (same as above)
- Special Abilities
- Custom logic to account for certain cards that you should or should not sacrifice under very specific circumstances. One example is Shock Troops, which should almost always be sacrificed since it’s going to ruin itself anyways (unless it received a Culture buff). Currently six cards have such custom logic.
Normal vs Hard AI
Over the past year I’ve released several sets of category weightings, including ones that favored more combat, more culture, and everything in-between. I’ve measured their various win rates and removed poorer performers over time. There is only one AI out in the wild currently.
So what’s the difference between the Normal and Hard AIs? They both use the same set of weights and the same categories of logic. But the Normal AI just chooses to ignore some of the logic some of the time, randomly. This is an attempt to simulate a human player occasionally missing something.
So how well does the AI actually perform? Over the last 6 months:
- The Normal AI wins about 40% of games against all players.
- The Normal AI wins about 37% of games against players who’ve been playing for at least a week.
- The Hard AI wins about 61% of games against all players.
- The Hard AI wins about 54% of games against players who’ve been playing for at least a month.
So both AIs become more “beatable” as players get more experience. But the Hard AI in particular has the potential to stay fairly competitive.
But of course there are some players who beat the average and crush the AI 80-90% of the time. They have probably come up with a play style that is particularly effective against the AI's logic and weights, and over time I can try to improve the AI by analyzing their games in particular.
In my opinion, the AI’s main advantages are:
- The ability to consider all these different aspects of the game. Humans can do this as well though, and the best players do.
- The ability to remember cards that were drafted in prior rounds that aren’t currently on-screen.
- The ability to ignore prior strategies and adapt to changing circumstances. Sometimes humans (like myself) try to play out a particular strategy and are too slow to adapt when it’s clear that it’s not going to work out.
The AI’s main disadvantages are:
- It still doesn’t understand the nuances of every ability perfectly, and it over-simplifies many of them.
- It makes no attempt to Guarantee cards that work well together, or that are even known to be useful cards. Most humans know over time which cards are better Guarantees, but the AI always chooses these at random.
- It has no understanding of the current “meta” or which cards are perceived to be better than others. It does not have access to any card win rate data, even its own. Humans remember which cards served them well in the past and can take advantage of that knowledge, but the AI plays every game as if it was its first.
Some next steps towards improving the AI could be to analyze games against players that have high win rates vs. the AI, teaching the AI about more abilities, teaching the AI to be more strategic with the Guaranteed Cards system, and continuing to test alternative sets of weights.
And I’ll say it one more time. The AI does NOT cheat in any way whatsoever. It has no access to extra information. It does not manipulate the random draft. It does not know which card you drafted before it makes a decision.
I hope this has been interesting, and I’m happy to answer any questions!