As production of "Pavel Piezo - Trip to the Kite Festival" draws to a close later this year I reviewed the material I collected for the Postmortem and found it too much and too diverse to put in one huge article. So, I identified the topics that can stand very well on their own, which were not limited to this specific game or production, and decided to write three short(er) posts in advance to the Postmortem.

[This post is cross posted from gamedev.net]

The Leitner system

You may know the "Leitner system" from your days at school, university or pretty much any other situation where you tried to get new knowledge in your head. Using the Leitner system is often called "using flashcards" since those two go together very well, but actually that means “employing the Leitner system with the use of flashcards”. Wikipedia explains:
"The Leitner system is a widely used method to efficiently use flashcards that was proposed by the German science journalist Sebastian Leitner in the 1970s. It is a simple implementation of the principle of spaced repetition, where cards are reviewed at increasing interval.
...
In this method flashcards are sorted into groups according to how well you know each one in the Leitner's learning box. This is how it works: you try to recall the solution written on a flashcard. If you succeed, you send the card to the next group. But if you fail, you send it back to the previous group. Each succeeding group has a longer period of time before you are required to revisit the cards."

It is a very good method of learning vocabulary or any other information for us humans and I will show how we can employ this method within computer games to efficiently track a player's behavior and learning progress within a game system. Using that information, a game can react and tune any dynamic gameplay, content exposition or usage of game mechanics that we, as designers, choose. From here on I assume that you either know how the Leitner system works or at least had a good look at the graphics and animations at the corresponding wikipedia page. It is easy in principle and I won't explain it further.

Flashcards in Pavel Piezo

The "Pavel Piezo" games are a series about learning foreign languages. In the games, all questions, phrases and short sentences are entirely in the chosen (foreign) language. For the sake of readability I will write everything in english in this article.

We are using digital flashcards with the Leitner system to track a player's progress learning the vocabulary. But, we also use it for choosing which items (and therefore vocabulary) to put on the stage when a new level is started and for choosing items for the short vocabulary test between levels. For this, we extended the classic system by a few values.

Every time the player taps an item on the screen, she hears the spoken word within a short sentence ("That's a toothbrush."). This counts as one exposition and gets noted on the virtual flashcard that accompanies that word; its exposition counter is incremented by one. If the player finds an item he is currently tasked to look for, thereby effectively making an active recognition of the link between the spoken word and the picture of the correspondent item, a recognition counter gets incremented on that word's virtual flashcard. Finally, in the (skippable) short vocabulary test between levels, a question is played through audio ("Which one is the apple?"), four pictures are shown and the player has to tap on the picture of the apple. Doing that successfully on the first try increments a counter known on the word's virtual flashcard.

With this we know if and how often a player has been exposed to the link between a spoken word and the corresponding visual, if and how often this link has been actively recognized and if and how often knowledge of this link has been successfully queried. When choosing pictures for the vocabulary test, we employ the standard Leitner system to choose which vocables to query from the user but we also only choose words which the player had at least one exposition or better yet recognition within the game. Actually we are choosing words with a combination of high exposition, high recognition and low known counters.

At the beginning of the level we do not only choose the items by random to provide variety (see previous article) but we consider the information about which vocabulary the player has been least exposed to in previous levels or play-throughs. Additionally, if the system identifies items which the player has been exposed to many times but never identified it in the vocabulary test, we can make sure that this item gets chosen as often as possible. (Or maybe we have to check the corresponding graphical representations…)

Less obvious applications

That's all well and fine for games in which the player learns languages, but how about a few examples of game designs that don't suggest themselves so clearly?

Imagine a game where, to open a door, the player can either pick the lock or just bash the door in.

At one point in the game's story it is vitally important to pick some doors to not be detected by guards. If the game has stored tries and successes of previous lockpickings the system can react accordingly. Remind the player that there is lockpicking at all, offer to jump to the according section in the tutorial, make picking the locks in question extremely easy (or extra hard, if previous success suggests so).

Another application is for game mechanics. Let's take a shooter as example.

Our assumed player clearly prefers to jump in, shotguns blazing. But at one point in the game there's an event where, for story or design reasons, the sniper rifle is needed to successfully overcome the obstacles. With information on how often or little the sniper rifle was used before, and with how much or little success, the game can tweak the gameplay leading up to this point accordingly.

In our case supply of ammunition for the shotgun would run out, sniper rifles and ammunition would be placed in plain sight and enemies would be spawned that are easy to pick off with the sniper rifle. Effectively, the area leading up to the sniper event will be turned into an in-game-tutorial.

Of course, if we know the player knows very well how to handle the sniper rifle and just prefers to use a shotgun, we can leave her to her own style of playing, resting sure that the upcoming sniping will cause no frustration.

We can also make decisions based on a player's choice of classes in a MOBA or MMO.

If the player can choose different classes, different characters or skill sets, which one does he choose how often? How easy is it for him to find a group? How "successful" is the group? How much time does he spend with this character, class or skill set over all? We store this, and more if desired, in flashcards associated with the player.

(Keep in mind, we store this to have information about the player's preferences and act accordingly. Gathering information about how effective a class or combination of classes is against another class or combination of classes is a different beast.)

With the information we gather about the player's preferences and knowledge about the game system we can, for instance, fine tune grouping suggestions, making sure the player gets suggestions in which he can play his preferred style. On the other hand we could offer a reward for playing a rarely or never played class, thus, exposing him to content he would otherwise miss out on.

Conclusion

Gathering metrics about players, play styles, content exposition and usage of game mechanics is a deep rabbit hole. There are so many possibilities that it's hard to decide what information to collect and how. In my experience, starting out with an abstracted and generically applicable flashcard system that can be extended to ones own needs helps immensely to get started with key metrics and allows for easy expansion and tuning where necessary.