Simple prototyping with metrics and events

This post describes a method of educational game design that uses metrics and events as pillars. It also describes a tool called WIND, which is part of the FLAG Game Engine. WIND provides an interface to design and simulate games based purely on metrics and events that affect them. The WIND editor can be found at http://www.flagamengine.com/WIND. WIND is essentially a tool that builds a JSON object, of the same name, to be used with the FLAG Game Engine.  The algorithms that the data eventually passes through for game play is all open source and does not require the use of the entire FLAG Game Engine. The UI to interact with the WIND object is all HTML5 and is also open source. All source files can be found here: https://github.com/flagamengine/FLAG. This is thought process is meant to be a bridge between game designer and instructor or content expert.

The only validity to this technique is that I use it with some success.

Motivator and Method Metrics

What is the first thing to think about when starting an educational game design project? Should you define the genre, decide if it’s 2D or 3D, come up with graphics and aesthetics, design UI elements, define a target audience, use an engine or write code from scratch, multi-player or single player, is this a mobile, browser, console or pc game? All of these things do eventually need consideration just not right now!

The first thing you should think about, when designing an educational game, is to define a clear learning objective(s). This is a fairly obvious first step. Yet it often gets skipped over in favor of thinking about more detailed information about the game. The learning objective should be some information or skill that a player is more familiar with after playing the game. The game designer’s job is to help the learning happen without the player noticing. We will talk more about this in a bit when we cover the two types of metrics.

The second, and maybe more important thing to think about, when designing an educational game, is how will your game keep score and what will it keep score of? We need tangible and quantifiable methods of monitoring a player’s progress. Score, as in number of points, is rarely specific enough to accurately measure a player’s skill, or knowledge of content. We need metrics that are more closely aligned with the learning objectives of the game. When a game’s metrics relate directly to a games learning objectives it increases the odds of accurately assessing a player. For example, if we were designing a game about economics and one of the learning objectives is related to supply and demand, an obvious metric to keep track of is ‘money.’ Money, in this case, is our first type of metrics which I call a motivator metric. These are the metrics that a player sees as the ultimate goal. However, keeping track of a player’s money alone might not be a good indicator that they understand supply and demand. We will most likely need to keep track of other metrics that our player can monitor and or interact with. Let’s add ‘supply’ and ‘demand’ to our metrics list. Keeping in mind that values like supply and demand tend to change over the time, let’s add one more metric called ‘turn’ to measure time. The supply, demand and turn metrics are what I call method metrics. These are the metrics that a player must directly influence, or be influenced by, in order to be successful. During gameplay a turn in our game might consist of a player choosing to invest in a particular product called Widget X. If we know that supply is high and demand is low, during that particular turn, we can accurately tell that this player does not understand supply and demand. Although the ultimate effect of the player’s action might be reflected by a loss in money, our motivator metric, we are able to specifically identify the area of this player’s weakness by examining the values of the method metrics. We see these methods of measuring skill often in professional sports. In football, we keep track of yards gained, in addition to touchdowns scored, to more accurately rate a player. In hockey, we keep track of shots on goal, in addition to goals scored, as an indication that a player might be headed in the right direction.

By getting the player to focus on motivator metrics as their goal while forcing them to use the method metrics to get there, we provide them with deeper, more thought provoking learning experience. At the same time, we can use this combination of metric types to accurately assess a player’s strengths and weaknesses. If the economics game example is designed correctly, our player should be focused on making more money. The player may not necessarily realize that they need to understand the concept of supply and demand to get it. The moment they do realize it, that’s when the learning happens!

Events and Effects

Now that we have put some thought into the metrics of our game, the next step is to come up with ways that our player can have influence over, or be influenced by them. Basically, we need to give the player methods of action. This is where a rush of possible gameplay decisions might come flooding into your head. Is this a platforming game or a first person shooter? Is this a real time strategy or a role-playing game? Should the A button be for run and the B button be for jump or vice versa? Should it be like Angry Birds or more like Candy Crush!? Wait, wait, wait. Remember how we decided not to make all those detailed decisions? For now, let’s just refer to whatever a player does, or whatever happens to them, as an event. The main purpose of an event is to alter, or affect the metrics. Depending on the type of event, a player’s decisions, or previous decisions, may determine how, or to what degree the metrics are affected. Changes in metrics as a result of an event we will refer to as effects. Let’s first look at the types of events.

We can categorize all events into four types. It doesn’t seem like a lot by it is true if you think about it. The four event types are happenstance, multiple-choice, multiple selection and slider. It’s not important to think about the context of the event at this point. Meaning, a multiple-choice event may not appear as a test question to the player during gameplay. Instead, it might be in the form of which chest to open or which hallway to go down. For now, we want to think about the content of the event and what the resulting change to our metrics. You can design just about any game using these four events.

Happenstance events are typically something that simply happens to a player triggering metric effects. ‘You pass GO, you collect $200.’ Although a happenstance event might occur as a result of an action by a player, which in this case was passing through the GO space on the board, the player has no input on the results of this action. He does not get to choose if the effect on the money metric is +$100 or +$300, the effect is always +$200. Simply put, the metrics are altered using the event’s defined effects with no option.

Multiple-choice events can have any number of effect options. However, when the event is triggered, a player must choose only one option that will be used to affect the metrics. These events follow a standard multiple choice question format. The choice of option A will result in one possible set of effects to the metrics. While a choice of option B may result in a complete different set of effects. The player may not always know the exact effects of their choice but they always have the ability to choose. As I mentioned before, a multiple-choice event does not have to mean that a player will be faced with a multiple-choice question in the same way they would be with a written test. The event might be a visual decision such as which hallway to go down or which door to open. The way the multiple-choice event is presented to the player, as with all the event types, is not particularly important at this stage of development. What is important is that we are providing our player actions with purpose. That purpose should be to move the metrics one way or another. If an event has no effect on the metrics it will provide no indication of player progress. In educational games the goal, both as a game designer and an instructor, is to be assessing the player as much as possible without them realizing it.

Multiple selection events can also have multiple effect options. When the event is triggered, a player can choose from several options that will be used to affect the metrics in combination. Unlike multiple-choice events, the player now has the ability to select more than one choice. Again, visually, it doesn’t matter how this type of event will be presented to the player. It could be a weapon load out or a choice of which characters will join you on your quest. The important thing is that each choice has some quantifiable effect on the metrics.

A slider event must have two available effect options. These options represent the two possible extreme effects of the event. The player must select a value within a range. The player-selected value determines to what extent the metrics are affected using the values contained in the effect extremes. For example, if a slider’s range is between 0 and 100, and a player sets the slider at a value of 50, the slider is set half way between the low extreme and the high extreme. If the low extreme effect on a money metric is +$200 and the high extreme effect on the money metric is +$400, the player’s money metric would receive a value of +$300 as result of this event.

Anything that a player could do or could have happen to them, during gameplay, should fit into one of these four categories. How the events are visually presented to a player can be thought about later. For now, we are just looking to give the player the opportunity to interact with and alter the metrics.

Balancing

By removing some of the details of game design, mainly the details that have to do with visual aesthetics, we can focus on the core of our educational game using metrics and events. When you start using this technique, depending on the scope of your game, you may quickly realize that you need a large number of metrics and an even larger number of events. Some events might affect metrics in a big way while others might have subtle effects. Some events might affect multiple metrics at once while others only affect one. Many games do not follow a linear path allowing for more exploration by the player. Meaning, you may not know the sequence of events that will take place. Managing and predicting the status of metrics during gameplay can quickly become a daunting task. How do you know when the effects of a particular event are too much? What sequence of events in your game provide too much of an advantage versus which ones provide too much of a handicap? All these considerations are what go into the balancing of a game. If you are an Excel wizard, you might be able to chart all these things out using a spreadsheet however, wouldn’t it be nice if there was a tool designed specifically to test these techniques? Luckily there is!

That ends the theory, or the ‘why’ part of this post. The next section is the ‘how’ part. We will be working through a simple example using the WIND editor.

WIND

Creating Metrics

Let’s start using WIND by initializing some metrics. The WIND editor can be found at http://www.flagamengine.com/WIND. Under the Metrics tab in the WIND editor, click Add. The only information required to create a metric is a name and a start value. Let’s enter ‘money’ in the Metric Name input box and place a value of 100 in the Metric Value box. We will talk about Extras later and for now we can leave the Allow Negative toggle checked. This simply determines if metric is allowed to pass below zero or not.

Creating Events

Now that we have a metric, let’s create an event that has some effect on it.  Open the Events tab and click the Add button in the first section. After adding a new event you will see some of the fields have been pre-populated. Let’s change the Event Name field from ‘Event 0’ to ‘Get Paid.’ Let’s leave the Event Type dropdown selected on Happenstance and enter some display text for our event in the Main Text input box. You can type something like: ‘It's pay day! You made $200.’ Now that we have some base information entered about our event, it’s time to define the effects the event will have on the metrics.

In the Effects Options dropdown you should notice that there is one option named ‘0’. This indicates that there is one available effect option. The zero simply indicates that this is the first, in a possible list of options. Since this is a happenstance event, there is only one option in the list. Remember, the player does not have a choice of effects with this type of event. Click the Edit button in the Effect Options section. The effect pop-up menu should appear. Here we have the ability to add some more specific text dealing with this effect. We don’t really need any here since we said all we needed to in the main text of the event. Below the Text input box you should see two effect categories. We will talk about these more later, for now will want to deal only with One Time Effects. Click the Add Effect button in the One Time Effects category. You should see an effect has been added. Effects, at their core, are simple linear equations. The left side of the equation can be one of two things. The first option is for the left side to be one of our metrics. The second option is for the left side to be an eGroup. An eGroup is simply a predetermined sequence of effects. We will use them later. For now, let’s set the two dropdowns on the left side of our effect to Metric and money. In the operator dropdown, let’s set the ‘+’ sign to indicated we will be doing some addition to our money metric. The right side of the equation can also have two possible values. Possibility one would be to use one of our metrics. For example, we could add the value of our money metric to the value of our money metric, effectively doubling it. Possibility two for the right side of our effect equation is a Number. Let’s select the Number option for the right side of our equation and enter a value of 200. Click Ok to close the effects pop-up menu.

Running Events

It should be pretty clear what we have done here. We have created an event that, when it occurs, adds 200 to our money metric. We can test the effects of our new event by opening the Sim tab. Expand the Control Panel. You should see our ‘Get Paid’ event listed in the Manually Run Events dropdown. This is where you can select an event to display and manually interact with it. Click the Display Event button. On the left side of the screen you should the text we entered for our event along with the Run Event button. Although this may not be the way a player experiences the event, or how it is displayed visually in our final game design, it works nicely at this stage of development.  Click the Run Event button. You just simulated your first game play event! You might notice, if you look back in the Metrics tab, that the money metric is still labeled with a value of 100 even though you just ran an event that was suppose to add 200 to it. The values associated with the metric listed under the Metrics tab reflect the start values of the metrics. To see the current values of the metrics we need to look under the Sim tab and expand the Player Metrics section.  Here we can see the same list of metrics however the values reflect the effects of the events that have taken place. An additional way to see the current value of metrics is to click the View toggle beside them. This will display the current metric name and value in the left hand side of the screen. This can help you decide later which metrics a player must be constantly aware of and which ones they might have to dig to find the value of.

Player History

When designing educational games, and assessing the progress of a player, knowing the current values of metrics is important, however knowing how the player got there might be equally as important. The Player History section under the Sim tab does just that. It keeps track of the sequence of events that the player takes part in and the decision or option they used during the event. In this case, the 0 under the Event column represents the index of the even that occurred. The 0 under the Option column represents that the first, in our list of effects options, was chosen. Since this was a happenstance event, there was only one available option. We can clear the Player History and restore all the Player Metrics to their initial start values by clicking the Reset button under Simulation in the Control Panel.

As our games get more complex, with addition of more metrics and more events, we may find it tedious to test every possible combination of event sequences. To do some automated testing we can use the Auto Run Events feature in our Control Panel. By typing a number value into the Turns input and then clicking Run Events, we can populate the Player History with a random sequence of events along with simulated choices.

One Time and Recurring Effects

Let’s go back and create a slightly more complicated event. Open the Events tab and click Add to create new event. Edit the Event Name to something like ‘Buy a House.’ For simplicity, we can keep this a happenstance event. In the Main text input let’s just put something like ‘You are buying a home.’ Click Edit under Effect Options. In the Text field let's add some additional information about the effects of the event. ‘This house costs $100. It also costs $25 per turn to maintain. ‘ For added depth to events, we separate effects into two different types. These two types are called One-Time Effects and Recurring Effects. One Time Effects occur only at the time of an event. Meaning, the metrics are altered only once. Recurring Effects occur after an event for every event that has taken place prior to the current event. Meaning, after a second event takes place, if the first event contained any Recurring Effects, those effects will then be applied to the metrics. After a third event, both the first and second events are checked for Recurring Effects and they are applied to the metrics and so on.  To demonstrate this, let’s first add the cost of the home by clicking Add Effect in the One Time Effects section. Make the effect subtract 100 from the money metric. Now let’s add the maintenance cost by clicking the Add Effect button in the Recurring Effects category. Make this effect subtract 25 from the money metric. Click Ok to finish editing the effects of this event.

Return to the Sim tab. In the Control Panel section, select the ‘Buy a House’ event from the Manually Run Events dropdown. Click Display Event. On the left hand side of the screen our newly created event is displayed and ready for action. Click the Run Event button. As a result, you will see 100 get subtracted from the player’s money metric. Remember that was the One Time Effect of buying a house. Now select the ‘Get paid’ event from the dropdown and click Display Event. Once the event is displayed, click the Run Event button on the left hand side of the screen. Notice, that although our ‘Get Paid’ event initially added 200 to our player’s money metric, this time it only added 175. That is because the Recurring Effects of the ‘Buy a House’ event subtract 25 from our money metric for every event after a house has been purchased. It’s the maintenance cost of owning a house!

Extras

Additional things can be done to metrics themselves to enhance or extend them. One way of extending a metric would be to have it’s value change over time. Let’s create a new metric to demonstrate this. Under the Metrics tab, click the Add button. In the Metric Name input let’s type ‘cost of house.’ For the Metric Value let’s keep the 100 we used before. In the Extras input box add this text:

{"byTurn":[100,200,300]}

Extras will typically be written using JSON notation. Here we have created an object containing a ‘byTurn’ property that has a list of values associated with it. The ‘byTurn’ property is a built in function of WIND that allows values to be read according to how many events are in the Player History. Meaning, if there are 0 events in the Player History, the value at the 0 index will be used. If there is 1 event in the Player History the value at the 1 index will be used and so on. Click Ok to finish creating the metric.

Now let’s use our newly created ‘cost of house’ metric. Return to the Events tab, make sure the ‘Buy a House’ event is selected. Click Edit under Effect Options. First, in the Text input box, change the first sentence from “This house costs $100.” To “This house costs *cost of house* dollars.” This inserts the value of the money metric into the display text. Second, in the One Time Effects section change the right half of the equation from a Number to a Metric and select the ‘cost of house’ metric. Click Ok to finish editing the effects.

Open the Sim tab. Click Reset under Simulation to clear the Player History. Select the ‘Buy a House’ event from the Manually Run Events dropdown and click Display Event. Click Run Event to see the changes in the Player Metrics. When you run the ‘Buy a House’ event you will notice that just like before the player’s money metric had $100 subtracted from it. Let’s run that same event a second time. Now you should notice that the player’s money metric has had $225 subtracted from it. Two things have changed the effects of the event. First, since our Player History already contained one event, and the Extra stored in our ‘cost of house’ metric contained a ‘byTurn’ property, the cost of a house changed from $100 to $200. Additionally, since we already own a home, we had to pay the maintenance cost of $25. If we run that event a third time, our player’s money now has been reduced by another $350. For the same reasons as before, the cost of a house has now changed from $200 to $300 and we now paid $50 for the maintenance of our two previously purchased homes.

Even in this very simple example, you can start to see the challenge in both keeping track of metrics and in designing balanced games. You can also start to see the roles being played by our two metrics. The ‘money’ is our motivator metric. It is what the player sees as the results of his/her actions. The ‘cost of a house’ metric is a method metric. It is what the player should be using to make better choices with the management of their money.

Prerequisites

During the course of a game, we might want a few built in checks before events can occur. One of those checks might be that a particular event cannot occur unless another predefined event happens first. This check is called a Prerequisite. Let’s add a Prerequisite to our ‘Buy a House’ event. Open the Events tab. Make sure the ‘Buy a House’ event is selected. Click the Add button in the Prerequisite section. A Prerequisite selection box will appear with a dropdown list containing all of your created events. Select the ‘Get Paid’ event from the list and toggle the Match Amount to true. This has told WIND that the ‘Buy a House’ event cannot occur until after a ‘Get Paid’ event has occurred. In addition, thanks to the Match Amount toggle, it has declared that for every ‘Buy a House’ event, there must be at least an equal amount of ‘Get Paid’ events. Let’s test these new checks on our event.

Open the Sim tab. Click Reset under Simulation to clear the Player History. Select the ‘Buy a House’ event from the Manually Run Events in the Control Panel. Click Display Event and then the Run Event on the left hand side of the screen. You should immediately see and error message indicating that the prerequisites for this event have not been met. Return to the Control Panel. Select the ‘Get Paid’ event and click Display Event. Click the Run Event button on the left hand side of the screen. The event should run as planned. Return to the Control Panel and reselect the ‘Build a House’ event. Display and run the event. The event now should also run as planned confirming that first prerequisite check is working. To see if our Match Amount check is working let’s type a value of 10 into the Turns input of the Auto Run Event section. Click the Run Events button. Now look in the Player History section. Make sure that there are never more 1’s than 0’s. This will indicate that event number 1, which is the ‘Build a House’ event, can never occur more often than our 0 event, which is our ‘Get Paid’ event.

Repeat Limits

Another way of regulating the events in our game would be to put a repeat limit on certain events. This will affect how many times an event can occur during a particular game.  Let’s put a repeat limit on our ‘Get Paid’ event.

Open the Events tab. Select the ‘Get Paid’ event. In the Repeat Limit section enter a value of 3. This will indicate to WIND that a player will only be able to experience the ‘Get Paid’ event 3 times during their game. In fact, along with the prerequisite check we put on our ‘Build a House’ event, we are also limiting the amount of times that event can occur as well, since we need to get paid anytime we want to buy a house.

eGroups

eGroups are groups of effects. Sometimes you need a particular sequence of effects to take place in multiple events. Adding the same linear equations to events over and over would get tedious and become difficult to edit. Instead, you can build the sequence as an eGroup. Let’s add an eGroup to our example.

Let’s first add one more metric to our list. Name the metric ‘cost of maintenance’ and set the value to 30. Now open the eGroups tab and click the Add button. In the eGroup Name input box enter something like ‘house purchase.’ Click the Add Effect button. Select the ‘money’ metric in the first dropdown. Select the minus sign from the second dropdown. Select Metric for the category and the ‘cost of house’ metric from the third column of dropdowns. This replicates the One Time Effect we currently have in our ‘Build a House’ event. Click Add Effect again. This time select the ‘cost of maintenance’ metric and subtract 5 from it. This will reduce the ‘cost of maintenance’ every time you build a house. Simulating some over-lap in the costs of owning multiple houses. Click Ok to finish creating the eGroup.

To include the new eGroup in our game, return to the Events tab and make sure the ‘Build a House’ event is selected. Click Edit in the Effect Options section.  Edit the second sentence in the Text to read: ‘It also costs *cost of maintenance* dollars per turn to maintain.’ This will show the value of the ‘cost of maintenance’ metric in the display text for the event. Then, in the One Time Effects, change the first dropdown from Metric to eGroup. You should see the new ‘house purchase’ eGroup appear in the second dropdown. This will trigger the two effects we placed in the eGroup. Also, in the Recurring Effects, change the right half of the equation from Number to Metric and then select the ‘cost of maintenance’ metric from the lower dropdown. Click Ok to finish editing the effects of this event.

Hopefully, by describing these techniques and by giving you a tool to prototype with, your next educational game will be a success!