informa
/
Features

Constructing Artificial Emotions: A Design Experiment

Veteran game designer Daniel Cook follows up his much-discussed 'Chemistry Of Game Design' essay with a new, fascinating in-depth game design article discussing how to create emotions through games, from stimulus to biofeedback.

Bacchus: The Orgiastic Ritual Game

Bacchus is a multiplayer dancing game with a religious theme. The selling point is its ability to evoke intense emotions.

Imagine if you will, a decrepit theater filled with writhing, dancing people. The lights flare and swoop in time and the people chant in unison. A massive screen shows a mirror image of the hall like some surrealistic portal into an alternate universe. Instead of blokes and lasses in street clothes, the onscreen spirits are clad in ornate ritualistic garb. The movements on each side of screen are eerily synchronized. The pitch of the chant rises.

The screen zooms in on a girl in the center of the room. The crowd, as one, turns and watches her figure on the screen. She begins to dance. At first her movement is controlled and intricate. The screen pulsates and she yells to its beat. The room takes up her words and amplifies them, giving them god-like resonance. Bass mixed with reverb mixed with primal, guttural passion. Her dance becomes wild. The pace increases and she begins to confess.

The theater reacts. Each word she utters shimmers on screen, merging with ghostly photos from her past. In a beat, the entire room witnesses her sorrow over the death of her mother, her time alone in an empty apartment, and her first kiss. An inhumanly beautiful electronic chorus rises, matches and turns her words into a song. Her movements become a blur. Her glowing eyes are ecstatic. At the peak, her spirit on the large screen explodes in light and the girl collapses to the floor in fervent religious swoon.

The crowd goes wild. The screen zooms out and the next god dancer is chosen.

Later, the girl writes to her online friends that the night she danced was the single most powerful spiritual and emotional experience in her entire life. It was the night she was touched by a higher power while playing a video game.

Your browser may not support display of this image.
William-Adolphe Bouguereau - The Youth of Bacchus (1884)

Bacchus: The Gedankenexperiment

The game Bacchus is a thought experiment, not a real game. It exists merely to explore, in one design, several effective, yet rarely-used techniques for inducing emotion through gameplay. It happens to have a religious theme, but I’m primarily interested in exploring how designed experiences can yield intense player emotions.

The game designer’s palette of emotion has traditionally been limited to boredom, frustration, and triumphant mastery. There is very little published research on how to evoke a broader range of emotions and designers have very few practical or theoretical tools at their disposal in the quest to create meaningful, emotional experiences for their players. Designers interested in evoking emotion fall back on:

  1. Stealing techniques from non-game media. “And then we show a movie of the faithful heroine being stabbed by the evil villain!”
  2. Copious handwaving. “See, this pink pulsating blob represents ‘Feelings’”, explains the designer to the confused player.

The resulting experiences are far more emotionally simplistic than we might dream of creating.

To expand beyond the present constraints, I set forth a personal challenge. What if you wanted to create a game that pushes the player through a sequence of emotions, from joy to sorrow, to perhaps even religious ecstasy? What current or future techniques would you use? Is it even possible for a game to evoke a rich palette of emotions?

In order to build a game that induces such a complex emotional spectrum, we need to dig into the fundamentals of evoking emotions in games. It turns out that many folks in the scientific community have been studying tangentially related problems for quite some time.

What Is Covered In This Essay

This essay has five parts

  • Two factor theory of emotion: First, we’ll look at the psychology behind our emotions. I’ll lean on this to explore four pragmatic techniques that are demonstrated in Bacchus.
  • Technique 1: Tapping existing emotional memories.
  • Technique 2: Using relevant stimuli in order to evoke an emotional response.
  • Technique 3: Biofeedback for controlling physical state.
  • Technique 4: Social norm setting for seeding appropriate cognitive labels.

With each technique, we’ll cover the theory, how you can put the theory to use, how technology can help, and some of the limitations.


The Two Factor Theory of Emotion

Let’s begin with some basic cognitive science. The framework I’ll be leaning on throughout our investigation of artificial emotions is a well-known cognitive theory called the Two Factor Theory of Emotion, by psychologists Stanley Schachter and Jerome Singer. The theory states that in order for an emotion to be felt, two factors must be present:

  • Physiological change: The person feels elevated heart rate, sweaty skin and other elements of physiological arousal.
  • Cognitive label of the physiological change: Based off the context of the situation, the person assigns a label to the physiological change.

Simply put, when your body reacts physically to some stimuli and you mind assigns meaning to your physical state, you synthesize an emotional response.

An intriguing experiment

The Two Factor Theory of Emotion is certainly mildly intriguing as an analytic description of how emotion works, but it has a far more practical application in the realm of game design. In the process of proving their theories on emotion, researchers spent much of their effort on figuring out how to dissect the component aspects of emotion and reassemble them into new emotions of their choice. In effect, they figured out how to reconstitute artificial emotions within their subjects. Their experiments provide us with practical examples of how we might build our own systems of generating artificial emotion in our players.

It turns out that the physiological changes that accompany many emotions, such as fear and lust, are remarkably the same. There is a wide range of stimuli, including loud noises, intense memories or even a fear of heights that activate the sympathetic nervous system, prepping the body for action in the face of stress. Your heart rate elevates. Your palms become sweaty. Your alertness increases and body hair stands on end. Different stimuli, same response.

Due to the ambiguity of the physical response you rely on your brain to determine what all this activity actually means. Should your run, should you fight, should you laugh? In a heartbeat, you brain need to figure out what is happening and synthesize the correct response. In this moment, your carefully calibrated gray matter can be tricked.

One of the colorful experiments that demonstrate this effect was performed by psychologists Dutton and Aron in 1974. They wanted to see if they could alter the context of a situation so that the subject would instead experience lust instead of fear. In their study, a highly attractive young woman approached a sample of young men and asked them to fill out a survey. The experiment had two components.

  • Inducing the appropriate physiological response: Two survey situations where tested. The first was a safe location on a trail. The second was the midpoint of a very narrow bridge overlooking a deep crevasse. The researchers knew that merely standing on the bridge elevated the heart rate and caused anxiety. Standing on the path induced no anxiety.
  • Setting the desired cognitive label: At the end of the survey, the woman passed each young man her phone number and encouraged them to give her a call if they had any further questions.

Almost twice as many men (60%) gave the girl a call if they had been surveyed on the dangerous bridge than on the safe path. Due to the strong contextual signals in the form of presence of the attractive woman, the men misinterpreted the fear-driven activation of their sympathetic nervous system as authentic lust.

A potion for emotion

Other experiments validated the theory by inducing both happiness and anger in their subjects. These studies suggest the following general recipe for concocting artificial emotions.
Your browser may not support display of this image.

  • Inducing the appropriate physiological response: Put the player’s bodies in the appropriate physical state associated with the desired emotion through any means necessary.
  • Setting the desired cognitive label: Provide strong contextual clues that make the user misinterpret the physical sensation as the designed emotion.

By evoking both states in the player, the mental and the physical, designers can greatly increase the likelihood that players will experience the desired emotional response to a game.

With our theory in hand, let’s look at the set of practical techniques that help us generate the artificial emotions at the heart of Bacchus.

Technique 1: Active recall of emotional memories

“Each word she utters shimmers on screen, merging with ghostly photos from her past. In a beat, the entire room witnesses her sorrow over the death of her mother, her time alone in an empty apartment, and her first kiss.”

In Bacchus, the player recalls intensely personal emotional moments as part of a public confession. It turns out that this is a great technique for evoking a both a physiological response and a set of cognitive labels.

Theory: When you experience an intense emotion, a primitive portion of the brain called the amygdala kicks in and ensures that you store a vivid, emotionally charged memory. When you recall the memory, your brain also ensures that you remember the emotional element.

Your browser may not support display of this image.

Remembering an emotional event causes that you to re-experience that associated emotion.

Your browser may not support display of this image.

  • The recollection of an emotional memory triggers the recall of the emotions associated with the event.
  • This in turn activates the physical reactions associated with that memory.
  • You understand what the physical reactions mean in context of the memory and the labels you have assigned to the emotions within.
  • You experience that emotion.

For example, when war veterans recall a traumatic experience, they often experience elevated heart rate, perspiration and other signs of panic. The memory of the veteran’s traumatic event triggers a replica of the physiological response that occurred during the event. The emotional panic they feel is very real, very physical and easily measured. Other extreme examples of this phenomenon include many phobias such as fear of small spaces, flying, etc., all of which are typically rooted in some traumatic experience.


The basic system of recording and recalling emotional memories that underlie these reactions is present in every single one of us. In normal circumstances, the recall of emotional memories is a healthy and helpful aspect of basic human cognition. Storing emotions in good and bad times and then recalling them instantaneously if a similar situation occurs is a great evolutionary benefit. When you see the mountain lion the second time around, your body is instantly primed for flight. You don’t have to think or analyze the situation. You simply feel the correct course of action. The upside of all those millions of years of evolution for game developers? Your players have spent their lives collecting a deep pool of strong emotional memories that is just waiting for you to tap into with the appropriate game design.

Technology: In Bacchus, the confession is a game mechanic that encourages the players to tap into their emotional memories. Confessions are effective because the player self-selects memories with strongest, most pertinent emotional content and voluntarily shares them. For most people, this has the effect of turning on an emotional fire hose.

In order to make the recall as intense as possible, we want to augment the verbal recall with a few simple technologies. Human memory is stored like a sparse set of data points in a connected network. When a person remembers, they pull on related data points to flesh out the memory. You can increase recall by feeding the user related data points, thus lighting up more bits of their memory and increasing the clarity of the final result.

The classic example is a group of people remembering an event. A single person might remember a few key elements and have a fuzzy memory of the event. However, when a group of people gets together, each one contributes a small piece of additional information that helps light up a highly detailed map of the memory. Technology can serve a similar role.

  • Voice recognition: Publicly stating the confession allows voice recognition software to turn phrases into something the computer can understand.
  • A database of tagged photos: In the next twenty or thirty years, it is easy to imagine that many younger people will have lived a highly documented life. An individual will be associated with a massive archive of thousands of photos, videos and electronic messages that track almost any important aspect of their emotional and social life.
  • A free form association algorithm: Once we have a list of what the person is saying, we can start pulling up pictures that reference people, places and key times. When the player views these pertinent images, additional neural pathways are triggered that ideally enhance and clarify the recall of the memory associated with the confession.

The result of this particular system is the intense recall of very personal memories. With intense recall comes the highly desirable stronger physiological reaction.

Benefits: The main benefits of using the recall of personal emotions is that you are almost always going to get a solid emotional response, especially in newer players. You are digging up the raw materials of their most personal emotions and the results can be explosive.

Limitations: The downsides of using emotional memories to generate physiological responses are substantial.

  • Responses tend to be context-specific: The subject activates numerous cognitive labels in the process of activating the memories. When you are feeling sad about the death of your first dog, it can be hard to transfer those feelings to another topic. This can cloud any authorial intent on the part of the game designer when they try to introduce their own labels into the game.
  • They become less powerful upon repeated recall: If emotional memories are recalled in safe environments, they start losing their emotional power. The brain learns that maybe it isn’t worth getting all worked up about a false alarm. Many therapists use this fact to decondition those with phobias, but it becomes problematic for repeat players who are actively seeking the jolt that comes from recall.
  • Risk of trauma: Some people have severe psychological issues that are exacerbated by free form recall. Games that make people cry aren’t always healthy for the players that have serious issues to cry about.

Technique 2: Evoking emotional memories with relevant stimuli

“[T]he onscreen spirits are clad in ornate ritualistic garb… The room takes up her words and amplifies them, giving them god-like resonance. Bass mixed with reverb mixed with primal, guttural passion. An inhumanly beautiful electronic chorus rises, matches and turns her words into a song. Her movements become a blur. Her glowing eyes are ecstatic. At the peak, her spirit on the large screen explodes in light…”

In Bacchus, the player is surrounded by rich, evocative visuals and symbols. These also play an important role in priming the player to feel emotion. This technique builds upon some of the elements of emotional memories, but uses relevant stimuli instead of the blunt trauma of actual recall. We broadly evoke the player’s existing experiences with religion.

Theory: Once upon a time, I had a workmate that had an irrational fear of house plants. When she was a child, an aunt played tag with her in a greenhouse. At some point, the game stopped being a game. The girl desperately wanted the aunt to stop, but the aunt assumed the girl was still playing. The intensely emotional memory of being hunted, terrified, and surrounded by clinging, suffocating plants was seared into my workmate’s memory. This is in keeping with our discussion of emotional memories.

What I found fascinating is that recalling the specific event was not the only trigger for her phobia. Instead, stimuli peripherally connected with the memory, such as particular shades of green, a swaying vine, or a delivery of flowers with a bit too much leafy foliage on Valentine’s Day could set off a panic attack.

Emotional memories need not be triggered by recalling the exact event that embedded them. Instead, rich contextually coherent descriptions of similar situations can trigger those same pathways, though perhaps to a lesser degree. You may feel joy when you remember your first kiss with that shy girl from next door. But you may also feel joy if you hear the story of someone much like you who kisses the shy young princess. Enough nodes in your memory are triggered for you to react to the related story.

There is a rather broad cognitive theory of media at play here. In short:

  • The contents of media, be they sounds, images, physical or social situations inevitably intersect with rich existing palette of emotional memory found within almost any human being.
  • In order to process new stimuli, we tap into our memories and experiences relevant to the new experience. The nodes associated with intense emotional memories given priority during the act of recall since these are likely to contain the most relevant information to survival. An image of a hammer is more likely to generate the recall of that time you drove a nail through your thumb than the thousand previous nails you pounded successfully.
  • The result is a real emotional response to the stimuli, even though we may not be able to consciously make the connection with the original memory.

It can be a strange concept to wrap your head around. When you read a romantic book, your response is as much due to your past experience with romance as it is to the contents of the book. It is very likely that another person, one who has led an unnaturally lonely life, might read the same book and not be moved at all.

As a traditional author, your goal is to describe experiences that are relevant or highly correlated with experiences that your audience might possess. This is one very powerful technique for creating meaningful, emotional impactful media. This isn’t anything new. Much of what game artists and writers do involves the creation of relevant stimuli.. All those detailed graphics, booming sound effects and cliched story lines? That’s relevant stimuli, baby.

Technology: According to our little theory and building up on the lessons of emotional memories, you can predict the characteristics of highly effective relevant stimuli:

  • Detailed: The more detailed you can be, the more likely you’ll light up a rich network of nodes.
  • Personal: The more you can get someone to empathize with an experience, the more likely they’ll link it to their own personal memories.

Bacchus doesn’t rely on plot or well-rendered NPCs for its relevant stimuli. Instead it focuses on making the fantastical ‘spiritual’ experiences in the game personally relevant and highly detailed.

Your browser may not support display of this image.

The detail comes in the form of traditional imagery and sound effects that references common culturally relevant spiritual symbols. The ornate costumes worn by the player avatars are intentionally laden with various religious icons, glowing colors, angel wings, etc. The choral sounds are intended to recall childhood experiences with church choirs or even pop culture representations of spirituality found in movies or on TV.

Movies and books are typically limited to piling on the detail and hoping that it strikes a chord. Games have the opportunity to make all these details much more personal and relevant to the moment at hand.

The primary technique you see in Bacchus is known as "avatar mapping", where the player's actions are mapped onto an in-game character. It stops being about watching a costumed religions icon and much more about interacting and participating on a personal level with a religious situation.

  • Capturing movement data: In the next twenty years it should be feasible to build high dynamic range digital cameras that take crisp 20 megapixel shots at high framerates. Add in range-finding technology, a beefy processor and some multi-person image processing software and you have a system capable of tracking the subtle movements of an entire crowd in real time. Simple versions of this are already happening with the hardware like the EyeToy or Nintendo Wii-mote.
  • Mapping movements onto an avatar: All that detailed movement information can be mapped onto a 3D model in real-time. This technology exists today and will only get better. Check out the video below and extrapolate the fidelity that will be possible a decade or two into the future.


    Fix8 example

  • Voice mapping: Just as your visuals can be altered, so can your voice. When a hundred people yell out the same phrase in a crowded room full of pumping noise, the computer isolates a single voice. That single voice is converted to text, timing, inflections, emphasis and volume information. It is broken down its components and reconstituted. If an old man wishes to sound like a young boy, a middle aged woman, or a monster ripped from the studios of Hollywood, all it takes is the flip of digital switch. You talk and someone else instantly says your words.
All this is a crapshoot. If I’m lucky, I’ll actually trigger the recall of an actual spiritual experience.

Benefits: The use of related stimuli is the gold standard for inducing artificial emotions across practically every media known to man. Great novels, paintings, movies, and poems rely on their intense portrayals a human experience that is not our own, but close enough to tap into our personal experiences. As a creator, this is technique is quite cost effective and most of us have been trained in its application.

  • You can craft a single static experience that is broadly applicable to your audience. You don’t have to worry about customizing your message. They’ll expend the effort to find meaning.
  • You can rely on your personal experience for inspiration. This isn’t because you are special. Instead, it is because whatever you come up with will likely be close enough to what someone else has experienced. Humans are limited in what they can experience. If you describe how you are feeling when you are sad or happy, you are likely also describing how others are feeling when they experience those same emotions. This greatly reduces the need for detailed customer feedback, an expensive activity.
  • You can be sloppy. Relevant stimuli is like using a shotgun. Whatever you make is likely to be meaningful to someone in the audience. A wide range of modern art and music plays on this loophole by showering people with ambiguous messages. A small percentage ‘gets it’ and that is enough to pay the bills.
  • Setting up cognitive labels. If all else fails, relevant stimuli can still at least set up the appropriate context for interpreting the experience. Even if the player doesn’t tap into existing spiritual experiences, they at least understand from the overload of religious symbols that they participated in some sort of religiously-themed activity.

Limitations: Relevant stimuli is amazingly powerful, but has some limitations when it comes to use in games.

  • High burnout. After a very short period of exposure, players start ignoring relevant stimuli. Techniques like avatar mapping and other user-generated content strategies can keep content fresher, but replacing consumed content is still a costly issue to consider.
  • Limited interactivity. Traditional forms of relevant stimuli do not change with the user’s input. Movies, narratives and images are all about evoking a response, but they have no ability to adapt once the player responds.
  • Unpredictable. It is hard to guarantee that relevant stimuli will produce the desired emotional response. Look at the case of Harry Potter. Most saw the first book as a charming schoolboy romp and identified with its stimuli targeting their own feelings adventure, mystery and coming of age. Others saw it as an obvious promotion of witchcraft and Satanism. They reacted with anger and fear. What your audience brings to the work has a huge impact on how it is interpreted.

Technique 3: Biofeedback

"She begins to dance. At first her movement is controlled and intricate. The screen pulsates and she yells to its beat."

Not all techniques at our disposal create both a physiological response and cognitive labels. There are some that just do one or another. Such techniques can be used as building blocks in a larger system.

One well-studied technique that affects that body is bio feedback. This is particularly interesting to game developers since it is a fundamentally interactive technique and offers deep opportunities for mastery-focused gameplay.

Theory: In the classical model of human behavior, there is the somatic nervous system which controls voluntary actions like moving your arm and the autonomic nervous system which attempts to maintain homeostasis by automatically adjusting such things as body temperature or heart rate.

A surprising number of automatically controlled systems can in fact be influenced consciously. The most obvious one is breathing, as seen by pearl divers holding their breath. Other systems can be controlled indirectly by consciously adjusting related systems. For example by staying stationary, slowing your breathing and thinking calming thoughts, you can slow heart rate.

If only we could encourage the player to directly control their physiological state, they could

Latest Jobs

Sucker Punch Productions

Bellevue, Washington
08.27.21
Combat Designer

Xbox Graphics

Redmond, Washington
08.27.21
Senior Software Engineer: GPU Compilers

Insomniac Games

Burbank, California
08.27.21
Systems Designer

Deep Silver Volition

Champaign, Illinois
08.27.21
Senior Environment Artist
More Jobs   

CONNECT WITH US

Register for a
Subscribe to
Follow us

Game Developer Account

Game Developer Newsletter

@gamedevdotcom

Register for a

Game Developer Account

Gain full access to resources (events, white paper, webinars, reports, etc)
Single sign-on to all Informa products

Register
Subscribe to

Game Developer Newsletter

Get daily Game Developer top stories every morning straight into your inbox

Subscribe
Follow us

@gamedevdotcom

Follow us @gamedevdotcom to stay up-to-date with the latest news & insider information about events & more