Katherine Isbister received her PhD from Stanford in 1998 on the topic of reading body language in onscreen characters. Nicole Lazzaro founded XEODesign, a contract firm that conducts player testing on games at an early stage to ensure a quality emotional experience – she has fifteen years experience in the industry.

Gamasutra already covered 'Emotion Boot Camp,' the tutorial they gave at GDC 2006, but also had an opportunity to talk with them in depth on character and interface design, and discuss ways to tap into a vast market of people who want more emotionally involving play.

Gamasutra: What problems do you both see afflicting the game industry?

Dr. Isbister: From a character standpoint it’s a real opportunity time. Developers are feeling the pressure of new platforms and bigger budgets and yet are realizing that too many sequels and licenses are going to lose the interest of the broad audience they want. I've had mainstream developers that might have attended my talks a few years ago and seen it as a blue sky sort of thing and now they come to me and say "hey, we really want to do this stuff, lets talk about how to make it happen."

I've also seen students put together portfolios and get jobs based on issues around character. It’s a scary time, but there’s an opportunity for developers and people bringing new ideas to the industry.

Lazzaro: The industry is in a real quagmire, the technical demands of next-gen hardware are increasing budget and team size at an exponential rate and risk, likewise, is increasing at that same rate. But we do need to innovate, even though it’s becoming more difficult, so we need techniques that allow developers to tell early on what their design is going to play like. During prototyping the methodologies we use [XeoDesign] help developers detect early on what the experience is going to be like.

The other thing is to create experiences that are innovative in new spaces, so I think the casual games space is a very interesting opportunity right now because the cost to get in is so low. There is something of a glut of content, but you do have the ability for a $100-150K budget and a five- to twelve-person team to go in there an make an experience that can do really well in that space.

They're saying by 2008 it'll be a billion dollar business, that’s not too far away. If we keep chasing this curve of more and more polygons we're not going to get more emotional expression. The effectiveness of greater realism is hitting a plateau and the next phase is really about understanding what makes a compelling play experience and what goes on inside users' heads as they play and how to make that experience more emotional. Without emotion there is no game.

Body Language

GS: Gestures and body language and tone of voice supposedly comprise 80% of communication, the actual text is only a fraction of the signal. What are your hopes for the gestural input, particularly with the Nintendo DS and the Wii remote?

KI: We're already seeing some amazing work being done, Nintendogs was a stellar example of the kinds of interactions you can have that are emotionally significant for some people. I think the trick is to get designers thinking in new ways to take advantage of that tactile interface, and that means understanding the social component of what’s going on gesturally between people. It's like learning how to program, you need to understand what the routines and codes are happening between humans and how to use that as an instrument to get a really powerful game dynamic take place.

NL: Whenever you have a new platform with a new interface model you’re forced to innovate, you longer have that one button mouse where you’re clubbing everything in the world.

GS: Diablo.

NL: Yeah, basically that’s it, one button mouse, that’s all you’re doing. The other side is a twelve-button controller you use with two hands, or a keyboard with all those keys, not that fun. We’ve done work with Electric Planet years ago as a precursor to Sony’s EyeToy, and we did body language recognition to create “body games” for them as part of their R&D phase. With the Revolution the really great opportunity is that controller is so familiar looking, it looks like a remote control, so I know how to hold it already and that’s a pretty easy hurdle for people to get started.

If games come out on that platform that are very simple one-button kind of casual experiences, that’s going to make that platform very successful for that market. Not to say there won’t be hardcore games on that platform, but the significant thing is tearing down this 100 foot wall that most games have where its like “until you climb this wall you can’t have any fun in our playground,” and that’s ridiculous. Sure, you can ramp up later, you can add more controls and options, but just make the initial play experience more accessible.

That’s what we do with a lot of our clients, we asses that first thirty minutes of play and let people who’ve never touched that device play it, and we identify the barriers to entry. The DS has the potential to make it easier for people, the ability to touch with the pen is a very unique thing that’s different from clicking with a mouse or mashing buttons.

Holding Hands

KI: Our lab just got a grant for a motion capture system to study interpersonal gestural dynamics and I’m really hoping we can feed that back into these sort of designs. I’ve got a lot of NSF grants going towards that kind of research. Once motion capture gets to an affordable level we’re hoping we can have these dynamics boiled down to a computable level and literally create gestural interfaces.

NI: And I think Katherine, there’s more you could say about the psychology of touch.

KI: Well it depends on what the touch is, holding hands can mean different things in different cultures, that’s a really sensitive issue. We saw that in our workshop, we told everyone that a $100 bill was hidden on someone and people were afraid to touch each other because of the boundary we typically have between other people. It turned out it was in my back pocket. I think everyone has that hesitation, but if you can get people to do something a little risky they automatically bond and their intimacy level goes up.

GS: And really any kind of multiplayer game is about that.

KI: Yeah, what’s so amazing about rhythm games is that getting in synchrony with people is typically what we’re trying to do socially, and multiplayer rhythm games really allow you to do that. You step into this space and get in synch and instantly you’re more intimate with that person because you’ve gotten your whole physical self in synch with them. You’ll see that people who are close have the same gestures, it’s just something that happens naturally.

GS: Nicole classifies fun into four types. There’s “Hard Fun,” like in a first-person shooter, where you frag all the monsters and have three health left, there’s “Easy Fun,” like Katamari [Damacy] where you’ve got that time limit but its pretty relaxed, there’s “Serious Fun,” which is like a rhythm game or a simulation, and then there’s “People Fun.” The examples you gave, Nicole, pointed at multiplayer games and guilds and so on, but I’m interested in interactive drama, making the mechanics be the characters and their social dynamics. I’d like to hear what you think about that, in terms of implementation.

KI: The guy I got my PhD under has a book called The Media Equation (right). He makes the argument, based on a series of studies, that you can’t help but treat technology that acts social as if it were real people. I really believe that to be true. When you start creating these characters in a dramatic world, you’re using the same social senses to interact with them as you would a person – that’s the place we have to begin from.

It’s a hard problem but not an unsolvable one, I don’t think that there are “cyborgs” and “real people” and we’re clearly labeling them in our brains. They’re the same to the extend that we make that character.

GS: So you don’t need human-level AI to do human-level interaction?

KI: I’m much more for that Hollywood approach; the back of the set doesn’t look so great, it doesn’t matter, its about controlling the unfolding of the experience. I think that is the hard part of designing interactive experiences, it’s a question of “how do I make the reactivity of these characters feel authentic within the bounds of the experience I’m creating.” That’s a new part of the art form. It’s kind of a trial and error, tricky thing, you can’t see what you have until you prototype it. And you have to make the interface pretty enough that people perceive those characters as believable and are able to tell that what they’re doing is effective.

On Half-Life 2

GS: Dr. Isbister, what’s your assessment of Half-Life 2’s characters?

I think Half-Life 2 is a great example of the tremendous advances that have been made, as well as the challenges. Many of my students commented on how amazingly lifelike the characters were in that game. But I noticed a pattern--the ones who thought so were already hard-core first-person shooter players. To them, knowing the limitations of the medium and the genre, the movement and facial expressions, and the ability to interact during cut scenes were really exciting and liberating.

But, I had everyone in my character design class play the demo version of the game and I noticed that students who were not already fans of this genre felt really frustrated interacting with the Half-Life 2 characters. They expected even more interactivity and lifelikeness from them because of how good they looked. And so they were disappointed.

With the power to create such lifelike models and increasingly engaging motion, we run up against the problem that the interaction paradigms that are in place may need to shift, to make the whole experience work for players, and especially for a broader, less hard-core audience. Which makes grounding design in the player's experience [rather than relying on tried and true genre expectations] essential.

One other point about Half-Life 2; I had a student write a whole paper on the G-Man, and how amazingly intriguing and engaging he was. This student tracked down everything he could on fan sites about G-Man, and really went above and beyond in his paper. Curiously, the minimal interventions and the ambiguity surrounding G-Man were more powerful for this student than all the fancy graphics and motion of the rest of the game.

I'm sure there's something to learn here about creating powerful emotional reactions--how it doesn't always boil down to higher powered techniques and technologies, but depends first and foremost on leveraging human nature in the most clever and efficient ways.

NL: Games are already tapping into social cues, and it’s a symbolic thing like recognizing a picture or a cartoon, you don’t need them to look like real people. As long as there’s something about it that gives it an anthropomorphic quality, we react to it and we react with it. A game like Diner Dash really does a nice job of this, a lot of the emotions in the game come from the fact that we know it’s a restaurant, so we have a situational narrative, we know the goal is she wants to build a restaurant and earn tips, and the characters have emotional states. They get angry, they get frustrated, they get, you know, kind of happy, depending on how good their service is. The game uses social cues to communicate that to the player, and this makes the player feel like they’re serving customers. You don’t literally feel like you’re out in the real world being a waitress, with all the stress and everything else, but you feel like you’re doing something similar.

A lot of games are simulations of real work, you’re a zoo manager in Zoo Tycoon or a mayor in Sim City. In a sense we’re already there, but not many people can do it yet. Or they go for the obvious stuff, like “lets model hair, lets model clothes, lets model the food they eat,” – what do you want to model? Only the systems that are fun; you need to get these people in there as a basis for interesting choices, and you’ve got the makings of compelling gameplay. Katherine has a great example with Fight Night, where they eliminated the UI and mapped it onto their faces. When you’re boxing you know how well you’re doing based on what they look like, you don’t need a health bar, you see it.

KI: I think in the development process you tend to silo off character by character by character, when its really the space between the characters where the social interaction happens that makes it meaningful for the player. Part of it is adapting the design and production process so that the whole team understands the effects you want to create emotionally.

On Façade

GS: Speaking of which, what do you think about Façade (below)? Obviously its good that it was made, lots of people I ask about it say “well it was interesting, but I wouldn’t buy it.” What flaws are in that design and how could that be improved?

KI: I don’t know if I’m the target audience for Façade, I can’t even watch suspense movies where good things are happening to bad people and its really stressful. The whole premise of Façade is these two difficult people that you’re supposed to help and I just wanted to get out of there! [laughs]

I love Andrew and Michael and I think their work is great, but what’s tricky is that your typical game has crystal clear affordances and feedback loops and they’re trying to create a really flexible space. They lost clarity in the feedback and its hard to know where you are in the system, and that’s not necessarily a bad thing. I think we can all learn as players to tolerate some ambiguity and enjoy that process, but I think that mechanism is going to need a lot of iteration to find the right way to put that in front of the player.

GS: What about the idea that because you’re playing with social dynamics, you already have training? As in, you don’t need to know that jumping on turtle shells causes them to slide all over, you know what social situations are about because those are the rules you’ve been dealing with your whole life growing up in society.

KI: Not necessarily, take Diner Dash for example. That’s an unfamiliar game mechanics and the character expressions were a way of translating that to the player so they know how to game the system. I’m a big proponent of feedback.

NL: And the problem with Façade is that even though it’s a social environment, there aren’t enough constraints to really create a situation, you don’t really know going in who the characters are. You need something to hang your hat on if you want hard fun, if its just easy fun, where you’re exploring the space then you need to be able to explore those conversation trees in a way that is rewarding.

Façade works for some people, they find it interesting, what the characters are going to say next. If they’re going to broaden their audience they need to have more variety in what comes out of character’s mouths that gives the player a sense that they’ve had an effect. I got the feeling that I was just slipping through that space and I’m really sure how and why and what I’m doing. The central thing is, and this is what we help clients with a lot, is there’s a real difference between a technology and a product. Façade feels to me like a very exciting feature that could be incorporated into a larger game. By doing that you would then create that driving scenario. If it’s a Baby Shower for example, [laughs] or something else, you pull in all of people’s expectations of a baby shower, and then you could have humor, where someone’s inappropriate in that context or you can provide the stereotypical dramatic shock. It does cramp down the space but it creates gameplay.

KI: Constraint is design.

NL: Yes, constraint is design, I like that Katherine. The only way you’re really going to get there is to bring in users and have them play and look at that experience and say “hey designer, is this what you intended?” A lot of designers, myself included, become fascinated by the systems that we create, and all the wonderful intricacies that we put into the code and artwork. But that’s not enough on its own, it all amounts to an experience inside the head of the user.