Learning New Moves: AiLive's Wei Yen Teaches Wii New Tricks

Dr. Wei Yen is the chairman and founder of AiLive, a company that deals with artificial intelligence for entertainment. He was integral in the creation of the Nintendo 64 and the GameCube, so it stands to reason that his new project, LiveMove, first revealed in October, surrounds the Wii. It removes the necessity of coding for the Wii’s motion controller, and aims to not only make the development process cheaper and easier, but also to open up the creation of games to those outside the traditional industry.

In this exclusive interview, Dr. Yen talks with us about the ideas and science behind LiveMove, why Nintendo was the first user, and how the tool could bring new blood into the industry. To see the tool in action, check the video on Gamasutra.

Gamasutra: Can you give us some insight into your professional background?

Wei Yen: Related to the game industry, I spearheaded OpenGL in 1989. I was the senior vice president of pro market technology at Silicon Graphics. So yes, I spearheaded OpenGL, and delivered the N64 for Nintendo in 1994. Then again, just as far as the game industry is concerned, I was a founder and chairman of ArtX, which later merged with ATI. And so (at ArtX) I delivered the GameCube for Nintendo. So now it’s Wii.

I was running the product division at Silicon Graphics, so most of my big customers were game developers anyway. So I’ve never been directly in the game industry, but I’d been around the game industry for over 17 years.

GS: So you’ve been working with Nintendo for a long time then.

WY: Oh yeah, for a long, long time. And a lot of senior Nintendo people were my partners for many years.

GS: So did Nintendo come to you to do this LiveMove idea, or did you come to them?

WY: The Wii remote is completely Nintendo’s idea. But both of us are very interested in natural control – to give the player more natural control of their games, and participate in it more naturally. So when I saw the Wii remote a year ago, I really liked it.

And the more natural the control is, the easier it is for the game designer to go straight into working out gameplay, without have to virtualize indirectly through a game controller. But by the same token, you’re making the coding harder. The situation can be so limited that you may not be able to implement some of the ideas you have.

Another thing associated with the expense of coding is that you can’t experiment. In other words, you can’t quick prototype your game design. Game development is very costly, and quick prototyping is very important to the game designer. So if you can do this, and if you can go directly into the gameplay, this will unleash an incredible amount of creativity on the part of the game designer. So to really unleash the power of the Wii remote, you really need to solve this problem. So we talked about it, and Nintendo said, “well?” – and we took the challenge, and we delivered.

GS: So did they propose the idea to you?

WY: Yeah, they asked AiLive if we could do something about the difficult situation with coding.

GS: Why didn’t Nintendo do these tools in house?

WY: (laughs) Well really, I don’t think these are the kinds of tools that just anyone can do. I think we’re way ahead of the industry. This is some deep, deep, deep science. It’s very specialized.

GS: Can you explain how it actually works?

WY: One of our core technologies is context learning. Context learning is basically the ability to let one or many sensors play together, and learn together. We stick an artificial intelligence brain into as many sensors as you want to use, and then you can, through the sensor, set input. And Wii basically learns. So you can train the AI brain. We have this technology for game characters – you can use an AI brain for game characters, you can put one or thousands of these AI brains into characters, and then the designer only has to train them by example.

So how do we do this? We take an AI brain and stick it into a Wii remote. Et voilá! The rest is what you see.

GS: Are you talking about a physical kind of sensor, or in code?

WY: It’s any kind of sensor. Either one, or multiple. It can be radar, or multiple radars, or infrared, or gyro, or all kinds of sensors.

GS: So how do you demonstrate what the remote should be able to do? How do you use this tool to create controls?

WY: The game designer demonstrates, and the AI brain will learn. It just does what the designer wants it to do.

GS: They take this remote, demonstrate the actions, and this is recorded somehow?

WY: Then the brain learns, yeah. You’re basically training it.

GS: Can you give a specific example of something you might be able to do?

WY: Almost anything. Anything you can conceive with the limitations of the hardware. If you have the Wii remote in your hands, whatever you want to do, Wii will be able to do it.

For example if a game designer wants to tape the Wii remote to peoples’ feet, and try to do a kickboxing game, the Wii can play along with how you kick, the direction, the speed, how much force you use, and so on. So basically you can train it however you want. It will learn.

GS: So if the game designer places some examples of kicks, then the Wii can compare this to future users’ kicks?

WY: Yes, when the player plays, the Wii can determine the direction and speed and everything.

GS: So it learns by example, and that’s output as code?

WY: Yeah, it outputs for game code. Wii really is an analog controller. Today’s traditional analog controllers, I personally really don’t like. They’re really digital. But in the natural world, there’s nothing digital about it – everything is analog. The Wii remote has some digital, but also some truly analog parts. Analog control is what the natural world is all about.

GS: This seems like it’s aimed at independent developers, is that what Nintendo’s trying to do?

WY: This isn’t the question you’re asking, but I want to answer something else, if you don’t mind.

GS: Fair enough!

WY: My question is this: why did we take such detour to make this? Because LiveMove is really a detour for AiLive. We basically just did it because of my relationship with Nintendo, and we like the Wii remote and believe in it. So that’s why we took a detour to do this. The question is, why did we want to make this tool for developers?

Basically, big studios have lots of money, but they also don’t take many risks, do they keep developing the same genres they’re familiar with. Usually they don’t venture into new genres. So there are two goals we want to accomplish. One is, we want people to venture into new genres. In order to do that, to create a new genre with some originality, you do need to give people the ability to do quick prototyping. The second goal is, we also believe there’s a lot of creativity – I mean I’ve been in small companies and big companies – I’ve founded five companies so far, and every one succeeded. The difference between the big and small companies is that small companies can take risks. But, small companies don’t have enough resources.

So how so you unleash the creativity of these small companies, and let them get into the game quickly? The problem is how to unleash a lot of these small developers, and how to also let both big and small companies get into new genres. In the big studios today, in the graphics world specifically, it’s a tough job. People want originality, but on the other hand, originality means it will become a blockbuster, or it will fail miserably.

So big studios are trying to walk a fine line, developing sequels. So while on the one hand they’re looking for originality, on the other hand they’re really afraid of doing anything original. So if we can get some individuals, or newcomers, the undiscovered creators, to have the ability to quick prototype, even in the big studios, we might get some small groups of people saying ‘hey, we have this idea – we want to do this.’

So the idea was to create new genres, and to unleash the smaller developers, and the – I hate to use this word – but the secondary development teams in bigger studios.

GS: How will these independent developers apply for LiveMove, do be able to develop for Wii? Is it part of Nintendo’s approval process, or is it something someone can buy and work on on their own?

WY: You can do both. Right now, through Nintendo we already gave out several hundreds of them. But if you’re asking about people who aren’t Nintendo-authorized developers, I don’t see why not, but I don’t see why they would want to do that either.

GS: Well, they might be able to develop something on their own, and submit it to Nintendo to see if they like it.

WY: I don’t think Nintendo is so harsh in terms of approval and licensees anymore. I think the situation you’re talking about is the situation of the old days, I think now, all three companies, Microsoft, Sony and Nintendo are much more liberal in terms of approving licensees.

GS: Well, maybe in some ways. So is this meant for big games mostly, or also smaller Virtual Console-like games?

WY: I personally really want to see many, many little games come out. Wii is the perfect system for casual games, and I also feel that because the Wii remote unleashed not just one dimension, but multiple dimensions. So there’s the possibility of many many new games coming out. Many of them will be casual games. If you’ve seen the demo we did on our website, what you see there, we made in a half hour. We used LiveMove and created a game in a half hour. I think if we put that game on the net for people to play, it would be an excellent game.

But I think there are millions of that kind of casual game people can do. I’d personally like to see lots of those, not only big games, through electronic download, so developers don’t have to put up the inventory money. It makes the threshold much lower for publishing your own game.

GS: Do you know if Nintendo will be using this product themselves?

WY: They already did! Nintendo is actually the first user.

GS: What games did they use it on?

WY: (laughs) I can’t say.

GS: Of course not – I thought I’d ask anyway.

WY: Keep trying!

GS: How long did the development of LiveMove itself take?

WY: Oh, about three months.

GS: That seems really fast!

WY: Well we have tons of technology in-house already. That’s why I said this is a detour for us. We didn’t do this for a living! We do this because we believe in Wii. We believe in Wii’s approach, and we’re passionate about people creating new genres. The game world is getting boring. There are just not enough new genres. We basically did this for fun.

GS: Your main business is AI, so you modified your AI approach to teach the Wii remote?

WY: Yes. More like adapt, but yes.

GS: Do you think something like this would be feasible for the PS3 Sixaxis controller as well?

WY: (laughs) This is tough for me! Ken (Kutaragi) is also a friend of mine. I mean…it was a cakewalk even for Wii – the PS3 would be even easier. I feel that the idea of using this analog control came from Nintendo. So that’s that. I’ll stop right there. I respect Ken, and he’s been a friend of mine for a long time. And AiLive is a commercial company, so we would do this with anybody, but I think the first attempt of bringing this natural control to the game player is from Nintendo. We should give credit where credit is due.

GS: So do independent developers apply for LiveMove through Nintendo, or through you?

WY: Right now they get it from Nintendo, but they can get it from us too. The reason they can get it from both places is that Nintendo acquired a bunch of licenses, just for control. But once Nintendo runs out of those assets, you can get it from us. Even before. We’ll let Nintendo control that first though, until they exhaust those licenses.

So I guess the answer to your question is “yes.” Nintendo grabbing a bunch of licenses is more out of convenience of allocation for them.

GS: How many people have shown interest so far in LiveMove?

WY: We sent out I think 400 in the first two days.

GS: I guess that’s pretty good! Were they mostly small companies, or big ones as well?

WY: I think there are companies you’ve never heard of, but also a who’s who of the industry.

GS: Do you have any estimates for how much time and money this would save developers in terms of prototyping and control?

WY: I think tons of effort, first of all. Second of all, I think some things aren’t even possible using traditional coding. Don’t underestimate the work here. The Wii remote is really capable of operating in five dimensions. If you’re really using them all, it’s capable of five dimensions, not only X, Y, and Z, but also the speed, and the force. Combined with some other digital buttons, it’s actually more than five dimensions.

So if you really have an interesting idea, you may not be able to code it at all. But also the prototyping saves a lot of time and money, because you don’t have to force people to code before you can test things.

GS: Do you think this would ever translate to people being able to develop games themselves, like home development?

WY: Well, we did it! We did that (Balloon Pop) ourselves in less than an hour. That’s the reason why I hope LiveMove will create new genres, casual games, combined with electronic download.

GS: Can you say what kind of code it outputs?

WY: It basically outputs a brain with the data. Literally an AI brain.

GS: That’s hard for me to get my mind around.

WY: Just think about sticking a human brain into the remote. After it learns, it remembers, and understands what he wants. Once he understands what he wants, the player is playing with the AI brain, giving it the data it wants, or not. So the AI brain interacts with the game code.

GS: That’s interesting.

WY: It’s fun! And we can actually put thousands of those AI brains in game characters. But it’s not traditional so-called game AI. Part of my discipline is artificial intelligence. Traditional game AI, for artificial intelligence people, it’s really not AI, it’s really more mechanics. So if you take the idea of game AI and think of sticking that into what AiLive does, you will get it wrong. But if you think of artificial intelligence literally, you’ll get it right.

GS: How did you keep the cost so low for this?

WY: Ah, well the truth is this tool is almost sold as a give away. It’s one or two day's of an engineer’s salary basically, so it’s almost a give away. But we did that because we believe in the power of these peoples’ creativity. We really want to rally those people who have creative ideas but who might be outside the industry. And one way to rally those people is to give them the ability to try out their ideas. For the price, basically two people looked at each other, shrugged, and said “all right.”