Daily news, dev blogs, and stories from Game Developer straight to your inbox
Sponsored By
Can The Xbox One's Kinect Read Your Mind?
The Xbox One's new Kinect camera can supposedly measure your heartbeat, see where you're looking, and guess at your facial expression. Such biometrics open up some cook but crazy avenues for scaling game difficulty and marketing.
3 Min Read
Well, no. Of course not. That's a silly question. Why would you even ask it?
The updated supercamera on the Kinect 2.0 has been shown to be capable of some pretty amazing things, though. Microsoft demonstrated how it can tell where you're looking, estimate your heart rate from the color of your skin, and even infer your mood from your facial expressions. Finally, it has a sophisticated voice recognition system and the ability to see in the dark, which will come in handy when it wants to sneak into your bedroom at night and listen to your breathe for hours on end.
And though it hasn't been discussed, I wonder if the Kinect 's high definition camera could be programmed to measure one other important biometric: pupil dilation. This would be both awesome and worrisome, because while not exactly a mirror into our souls, the eyes can reveal a lot about what goes on in our minds. 
How Kinect sees you: a pulsing sack of meat and emotions.
(Image from Wired's Kinect demonstration.)
Daniel Kahneman wrote in his book, Thinking Fast and Slow, about how pupil dilation is a darn good proxy for mental effort. In a series of experiments he asked people to take a multidigit numer, then increment each digit in the number by one to form a new number. So 348 would become 459. The would then do the same to a new number, using a metronome to do one new number/sum every two seconds.
Try it yourself and you'll see that the task gets very difficult pretty quickly. If not, try adding 2, 3 or even 4 to each digit instead of 1, BRAINIAC. And if you had someone eyeball your eyeballs they would clearly notice that your pupils would steadily grow larger and larger as the mental machinery behind them started to work harder and harder --right up until the point where you gave up, when they would snap back to normal size.
In what sounds less like science and more like an exhibit you'd make a face at while visiting the museum of contemporary art, Kahneman and his colleagues would train a camera on subjects during these experiments and broadcast an enormous image of their eye onto a television in the hallway so that the pupils were about a foot wide and dilation was easy to measure. The results were pretty consistent: the more mentally taxed we are, the bigger our pupils get.
If the Kinect (or any camera) could detect pupil size, it would open up a whole new level of scaling game difficulty. A puzzle game could be made more and more difficult until you're taxed just the right amount to get you in the zone --something psychologists call "psychological flow." Imagine playing a rhythm game like Guitar Hero and having the game adjust the speed of the note highway until you're pushed just to the brink of your abilities based on how hard you're concentrating on the task.
The Xbone is Watching
Or what about knowing when to offer you a helping hand? If the Kinect can tell the point at which you've given up on a puzzle or sequence because your pupils shrink back to normal, it might offer you a hint. Possibly in a condescending tone.
Another, more unsettling implication would be that if the Kinect could tell when you are stressed and mentally taxed, it could use that opportunity to sell you stuff. Willpower is like a muscle that can be exhausted by any mental activity, and when it's depleted we're more likely to do dumb stuff like make impulse purchases or, one might imagine, place an impromptu Skype call to an ex-boyfrined when we really should know better.
Jamie Madigan writes about the psychology of video games. Read more at www.psychologyofgames.com and follow him on Twitter or follow him on Facebook.
Read more about:
BlogsAbout the Author(s)
You May Also Like
