Q&A: NaturalPoint On TrackIR's Head Motion Tracker

NaturalPoint is an Oregon-based company working with image tracking technology and computer control devices. TrackIR is the company's main focus; a USB PC controller that tracks the users head motions through six degrees of 3 dimensional movement using reflective markers placed on a hat, or headset. Currently, the majority of the approximately 60 games supported by the unit tend to be flight simulators and racing titles that use TrackIR to control in-game camera, though this list does include high profile titles like EVE Online, Colin McRae Rally and Microsoft's upcoming Flight Simulator X. "The key to the whole experience is how your monitor comes alive when your head motion is instantly amplified into view control," explains NaturalPoint developer relations manager Warren Blythe. "I mean, you're not just triggering animations -- you're actually controlling everything in real time. It's hard to convey in text how rewarding it is to use your head to control your view." With the fourth iteration of the unit offering more flexibility in terms of its ability to track the user's movements, the company is hopeful that developers will find different ways to implement TrackIR. Gamasutra contacted Blythe via email to discuss TrackIR, its current level of usage within the industry, and his hopes for the unit's possible usages. When was NaturalPoint established? Jim Richardson (President/CTO) and Birch Zimmer (Lead Programmer) formed the company back around 1997. Jim's cousin had been completely paralyzed after his hoodie was caught on the handle of a closing garage door, and his family was frustrated by the high cost of laser eye tracking devices at the time (around 15-20 thousand). So Jim and Birch ended up dropping out of college to form the company and bring low-cost high-performance optical tracking devices to the masses (their initial "ION eye tracker" cost about $2,500). The company transitioned over to head tracking in the year 2000, when they realized we could sell awesome head tracking tech for a couple hundred bucks, while the nearest competition was over 2 thousand. How does TrackIR work? Basically, TrackIR lets your computer know exactly how your head is moving and rotating in 3D space, which leads to intuitive new ways to interact with a game. Specifically, TrackIR is a specialized infra-red camera which sits on your monitor, captures images 120 times a second, and processes these images into simple data that is sent to your computer through a USB cable. With 3 reflective markers in a certain configuration on your hat or headset, TrackIR can accurately detect your motion in all "6 Degrees of Freedom" (or "6DOF" which describes all the ways you can possibly move and rotate in 3D space). A small application runs in the background to let you adjust the sensitivity of each degree of freedom before it is sent to the game. Currently, about 60 games and simulators have special support for TrackIR, so you can control the view separate from the other inputs you're already using (keyboard, mouse, joysticks, etc.). How many versions of TrackIR have there been, and what advances have there been in the technology? We've released 4 models so far, with continuous improvements. Increased responsiveness thanks to higher frame rates, better smoothing algorithms, and resolution enhancements. We've widened the field of view, so users can enjoy a broader range of motion. At first TrackIR was just emulating a mouse, so it was a big jump to have it supported in games as it's own separate input, and another big jump to add full 6DOF support (because then TrackIR could do a lot of things that just weren't possible with mouse-look). We've also done a lot to improve light filtering and tracking algorithms, so you don't have to worry about interference from reflections that occur on your glasses, or a shiny forehead, etc. How quickly does this sort of tech become outdated? As fast as we can make it outdated? It's tricky to nail down from a tech perspective, because the system is potpourri of special components, carefully combined for max performance at min cost. We don't talk about it as much, but alongside the hardware design and software improvements, we've been honing the ways we effectively acquire and mix components. I think gyros and accelerometers are already outdated, since they just can't guarantee the level of fidelity that optical offers, but they still seem to be popping up in the latest console controllers because they've come down in price. This cost-effective mix keeps everything moving very fast. From a broader perspective, excellent 6DOF tracking without markers of any kind may be possible within 5 years. This seems like the Holy Grail at the moment for any optical tracking device. But we doubt that quest will end up with a solution that's as fast, precise, and affordable as TrackIR. For example, we've improved on our initial device for 5 years now, but that first version still offers more precise tracking data than an EyeToy. For game players, motion tracking is already available and will just be refined with different tech. So we're working to stay well ahead of the curve behind the scenes, and reinforce that premium head tracking isn't going away any time soon. What is the learning curve for the device? Anyone can pick it up and go in minutes, because it's intuitive. You already lean forward to zoom in, for example. TrackIR ties into the all little jumps, quirks, and subtle head rotation you already perform while playing a game. But it can take 2 days or more to tweak your personal settings for how your head motion is amplified. Different people just like different levels of sensitivity. Some like to start at a very slow one-to-one ratio, so it's like looking through a window, and then slowly ramp up to a point where they can turn 180 degrees in the game by turning their head less than an inch. Some really like to tweak all their settings individually, deciding just how little they want to lean forward to fully zoom, verses how much they want to rotate left and right to fully turn the view. A few like to set dead zones that ramp up to full sensitivity, so the device only starts to work after they've moved a couple inches, but then moves faster the further they move. There's a ton of ways to customize the motion. What has fan reaction to TrackIR been like? Our fans, especially the flight sim audience, are incredibly loyal and usually very vocal online. They're generally excited to be part of a premium community that is looking forward to new ways to play, instead of looking backwards to overly simple control abstractions of the past. Price wise, we get very few complaints about the TrackIR 4:PRO having an MSRP at $179.95. For a while we sold a version of the TrackIR 3 that only ran at 80 fps for $30 less than the 3:PRO (at 120fps), and nobody bought it. It was less than 10% of TrackIR sales. We still sell 3:PRO on our website for a lowered price of $119.95, and its sales are even more eclipsed by 4:PRO. I think when you compare the benefits of a new graphics card, or physics card, or other serious things people acquire for better gaming, the $179.95 cost is rather slim. With Microsoft introducing a great new force-feedback rumble-ready racing wheel at just $149, we're hoping more mainstream gamers will realize that a $40 plastic joystick isn't necessarily offering the premium experience. When the odd person complains that TrackIR offers an unfair advantage (always people who don't have the device), I expect they're not happy with the price. I wonder if a lot of people really just want an auto-win button so they can rack up higher scores without all that bother of "enjoying gameplay." And then I curse them with my ancient black magic. What has developer reaction to the product been like? The developers I've talked to are very supportive. They see so many games that are slightly tweaked versions of last year's title, that they dig the idea of moving game play forward. But we hit a lot of resistance from marketing departments and publishers, because they're worried it'll take more than an hour to add support, or can't see past their game helping to sell our hardware. Especially mass market titles where they're on a tight schedule to deliver a certain pre-sold package. I've had marketing people tell me flat out that they won't show the technology to their dev team, because they don't want them to think about it. Also I think popular opinion for many years was that third party controllers were crummy knockoffs for suckers. The deluge of cheap joystick cash-ins over the past 20 years has dragged the whole idea of third party peripherals into the toilet. It's cool that Red Octane shook everybody up by offering a quality joystick that tied directly into your impulse to play with a guitar. This helps other companies see that intuitive game controllers appeal to the masses. But this has also kind of hurt us, because now marketing people advise us to stop trying to enhance all the current control schemes in their game, and just focus on funding one original game that is only played with head motion. What kinds of support are developers implementing into their games, and how many developers are on board? About 100 developers have the technology and are looking into, or have already implemented, support. The majority are slapping in 6DOF view control for the first-person point of view, because it's quick and highly immersive. A couple have released support for a third-person point of view (because they don't have a cockpit or first-person perspective), and several more developers are looking into this approach. One dev group is exited about controlling character movement with TrackIR (so leaning forward would make you walk forward, instead of zooming the view), and they're talking about freeing up the mouse for a more direct correlation to your hand movements. Hopefully that will spur new innovative uses. I keep hoping someone will use it in a boxing game, or with a motorcycle, so that your leaning affects the physics of what your other controls can do. We'll see. How widely do you see it being implemented? Can you see the device being used for other types of games outside of racing, flight sims and FPS? I think it is possible to benefit any game that involves looking at what you're doing. Across the board, we dream of new uses for FPS, puzzles, MMORPG, RTS, simulations, etc. We've also talked to some serious game developers who just want to see exactly how the player is sitting and reacting to the game play (I'd love to see abstract uses where your posture was quietly considered by the game's AI!). But genres that aren't focused on simulating reality will probably skip it for the next few years, because it just isn't in tune with their game's goals (or their customers requests). At the moment it's a simulation standard and there's a lot of interest from tactical FPS developers, because the device ties right into the goal of realism and immersion. You've mentioned that you're looking forward to indie devs trying something different with TrackIR -- where else would you like to see it being used? It'd be cool to control a player model's head and torso, for dodging bullets and melee attacks and such. And I'm dying to see it used across multiplayer games. Someday a squad leader will ask if his team heard him, and all their heads will bob in reply. Or a role player will chastise her boyfriend for looking too closely at a passing elf. Just knowing that you're actually seeing a representation of where other players are they're looking around, in their far away homes, will be very cool. I'd also love to see TrackIR working alongside 3D joysticks like Novint's haptic 3D Raptor device, or helping to promote DS style gaming on the PC by enhancing Wacom's Cintiq stylus screen (though I guess Wacom has never been interested in gaming). I'd like to indie developers and indie hardware makers were all working closer together to offer new forms of interaction. How are you planning on helping indie developers implement the device? Well, I'm talking to a cool fellow at GarageGames about collaborating on a competition that'll encourage indie devs to do extremely nutty things with the tech. We can then offer all their work to our community for recognition, and maybe help put a few in touch with the right people to help their career. NaturalPoint is in kind of a unique position where we've been around for a while, and know a lot of developers, publishers, and press -- so if someone who loves making games has an idea for a killer app that can only be played with TrackIR we definitely want to help them make a name for themselves. What competition do you have in this field? Do you, for example, view the EyeToy as a competitor? No, we don't see EyeToy or the Vision Camera as competition. Those are webcams best suited to showing you video of yourself on screen, and snapping little color photos. If their gesture games get better, we might worry, but for now they're offering fairly goofy and simple experiments that focus on big movements that take a lot of energy. Those devices are in a tough spot from a control perspective, because they require a notable amount of the processor's attention to process the streaming video. And developers are usually focused on using every inch of the processor they can to eek out the best graphics and physics and AI. So a hot FPS developer isn't likely to throw away 30% of the processor to allow sluggish gesture based view control, because then they'd be losing graphical quality in a market where most consumers equate the best looking with best quality. TrackIR sidesteps all that by just offering pre-processed reliable data, fast. Nor do we see the expensive flight sticks and racing wheels as direct competition, because they're only designed to supplement one kind of game play. We see our competition more as the high end video cards and other hardware components that a gamer doesn't exactly need, but picks up anyway to be sure they have the premium experience. Are you looking at implementing console support in the future? That's a tricky question. It's something we're constantly asked and advised to pursue (by customers, developers and publishers), and as an avid gamer I'm personally very interested. But for the moment NaturalPoint as a company is first geared towards researching and developing the best possible optical tracking hardware we can, for 3 different markets (gaming, accessibility, and integrated MoCap). I expect we're going to maintain TrackIR throughout the end of the year, while pushing OptiTrack forward (our general MoCap product - democratizing the motion capture industry!). Before we jump into consoles we want to see how well Wii delivered on the promise of new kinds of intuitive gameplay, and whether Xbox360+VisionCamera and the PS3+EyeToy did anything to improve gesture gaming. Hopefully we'll have our heads straight on consoles by Christmas!