• by Steve Smith , Staff Writer @popeyesm, May 11, 2010

The next game controller you pick up in your living room ... may be you. After decades of fumbling with increasingly complex 12-button, dual-stick, thumb-spraining console controllers, Microsoft is promising to remove the layers of digital abstraction that have been with us since the first desktop mouse arrived. Now, playing an Xbox game could be as simple as walking into view of your TV and starting to swat at incoming dodgeballs, drawing your arm back to pass a football or talking directly to that on-screen elf in an RPG.

The Xbox "Project Natal" is full-body motion control that vaults far beyond the groundbreaking interface of Nintendo's great success with the Wii-mote. It promises to eliminate the controller altogether and, in the process, open up the Microsoft console to broader audiences, revolutionize the way we inhabit digital worlds and perhaps put Microsoft where it always wanted to be - at the center of the living room experience. "It has everything to do with breaking down barriers and getting to the mass market," Microsoft's games division vice president of strategy and business development told Xbox historian Dean Takahashi in an interview at the Natal reveal during the E3 convention last year.

Clearly this high-profile, high-risk project is about more than games, though, as the company explores a new vista in "natural human interfaces."

"If it lives up to the promise of capturing motion and letting people use their bodies, it will be joyous," says David Edery, coauthor of Changing the Game: How Video Games are Transforming the Future of Business and former worldwide games portfolio manager for Microsoft's Xbox Live Arcade. "People will fall in love."

If being the operative word. In order to super-simplify the digital interface, Microsoft is hoping to introduce by next Christmas some of the most complex technology ever planted in our living rooms. Project Natal adds to your Xbox a gadget-packed sensor bar with an RGB camera to capture images and do facial recognition. On top of that there is an infrared "depth sensor" to locate multiple occupants in a room, and a sensitive microphone that will recognize and distinguish everyone's voices. The hardware works with Xbox software to track 48 points on the human body and translate them without lag into on-screen action. At the same time, Natal is promising to recognize and distinguish multiple players, understand facialexpressions, respond to voice commands, and do all of this without being confused by the ambient yells, IKEA tables and the family pets that clutter real-world living rooms.

Natal is a very tall tech order, but its ambitions to open up the Xbox to mom, sis and even grandpa are critical to Microsoft's long-term living room strategy. "There are reasons why the women in households don't touch the Xbox," says Michael Pachter, managing director of equity research at Wedbush Securities. "It seems like a hardcore gaming machine, and they are scared to death of the controller."

The Nintendo Wii became the surprise hit of the current generation of gaming consoles because its simple motion controller overcame console-aversion and made serious inroads outside of the gaming core. By turning the entire Xbox interface into a controller-free series of familiar gestures, Microsoft wants to recruit new gamers, of course. But it also wants to accelerate the Xbox's role as an online portal to its newer Netflix movies and Facebook connections. "I think of it more as a dashboard interface than as a gaming interface," says Pachter. "There will be games that will be fun, but I don't think that is their primary goal."

End Game
Practical demonstrations of what shape Project Natal will take for games and more have been scant in a development process where the Redmond behemoth and third-party partners like EA, Disney and Capcom are on lock-down. Microsoft has shown a Breakout-style ball-at-the-wall game, "Ricochet," where players employ their whole body to swat back an incoming object. In a "Paint Party" demo, you fling paint at a screen, use your body to form stencils and issue voice commands to activate operations. In a racing game under development, the driver simply holds his hands in the air and moves his foot forward to steer and accelerate. And virtual characters in Lionhead Studios' "Milo and Kate" have been shown conversing and responding to players' voice and facial expressions.

The risk and the promise of Natal is that it isn't just another console peripheral. "It is a hugely generic thing that people can take in a lot of different directions," says one developer acquainted with the project. "It is a platform, a serious new way of looking at games." And there is no telling what happens when - or if - they'll succeed in getting people off their couches and deep into the game. When Nintendo's Wii-mote appeared in 2006, its use beyond virtual tennis and the simplest swordplay in Legend of Zelda was not immediately apparent. But motion-control launched an entirely new, bestselling genre of activity and exercise products like Wii Fit.

Matt Story, director at Denuo, says he and his clients are already talking to Microsoft about tying brands to Natal because this is much more than a cool new Call of Duty roll-out. "They are going to get behind it like a console launch," he says. "Obviously when we look at partnerships and marketing opportunities, we want to be with entire programs and IP that will break through the cluttered nature of what is going on out there." Beyond riding the buzz of a major platform launch, however, Natal will have to prove long term that the early adopter gaming base really will be able to evangelize that hardware to Mom, and meaningful genres of games evolve from developers that invite brand integration. "When they have it out for two years, that is when it becomes exciting," says Story, "when you can tie to an actual product. Right now it is name association that is driving buzz."

Just as the Wii and the iPhone demonstrated, getting beyond the gimmicky feel of a new interface takes time for both developers and users. Natal's true potential may lie far beyond the game. All of the figure recognition and imaging technology embedded in the system could take augmented reality to new practical levels, for instance.
Mom and Dad may not want in on full-body Xbox dodgeball, but they may be up for a stroll through a living room shopping experience. AR techniques could be used to try on the latest Lands End down coat virtually. In this scenario, the Xbox ties Microsoft into a massive ecommerce economy. "You could step inside a catalog," says Edery. And that alone raises another set of curious marketing and partnership questions and possibilities. Who gets the rights and the revenue shares from augmented reality shopping?

The Rabbit Hole
Unquestionably, Natal has bigger ambitions than gaming. The Xbox interface has just gotten started streaming Netflix films and Last.fm customized radio through what it always conceived as more of a set-top box than a game console. Gaming on the Xbox (perhaps on Natal, too) has always been a Trojan horse for a much a bigger play. "At the end of the day, Microsoft is thinking of the end game," says Edery. "Xbox was their attempt to dominate the living room. Natal is the next step in turning the device into a platform that can become the focal point of the living room and blow it out to an enormous audience."

Whatever the short- and medium-term iterations of Natal, the excitement it seems to be sparking among developers and marketers comes from its radical re-imagining of the way we interact with computers and the digital world. The two-dimensional layer of abstraction that mouse, keyboard and button-riddled controller have put between us and the digital spaces we create may have been limiting us in ways we can't appreciate until we move beyond them.

The Wii-mote, smartphone and tablet touch screens, augmented reality and now a full-body controller are of a piece. They all move us toward connecting physical movement more directly to virtual actions, or kinesthetics. The 2-D interface distances us from involvement with the digital world we were controlling because it relies on abstract gestures translated into commands. Swiping, punching, grimacing, jumping and speaking to our computers open up new levels of emotional involvement.

"The very nature of our interacting with technology is going to be changing," says Josh Lovison, mobile and gaming practice lead at Interpublic Emerging Media Lab. A gesture-based interface lets programs like exercise trainers or even a CPR teaching tool physically ingrain practical motions in a user through repetition. Natal-like features such as facial recognition could re-invent the ways consumers relate to marketers. "We can move from a cost-per-click to a cost-per-smile or per-hug based interface," says Lovison. "Detecting responses and even measuring and leveraging micro-expressions to observe emotions - there is a wide spectrum. How deep is that rabbit hole?"

Well, no consumers or marketers are headed down any rabbit holes unless and until a wannabe game-changer like Natal actually gets the family off of the couch and deep into the game. Still five or six months from release, Natal gets its first close-up at the June E3 game industry fest. The device has two high hurdles to vault. First, will it work as promised? Second, can Microsoft get this hardware in enough households at this mid-to-late point in the lifecycle of the current generation of consoles? Only a handful of tech journalists have gotten their hands (and other limbs) on Natal, and it is still unclear whether such a complex technology and processing can keep up with anything but the most rudimentary titles.

At the same time that Natal is setting its own bar pretty high, Sony's Playstation 3 platform is countering with a more modest approach to motion control. Like Natal, the "Move" peripheral uses a camera to track motion, but it requires a handheld controller that registers 3-D location and orientation. While also positioned as a new "platform" for Sony, the Move seems to split the difference between Microsoft's radical re-imagining of controller-free interaction and the more familiar mechanics of the Wii-mote. Lovison is un-Moved by Sony's entry. "It is way too complicated. If I am not a core gamer, then the idea is to show how simple you can make it." Story, on the other hand, thinks that having Microsoft and Sony explore motion control could validate the approach and get consumers to address the biggest question mark: Will they get off the couch? "These things will change how we consume media if they are successful and allow people to get comfortable picking up new habits - if I am comfortable to stand up and move in front of the TV."

More to the point, will consumers feel comfortable getting off the couch to go buy the things? A critical problem for both Microsoft and Sony is getting enough of their motion controllers into living rooms at the midpoint in the lifecycle of current-generation systems. Most core gamers (including the hoped-for "evangelists" who are supposed to lure in the family) already own their Xboxes and PS3s. These "platforms" have to be purchased separately as add-ons, so both manufacturers have to make a strong case for current owners to buy in. Nintendo's Wii-mote was always a fundamental part of the basic system the company sold from the start. Sony and Microsoft will have to come up with some serious marketing efforts and bundling strategies to get the critical mass they need to establish true new platforms. "I don't think they will sell that many to the installed based without games," Equity Research's Pachter says of Natal. "To pack it in the [console] box - it will be an overwhelming success." Ultimately, in order to get into the living room, these motion control schemes may have to take the same direct-to-mom path that the Nintendo Wii did - by gaining buzz far outside typical gaming circles. "It will be made or broken based on the talk shows and morning TV," says Lovison.

The biggest gamble of all is that consumers are ready and willing to throw their full bodies into the game of interacting with media. The potential is nothing less than a new dimension (or more) in relating to brands and media. Products can be tried on in the living room. Packages can be twirled and examined, swiped away, unpacked and tested in virtual space. Viewers could have conversations with game characters or be prompted to interact with movies. Media makers of all kinds could engage their audiences in physical movement. The TV experience will have evolved fully from "lean-back" to "all-in."