Digital Foundry vs. Project Natal

Microsoft's announcement of Project Natal at this year's E3 was, for many, the event of the show; an exciting piece of brand new technology promising to revolutionise the way we play games. No joypads, no sticks, no buttons... no controller! Combining a traditional RGB camera with an infra-red sensor, along with a multi-array mic and Microsoft's own voice-recognition technology, Project Natal is all about bringing people into the game, literally. Capable of tracking and interpreting motion data for up to four people simultaneously, the human body becomes the controller. In effect, it's a pretty sophisticated motion-capture set-up designed for the home, and it's exclusive to Xbox 360. It's also pretty much the main reason I've come to gamescom - I have so many questions about the device, and none of the answers have been forthcoming in anything I've read about it so far.

The ever-engaging Kudo Tsunoda is creative director of Project Natal, and is our pilot through the presentation, which is essentially the same as the E3 demo albeit without the inclusion of Peter Molyneux's Milo & Kate. "Xbox is becoming the entertainment hub of the living room and we're getting a lot of new content - like downloading movies, you've probably seen the announcement for Facebook coming to Xbox, Twitter - those sorts of things," he begins. "We have a lot of different content out there that appeals to a lot of different people, and so it becomes very important that we develop a new control system where anybody can use our controls. Right now our current controller is very nice, I like playing games with it, it's awesome. But it's got a lot of buttons and a lot of analogue sticks and it's very hard for some people to use, and it's a barrier to some people for enjoying our console."

Tsunoda reckons, then, that the current joypad is intimidating for some people and puts them off console gameplay. Not only that, but with the system also doubling up as a media player, he talks about how his family would call him over to navigate through the interface using the 360 controller just to fire up their movies. "It became very important for us to create a new control scheme where anybody - no matter what your age or gaming ability - can just get in there can play with Xbox. No instructions, just very simple and easy to use," he continues. "But at the same time we wanted to give extra fidelity for core gamers. So, simple and approachable, extra fidelity - it seems like opposite things, but those are both things we can do with Project Natal."

Onto the first demonstration then: a 3D Breakout game. Powered by Unreal Engine 3, it places you in front of a wall with a barrage of red balls being fired in your direction. The objective is simple: use whatever body part you have available to bash the balls back to the bricks, which breaks them. A suitably energetic demo from Tsunoda ensues. "To play the game all you need to do is move your body," he says. "If you move your body, you'll know how to play the game. It's simple and approachable. I don't care how many buttons or how many analogue sticks you put on your controller: you're never going to get simultaneous movement of your entire Avatar as you can do with Natal."

He's right. What is particularly noteworthy here is that the on-screen Avatar in this demo is closely (if not identically) based on the entire 48-point vector skeleton Natal is processing internally. The way in which the Avatar matches even the slightest of your movements is quite remarkable. It's not just limited to limb movement - your entire body is being accurately mapped. Tsunoda mentions that the dashboard Avatars will also perform in a similar manner, immediately making them more individual, human-like and more closely related to you. I've never been enthralled by the Avatar concept, but this will work in really individualising them and making them seem more like mini-CG replicas of yourself. The applications within Avatar-driven games (and of course via Xbox Live) are potentially spectacular.

"Plus the great thing is that you don't need to stand in just one place and move," Tsunoda continues. "You can totally walk around the room, and your Avatar walks with you. I can walk backwards, I can walk forwards, left and right... it all just works, right?" Tsunoda proceeds to play the demo, jumping about like madman. "I've lost over eight kilograms since I started working on Project Natal," he says. I can believe him. [He really is completely bonkers to watch. The only point he reins himself in is to avoid kicking Rich in the face. - Ed]

A more lethargic performance from myself follows. Curiously, Natal doesn't immediately lock onto me. Tsunoda's assistant runs his hand in front of the sensor, which seems to reset it, and all is well. I'm going to guess that this is actually a good thing: Natal is presumably still attempting to track Tsunoda, but he's no longer 'on the grid'. "You can see also that because the sensor is seeing everything in the room in 3D, the Avatar changes," Tsunoda says, referring to the translucent game character on-screen. "It was an Avatar that looked more like me but has changed to an Avatar that more closely fits how he [he being me] looks." It's simple, good fun, and I'm particularly struck at how well Natal judges the velocity of moving limbs.

It takes some getting used to, but the gameplay has the feel of authenticity about it that you would want. Give the ball a solid whack, or a good kick, and the reaction on-screen is exactly how you feel it should be. Basic, fun, concept-driven stuff like this is exactly where Nintendo excels, and it's good to see Microsoft following a similar line of thought. Just from playing this small demo, it becomes obvious that Microsoft could make a hell of a good Wii Sports-style game with this, and the body-mapping looks so precise that any fitness software it would want to produce has the potential to be astonishingly good. In fact, you can transpose just about any of the motion-driven Wii games and imagine just how much more involving it will feel with Natal. And authentic: no more Wiimote-shaking to cheat the running sections in Wii Fit, for example.

The Breakout-style demo is also interesting in that you would think it will give us some idea of just how much latency there is in processing the full-body skeleton. Trying to get a feel for it during gameplay is difficult simply because there is already 'latency' in the human body itself when moving a major limb. What we do know from previous presentations is that Natal produces its 48-point skeletons at 30 frames per second. Burnout Paradise operates at 60FPS and aims for a 50-millisecond latency, so we can assume that 30FPS scanning gives us a baseline of 100ms. During the Breakout demonstration, I sneak in a move or two specifically aimed at giving us some idea of just how fast Natal updates.

Even running in real-time, the video of me waving my right arm up and down gives some indication of the lag, but it's important to emphasise the "some indication" bit in the strongest-possible terms. The video recording was made at 1080i - 60 fields per second - and this can be interpolated out with reasonable accuracy to a progressive 60FPS video. Frame-counting from moving my arm down to the on-screen Avatar following suit appears to be close to 200ms (12 frames, a fifth of a second). Before we go ballistic on this, it's got to be put into context in a number of respects:

• We don't know the latency of the display, which could be anything between 1-5 frames (it's a Samsung, so it'll likely be towards the lower end).
• We don't know the latency caused by the game code - it varies dramatically, even between 30FPS games.
• Of course, Natal isn't finished yet, and best guestimates say it'll be 14 months until it's out in the shops. There's a very strong possibility that the final production unit will be different.

With that in mind, an educated guess on the optimistic side would be that this Breakout-style demo operates between 133ms to 166ms - effectively the same range of response as Halo 3 working with the joypad. Assuming it's an ultra-quality, low-latency display, that delta shifts to 166ms-200ms - similar to Killzone 2/GTAIV controller lag. Certainly in this test, the latency is noticeable, and I think it's clear to see when the video is running in real-time, but until we get the tech into controlled conditions, preferably in a straight head-to-head with the conventional joypad performing like-for-like tasks (dash navigation, for example), a precise latency figure isn't possible. As I've said, what we see here is an indication of likely performance.