iPhone X: Apple bets the future of smartphones on Kinect's failed tech

Sooner or later we knew we'd reach this point. Apple's newly revealed iPhone X is the first $1000/£1000 smartphone, arriving with a slew of features and technologies that have already been trialled in mainstream gaming with varying levels of success. The new phone brings with it a state-of-the-art HDR OLED display and features augmented reality gaming capabilities, but the real star of the show is its front-mounted Face ID camera assembly. Yes, remarkably, Kinect technology is back - miniaturised and repurposed, but based on the same principles.

Multiple cameras are built into the iPhone X's distinctive 'notch' that cuts into the edge-to-edge display. This assembly includes a standard camera, IR, a 'flood illuminator' and what Apple calls a 'dot projector'. The basic idea is the same as Kinect though - to use depth sensors to build a 3D model of the user's face, with the other cameras used to create textures that can sit upon that model. The use of IR also means that - like Kinect - the device should work in all lighting conditions.

There's even a degree of overlap in functionality: specifically, biometric sign-in. Microsoft had a vision for a controller-less console interface and on iPhone X, this manifests in the removal of the home button. Just as you could sign-in to your Xbox by standing in front of the camera and waving, iPhone X does the same job in much the same way by lifting the device and tilting it towards your face, initiating a facial scan that unlocks the phone.

And therein lies the key differentiator. Kinect was built to cover as much real estate as possible - in fact, it required a fairly large room in which to properly operate, in order to get the entire human body into its field of view. iPhone X maps just the face, and draws all of its functionality from this one area of the body: unlocking the phone, animating emojis and... well, fundamentally, that's about it. And in common with the first time we saw this miraculous technology, there are key questions about the scale, scope and reliability of its real-world applications.

Kinect was a novelty that quickly wore off, where the effective use-case scenarios for the technology proved disastrously narrow - and as an interface, it was slower to use and far less reliable than the button-based controllers it sought to replace. There are echoes of this in the 'Animojis' Apple showcased last night, which captures audio and facial motion data which are then mapped onto avatars. The whole concept of the emoji is based around one-touch shorthand communication, completely at odds with the far more longwinded Animoji, which is to all intents and purposes, a variation on the video message, or an enhanced voice message. The emoji is also a common standard - it works across all devices, whether iOS or Android-based - the status of the Animoji here is uncertain. The bottom line is simple: will it work in Whatsapp or Facebook Messenger? In key respects, the parallels with Kinect are uncanny: yes, this is fun, but to what extent will it actually be used in day-to-day life? How quickly will we get bored with it?

And then there is the notion of how robust the functionality is and how reliable it is. Place any smartphone down on a table, press the home button and whether you're scanning a fingerprint or entering a PIN, it works. The iPhone X's Face ID system requires the user to physically pick up the phone and look at it, adding unnecessary friction to what is the simplest of tasks. The chances are that if you don't, the phone defaults to the PIN entry system, but this is a regression compared to Touch ID and already, even in the simplest of scenarios, we are seeing the value of the Face ID technology start to diminish. Other limitations are also starting to emerge - initial reports suggest that only one face can be scanned.

But it's the reliability of the system that is the key concern. A fundamental problem with Kinect was the fact that - to put it bluntly - it didn't work all of the time. Apple's own demonstration showed an iPhone X that repeatedly failed to carry out the most basic Face ID function - unlocking the device [UPDATE: Apple itself has offered this explanation for what happened]. Thankfully a second device was on hand that got the job done for the purposes of the on-stage demo, but for those of us who lived through the Kinect era, it all felt rather familiar. Once the novelty wore off, Microsoft's camera actually had the opposite effect to that intended - its shortcomings made you value the standard button. You knew that the required function would activate when you pressed it, you got immediate physical feedback and you got the reassurance that things would really happen. To replace this, Face ID needs to be infallible - and the key concern based on the demo last night is that it isn't.

The echoes of Kinect are perfectly understandable. It's clear that integrating depth-based facial recognition functionality into the iPhone has been on Apple's radar for years. In fact, in 2013, it acquired PrimeSense for around $350m - Microsoft's original partner for the first rendition of Kinect. This was augmented by the 2015 purchase of FaceShift, a motion capture specialist that created the means to track facial movements without the usual marker dots stuck to the target's face. It's taken four years to to reduce the hardware to smartphone-size, indicating the scale of the task and the level of technological achievement the iPhone X's 'notch' panel of sensors and cameras represents. However, last night's iPhone X showcase had to comprehensively answer the question of whether the core limitations of the technology had been addressed - and it's safe to say that all of the nagging doubts remain.

Beyond the Face ID sensor, Apple's big push to do something more with the smartphone came with the arrival of augmented reality functionality and gaming. Again, we saw some exceptionally cool demos here - turning your phone to the skies and seeing the constellations mapped out in real-time on the screen was a cool idea. And there are obvious uses for the tech here in massively enhancing maps-based functionality. Games were also demonstrated with the reveal of The Machines - an AR-based title that placed the in-game world on a real-life tabletop.

Demonstrated on iPhone 8, there were no real issues with the technology actually working as such - the tracking looked smooth and free from glitches. However, the obvious limitation is the requirement of a large, flat surface, plus the ability to walk around it. The jury is out on this one: similar to Face ID, Apple has taken an existing concept (in this case, gaming) and added conditions on how you can use it to something that previously worked just fine. The end result is undoubtedly cool, but the gaming applications seen thus far here and with other AR-based systems seem somewhat at odds with the idea of a device designed to do practically anything, anywhere.

Elsewhere, the design of iPhone 8, 8 Plus and iPhone X demonstrate that Apple has not taken its eye off the ball elsewhere. The A11 'Bionic' processor common to all of the new devices finally sees Apple fully embrace the many-core phone era, with two performance CPU cores and a further four high efficiency cores that can work in tandem, offering a potentially huge boost in performance.

Leaked Geekbench benchmarks suggest that the iPhone X offers 2x the single core performance of the Samsung Galaxy S8, and a 63 per cent increase in multi-core power - on top of the benefits of a slimmer, faster OS. Meanwhile, the A11's graphics core - the first designed by Apple itself - is said to offer a 30 per cent increase in performance over the A10.

There are also big improvements in the camera, with Apple revealing some remarkable functionality - real-time changes in lighting and the ability to pull the subject out of the background (features that work on iPhone X's front-facing camera too). Meanwhile, the long overdue arrival of wireless charging functionality on all models can only be a good thing. Storage is still an issue though - in the age of 4K60 video recording, 64GB of storage isn't in any way good enough, and to get the most out of the phone, 256GB is essential. Of course, as always, expandable storage is off the table.

The performance and camera features are, by and large, common to the iPhone 8, 8 Plus and iPhone X revealed last night, but it's the $1000 device that is Apple's big visionary statement - an effort to break away from the cycle of iterative updates that have made the smartphone market seem just a touch too predictable over the last few years. The edge-to-edge screen and OLED and HDR features are a big deal, but there's a strong sense of déjà vu surrounding the Face ID system. Kinect had the exact same vision: to offer a revolutionary, new, improved interface, but the actual user-experience was slower overall and lacked the rock-solid effectiveness of the good, old-fashioned button press. Based on what was shown last night, the world's first $1000/£1000 phone looks like a beautiful device, but is miniaturising Kinect really the best way forward? Based on the unconvincing demos shown last night, the future of the smartphone is still up for grabs.