In Theory: why cloud gaming is coming home
Project Shield and Steambox bring streaming gameplay to home networks - and next-gen console could follow. Digital Foundry asks why.
Having failed to find a sustainable business model in its original form, cloud gaming technology is coming home. By the end of the year, several major manufacturers will be offering players the ability to stream gameplay from PCs to mobile devices and set-top boxes in the home - with next-gen consoles perfectly positioned to follow suit.
Nvidia has already revealed its plans with the intriguing Project Shield announcement at CES last month. Integrating a state-of-the-art mobile processor in the form of Tegra 4, Shield not only allows for Android gaming on the move but also connects via WiFi to a GeForce "Kepler" GTX-equipped PC, allowing for any game to be streamed over a home network onto the handheld. Valve is set to follow suit with its entry-level Steambox, which in concept sounds very similar indeed to the OnLive microconsole - a low-power device designed for media streaming and equipped with interfaces for gaming controllers.
It's a fascinating turn of events - two very different takes on gameplay streaming tech from two major players in the industry. Valve believes that the traditional PC form factor doesn't really work in the living room, so the entry-level Steambox is a small, unassuming set-top box style arrangement that simply relocates gameplay hosted on the bedroom/office PC into the lounge. For its part, Nvidia retains the same networked PC element, but adds in Wii U-style functionality, allowing gamers to play their PC titles anywhere within range on what it describes as a Retina-level display.
What is common to both of them is their reliance on the principles of established cloud technology. The server hosts the game code, the output image is then encoded into a compressed video format which is transmitted over the home network to the target client platform. Here, it is decompressed and displayed on-screen, with player inputs beamed back to the server. We already know how Shield works in concert with Nvidia's Kepler technology - the firm's latest GPUs incorporate a hardware video encoder attached directly to the graphics core, designed for minimal latency. Rather than relaying imagery to the graphics card's video output, it is directed to the encoder then beamed out via your home network to the Shield portable.
"Every major piece of upcoming gaming technology features hardware video encoding hardware - manufacturers are preparing for gameplay over IP and the transition kicks off on home networks."
We'd put good money on the entry-level Steambox operating on the same principles. Quite where (and indeed how) the encoding process takes place isn't known, but as both AMD and Nvidia cards possess hardware video compressors in their latest cards, it may simply piggyback onto existing technology. Alternatively, the x264 software encoder could carry out the same task on the CPU, provided it is strong enough to run video encoding and gameplay in parallel (those who live-stream their PC games will know the challenges here).
Home streaming: a preview of future cloud performance?
Migrating the principles of cloud technology to a home network has the potential to solve many of the issues that compromised the immensely variable OnLive experience. By eliminating the vagaries of the internet - not to mention the various issues that can kick-in between the cabinet on the street and the client hardware at home - handling latency becomes a much more manageable affair. Image quality concerns can also be more thoroughly addressed: while encoding efficiency is always going to be compromised to a certain extent when compressing video in real-time, a home network offers up the kind of bandwidth OnLive and Gaikai could only dream of, more than mitigating the impact of poor quality video encoding.
Nvidia's Project Shield demos look impressive enough, but the chances are that you may already have a system in your home now that operates on similar principles, handing in excellent image quality and performance: Nintendo's Wii U. On a very basic level, the core principles behind the GamePad's display are almost identical - the base station console contains a hardware video encoder that compresses the image and beams it over a WiFi-like connection to the controller, where the data is decompressed and rendered on-screen. It's an elegant and simple solution - break open the GamePad and the internals amount to a WiFi-like transceiver provided by Broadcom, minimal logic and an LCD in combination with the traditional gaming controls. Image quality is generally very good, but the picture is a touch washed-out and artifacting can be seen but only if you look closely enough.
Latency is excellent - in our testing, the GamePad operated with a 33ms lag, exactly the same as the PlayStation 3D monitor we had the Wii U plugged into via HDMI. For Shield and Steambox, that latency could be increased if data passes through a router, of if the PC "server" is itself connected to the router via WiFi rather than a hard-wired LAN cable. We're also curious to see if the increased resolution has any impact on image quality and performance: Shield will be streaming 720p up against the 858x480 resolution of the Wii U GamePad, and we'd expect the Valve "Streambox" to follow suit. However, at his recent DICE keynote, Valve's Gabe Newell seemed very confident indeed that lag would not be an issue here - something we look forward to putting to the test.
Enter the next-gen consoles
This new home-focused iteration of cloud technology isn't going to be restricted to PC - the next generation consoles are invited to the party too. Both the new Xbox and its PlayStation competitor feature hardware h.264 video compressors built into their main processors, allowing for the streaming of live gameplay with no impact to system performance. The inclusion of this tech is there for a number of reasons. Rumours are already circulating that Orbis continuously records the on-screen action, with the ability to share the recording with others at any given point. We're fairly sure that next-gen Xbox will offer equivalent functionality since it too includes a dedicated video encoder, but that self-same hardware is perfect for positioning next-gen consoles as home gaming servers.
However, it's Sony's Orbis project that perhaps offers up more immediate possibilities - even if we factor out its plans for its recently acquired Gaikai technology (which perhaps we will hear more about this week). The platform holder has already released a complete, mobile cloud streaming client very much along the lines of Project Shield. On paper, PlayStation Vita - which could do with a shot in the arm - has all the necessary hardware required to operate as a GamePad-like device for the next generation PlayStation. Internally, it features h.264 decoding hardware that's more than fit for the purpose, it has excellent physical controls, and the 960x544 OLED screen is simply magnificent. It even has similar touchpad functionality to the upcoming Orbis Dual Shock 4 too. In theory, it could hook up directly with the new Sony console in a similar manner to the Wii U/GamePad WiFi connection, or operate via a router. A bespoke Remote Play app could be used to provide a standard cloud streaming interface (a PlayStation equivalent to the Android OnLive app if you like), but obviously with Vita's capabilities, the platform could offer so much more - the recent LittleBigPlanet update which interfaces the handheld directly with PS3 demonstrates the benefits of coding directly to the hardware.
The big disadvantage Vita has is that it's not a standard controller out of the box in the same way that the GamePad is for the Wii U, making it far less attractive to developers - after all, why spend precious resources on functionality only a minority of the userbase will use? The good news here is that a next-gen Remote Play should be entirely OS-driven: game-makers would simply need to flip a switch to enable it, or indeed disable it. That's exactly the same set-up we see with the PS3's existing Remote Play functionality, the difference being that the current-gen console has no hardware video encoder - it relies on a Cell SPU to do the job, and those resources can't be guaranteed. The resultant variable performance has seen the feature virtually ignored, its potential only explored by those in possession of hacked PS3s.
"Sony is exceptionally well-placed to make the most of streaming technology and it has already experimented extensively - albeit with poor results - with its Remote Play system."
Vita is a great fit for home streaming in other respects too. Its 960x544 screen is a quarter of the resolution of target Orbis titles, meaning that game images would be downscaled before encoding - so not only would players get the benefit of full native Vita screen resolution, image quality would benefit from an effect similar to 2x super-sampling anti-aliasing. Clearly it's not as good as a native 1080p presentation, but it should still look sweet - you can see a similar sort of effect on Wii U, where native 720p framebuffers are downscaled to the 480p GamePad display, smoothing out aliasing issues.
So, if Sony is well set for home cloud streaming, where does that leave Microsoft and its next-gen system, codenamed Durango? Well there's no doubt that it's capable of acting as a central gaming server in exactly the same way as Orbis, but sources suggest that a "second screen" GamePad-style strategy could well be more along the lines of an extension to its existing, under-used Smartglass tech, which would greatly benefit from the ability to receive streaming video - something that's unworkable for games on the current Xbox 360 set-up. However, from what we've managed to piece together, Microsoft's approach to streaming in general is somewhat more complex. An Apple TV-style unit based on ARM architecture, running Windows RT and perhaps featuring Xbox branding is under discussion. This would be a very basic games machine running Windows 8 apps, but geared more towards streaming video services like Netflix.
This piece of kit could operate as a standalone unit of course (an Apple TV competitor if you like), but it could also work just as well as a media extender on a home network, linked to Durango. In essence, the strategy here would be a reversal of Valve's ideas for its entry-level Steambox bringing remote PC gameplay into the lounge: Durango would take centre-stage in the living room, but the ARM box would allow for media - and gameplay - to be transmitted around the home. It sounds like an intriguing idea in theory, the only stumbling block we can think of being Microsoft's strong push for developers to integrate Kinect functionality wherever possible into next-gen gameplay. Replicating camera functions on an extender would be problematic, not least in terms of streaming all the data from a secondary camera back to the host console in the lounge.
Just one question: why?
OnLive didn't work particularly well and the company descended into bankruptcy, while Remote Play was often a laggy mess with virtually no support. Apple's AirPlay system operates on similar principles and while it "works" for gaming in theory, performance there isn't exactly sparkling either. It's safe to say that the entire notion of streaming games over network infrastructure - whether at home or over the internet - hasn't exactly wowed us thus far. Yet we now have Valve lining up entry-level streaming boxes and Nvidia launching its own handheld with a strong gameplay over IP component. At the same time, we have Sony buying up Gaikai and allegedly integrating its services - somehow - into its next-gen PlayStation offering. The question is, why?
OnLive's entire strategy seemed to be based on becoming the dominant brand in cloud gaming with all the undeniable advantages that represents - a strategy that saw Steve Perlman's outfit burn through cash at an extraordinary rate with very little in the way of results. The problem was that the infrastructure simply wasn't there to make the experience look nice, or play consistently well. Hampered by dodgy price-points for its software and held back by a lack of exclusive games that leveraged the advantages of server-based architecture, the cards were stacked against it.
"OnLive proved that the infrastructure is not there yet to sustain a good experience. Gameplay over IP at home gives us an early preview of how the tech could look once the major problems are finally overcome."
Our take on the emergence of Project Shield and the entry-level Steambox is that manufacturers are trying to build audiences for streaming gameplay first by showing how the technology can work looking at its best, getting them used to the core concept of playing on client hardware. Over time the broadband infrastructure issues will resolve to an acceptable degree, making the OnLive model of streaming gameplay over IP an attractive upgrade over replacing the bedroom/office gaming PC once it becomes too long in the tooth for more advanced games. Gabe Newell has hinted at datacentre-driven gameplay in the future, while Nvidia has already revealed that it's planning to roll out services based on its GeForce GRID cloud architecture. Sony and Microsoft probably accept that at some point within the lifetime of its next-gen systems, internet streaming gameplay could become viable and will want to be ready for that. Factoring in its Gaikai acquisition, Sony may well even decide to take point here.
In the here and now we welcome the arrival of the upcoming range home network gameplay streaming systems - Valve's approach to relocating PC gameplay to the living room is more than overdue, while Wii U has demonstrated the worth of being able to decouple games from the HDTV to play at leisure around the home. From a technical perspective we can't wait to put these systems through their paces - at the very least it should give us an enticing preview of how viable cloud gaming can be once the principle infrastructure issues are finally overcome.