In Theory: How Blu-ray players could host cloud gaming

Whether we like it or not, whether we're ready or not, cloud gaming is coming. The idea is simple - instead of buying consoles or high-end PCs, instead we log in to servers many miles away hosting the requisite hardware and while our control inputs are transmitted across the net, video streams of the gameplay are beamed back over IP. Say goodbye to the console and indeed piracy as we know it, say hello to the dumb terminal.

While the concept may still seem implausible, the technology that powers the likes of OnLive and Gaikai is now a known quantity. Basic human ingenuity, combined with an enormous Space Race-sized piles of cash has overcome many of the key concerns, and the astonishing truth is that future Blu-ray players, HD cable receiver boxes and even HDTVs themselves could easily handle cloud gaming video streams. The browser plug-ins, Flash players and "microconsoles" we've seen thus far are just the beginning.

Key to this mammoth convergence of technology is the video compression system that makes it all possible: h264. It has swiftly become the de facto standard video codec for all major HD transmission systems in the wake of the DVD and MPEG2 era. Blu-ray uses it, satellite HDTV systems like SkyHD use it. So do the cloud gaming systems and therein lies the common ground.

While OnLive talks about proprietary compression technology, insiders insist that the firm is using h264 in line with our thoughts in the wake of the firm's launch at GDC 2009. OnLive's main competitor, Gaikai, is typically more transparent, with David Perry's demo utilising a Flash player explicitly mentioning the codec's usage on-screen.

However, to sound a cautionary note, it should be explained that the h264 standard has been tweaked and enhanced to allow for ultra low-latency encoding, so it's not 100 per cent identical to the traditional iterations we've seen thus far. This is the human ingenuity we're talking about that has seen the best existing video compression scheme repurposed to work with streaming gameplay.

The ways and means with which this has been done have been explained at length by key codec developer Jason Garrett-Glaser, who has added the necessary encoding technology to the open-source compressor, x264, and is currently working with an unnamed OnLive competitor.

There are two major new elements that have been added to h264 to make gameplay streaming possible. The first is increased parallelism in the encoding. The image is split into slices, which are then encoded at the same time, thus minimising this particular latency hot spot. According to Garrett-Glaser, OnLive uses a similar, if less efficient system, where each frame is divided into 16 rectangles which are dispatched to 16 individual video encoders, effectively making the encoding process 16 times faster than it would be otherwise.

Garratt-Glaser maintains that the "slices" technique offers more efficiency because each slice can still draw upon video information from other areas of the frame. He has suggested in the past that OnLive's discrete encoder approach doesn't offer that flexibility.

The second technological leap is known as "periodic intra refresh". Currently, h264 along with most compressors, works on the "group of pictures" (GOP) principle. The video stream kicks off with a fresh key frame, or intra frame, with all the major information required to compose the image contained in that one packet of data. Subsequent frames are encoded only as changes in relation to other frames in the group, before a new intra frame is beamed down to repeat the process anew.

All fine and dandy, but no good for gaming: a large buffer is required to get decent efficiency and ideally it would contain video information from the "future", only available by buffering frames and adding drastically to the lag. Periodic intra refresh helps solve this issue by dividing the screen into columns with new key frame-style information constantly being updated column by column.

So, can conventional h264 decoders work with these enhancements? The evidence suggests so. Periodic intra refresh has already been deployed in live streaming using Flash's standards-compliant (if rather ropey) h264 decoder (presumably within Gaikai's own Flash player) while the parallel slicing technique, in a slightly different form, is already an element of the Blu-ray spec.

In short, there's plenty of evidence to suggest that the bespoke h264 playback hardware in millions of chipsets already out there could well be compatible with cloud gaming video streams.