Digital Foundry vs. OnLive UK

Performance

Tied in closely with the concept of low latencies is general performance. In our infamous article on how the system can't possibly work, we expressed surprise that OnLive would target 720p at 60 frames per second, bearing in mind that the internet standard of 30FPS would provide twice the amount of bandwidth for image quality and would tally closely with what console gamers currently experience.

The reality is of course that reducing lag as much as possible is key and every millisecond counts - be it scientific, "human perceptual" or otherwise. Put simply, the sooner OnLive pumps out a frame, the sooner it will get to the player and the more responsive the gameplay generally tends to feel. That is the premium, and it's clear that the firm values frame-rate over picture quality.

In our US analysis, we saw that relatively light games like Batman: Arkham Asylum sustained very high frame-rates, while others like DiRT 2 and Assassin's Creed II struggled with frame-rate consistency resulting in some awful input lag and a very inconsistent look and feel to both games.

In short, OnLive delivered a 720p60 video stream, but the games themselves weren't always running at this optimal frame-rate. 14 months on, things seem to be much the same - titles we tested like Warhammer 40,000: Space Marine and Just Cause 2 seemed to possess fluctuating frame-rates that made for inconsistent gameplay - and could be in no way described as a 60Hz experience. However, there are signs that some developers are optimising for OnLive's target platform.

DiRT 3 appears to be the game of choice for OnLive to demonstrate to journalists at the moment, so we took a look at its performance profile. Bearing in mind that DiRT 2 wasn't a particularly pleasant slice of OnLive gameplay, we were generally impressed. Here we put sections of streaming gameplay up against the Xbox 360 version of the same game.

Apologies in advance for the relative brevity of these clips: frame-rate analysis is all about counting unique frames. Usually, Digital Foundry tools take care of this programmatically, but the fact is that OnLive's macroblocking is so pervasive that our algorithms only pick up around 30 per cent of the duplicate frames - the rest are noted by eye, meaning that we have to check every frame manually (ouch).

While the 60FPS fluctuates, and drops significantly when multiple cars are in play, the fact is that DiRT does look remarkably smooth on OnLive, and the frame-rate advantage over the Xbox 360 game we've compared it with is obvious. However, doubtless you have noted that a lot of post-processing effects are shorn from the OnLive version in order to facilitate the speed boost and eye-candy like reflection effects on the cars are absent too. There's no anti-aliasing either - a bit of a blow after the almost pristine 4x MSAA we see on the Xbox 360 version.

Moving on to Deus Ex: Human Revolution, we undertook two tests - some indoors combat, where frame-rate generally tends to be pretty solid on console, and then an outdoors test where we approach the police station in the first hub, which sees a significant performance impact on both Xbox 360 and PlayStation 3. Generally speaking, the console games operate with a 30FPS frame-rate cap, while OnLive strives to hit 60FPS - with variable results. Indoor combat is remarkably smooth, but there is only a small performance boost in the outdoors traversal test. Also noteworthy is that these comparison tests clearly show that Deus Ex's remarkable lighting system has been significantly pared back.

OnLive's Steve Perlman reckons that a game like Deus Ex would be allocated to a more powerful server in the datacentre, but all of the visual compromises we see here strongly indicate that this game is running on the exact same platform as all the others we tried - and that this platform is remarkably similar to the one we tested last year. Financially and logistically it makes far more sense to settle on a "journeyman" PC spec that is economical and runs all games and, funnily enough, that's what developers who've ported their latest games to OnLive tell us too - though we do understand that plans are afoot to move onto a more powerful platform at some point in the future.

Overall, performance on OnLive is acceptable and in certain stand-out cases it's clearly an improvement over the current-generation consoles. However, as we said in our original OnLive piece, there's a reason why developers tend to cap at 30FPS or go all-out to hit 60: it's all about the consistency of the experience. In comparison, OnLive often feels like it's all over the place and this is sadly reflected in the shifting input lag. We see some signs of targeting the 60FPS the service promises with titles like DiRT 3, but Deus Ex is a pretty good example of the kind of inconsistency we generally see in a majority of titles. We get the idea that developers simply don't have the inclination, incentive, time or budget to give OnLive games the same level of optimisation their console counterparts receive.

Hitting 60 is the key to keeping latency low, so we would hope to see a greater push for optimisation on future OnLive titles, or of course the firm could deliver the 2011-class servers it talks about in its PR: 720p at 60 frames per second should be child's play even at close-to-max settings on all but the most technologically advanced titles, even with graphical hardware along the lines of the inexpensive GTX460, which could also take care of OnLive's 1080p aspirations too.