Hands-on with Assassin's Creed Odyssey on Google's Project Stream
How good does it look? How well does it play?
On October 1st 2018, Google revealed the existence of Project Stream, an early demo "pushing the limits of streaming technology". Many in the industry - including Microsoft - believe that streaming gameplay over the internet is a key part of the future of gaming, and Google has shared some of its vision with a select few beta testers, including Digital Foundry. The demo itself is a collaboration with Ubisoft, with Assassin's Creed Odyssey taking centrestage as the game of choice.
But before we continue, what's unclear at the moment is the extent to which this demo represents any kind of end product and mystery still surrounds the system. What we do know is that Google is working on a gaming platform codenamed Yeti. Phil Harrison - a key figure in the success of PlayStation - is involved, while the brilliant Dr Richard Marks (creator of PlayStation Move, EyeToy and other innovations) has also joined the firm. Our understanding is that Yeti is a bespoke platform, built on Linux and using Vulkan as the graphics API of choice. We are also told by sources that there will be deep integration with YouTube, not just in terms of infrastructure but also in being able to leap from watching a video into playing a game. Whether these innovative ideas will make it into the final product remains to be seen, but suffice to say, Yeti may well be a major next-gen contender, especially with that kind of backing.
So is Project Stream also Project Yeti? Are we actually looking at a full port of Assassin's Creed Odyssey running on the new platform? Or is the demo simply testing the streaming side of the equation - compressing each frame, beaming it over the internet and decompressing it for the end user? Right now, we don't know - and neither Google nor Ubisoft are telling.
Right now, the demo is accessible to the lucky few simply by accessing the Project Stream website using Chrome - yes, a full triple-A game streams via an everyday browser, and it even works on super-cheap Chromebook laptops. The streaming appears to be 1080p in nature and while the compressed nature of the video makes precise comparisons difficult, the Stream rendition of the game does appear to compare fairly closely with the PC version of the game running at the ultra high preset. There is some evidence that level of detail may have dropped down a touch compared to PC's best, but overall, the visual feature set here is impressively realised on the cloud and seems to offer a mixture of settings shared between PC and Xbox One X.
The streaming video quality is decent enough, but as the zoomer comparisons below demonstrate, it's not quite a pristine image. In common with all game streaming systems, the combination of fast motion and colourful imagery causes obvious macroblocking while banding on graduated colours - like the blue sky, for instance - is also evident. Perhaps not surprisingly (bearing in mind the PC version's prodigious appetite for CPU power), the Stream version of Assassin's Creed Odyssey runs capped at 30 frames per second. However, the video transmitted to the user is 60fps, and in detail-rich scenes, the duplicate frames can be seen to resolve more detail, adding a very, very slight strobe to image quality.
Picture quality is not bad overall, and will look great on smaller tablet and smartphone screens - undoubtedly devices Google will be looking at to support. The Stream demo recommends connections with 25mbps of bandwidth - we exceeded that somewhat with a 200mbps connection for our testing and the hope would be that the system will scale according to the capabilities of your broadband, delivering higher image quality where possible.
All of which leads us on to the inevitable question of latency. To the casual player, the Stream rendition of Assassin's Creed Odyssey feels similar to the Xbox One X release. It's an interesting choice for a streaming demo, as the game itself doesn't require instant response, and the AC titles have historically had a fair amount of in-game lag. Adding a cloud system to the equation involves compressing every frame, transmitting it over the internet, decoding it client side and synchronising it with the screen refresh. In short, there are plenty of parts of the pipeline that will add extra delay between button-press and on-screen response.
We used the old-fashioned approach to measuring latency with Project Stream - we pointed a 120fps camera at the pad and the screen and counted the frames between pressing a button and seeing the resulting action play out on-screen. We then compared that with doing the same thing on the PC version capped at 30fps and 60fps, then repeated the experiment on an Xbox One X. All of our tests were carried out on an LG C8 OLED display, which has a well-documented processing latency of 21ms.
| Project Stream | PC 30fps Cap | PC 60fps Cap | Xbox One X | |
|---|---|---|---|---|
| Latency (Minus Display Lag) | 179ms | 112ms | 79ms | 145ms |
It's not so much the latency measurements here that are important - lag can change in-game based on performance and even the kind of action you choose to measure - but rather the relative values. Project Stream runs Assassin's Creed Odyssey at 30fps, and here we see 66ms of additional latency up against the PC version running at the same frame-rate. This increases to 100ms up against the PC code operating at 60fps. This is not surprising as it's accepted higher frame-rates reduce input lag. The surprise here is that Xbox One X (tested at both 1080p and 4K output) is only 33ms faster than Project Stream, which would be class-leading cloud performance.
These measurements demonstrate why cloud systems like GeForce Now target high frame-rates - the faster you run the game, the more lag you cut that you can offset against the extra latency added by the streaming technology. In a best case scenario, you end up with a game delivering the fluidity of 60 frames per second, albeit with the latency you might associate with a 30fps console game.
Were this strategy to be deployed on Stream with the Odyssey demo, our numbers suggest that you'd end up with end-to-end lag on par with Xbox One X. As it is, locking to 30fps on the cloud isn't optimal - you adjust quickly and it is playable, but it's not ideal and would be more keenly felt on other types of games, like fast action titles.
Of course, we have no idea of the extent to which the numbers we have today are representative of final performance. However, the more you look at cloud latency across multiple systems, the more that 60-70ms of latency occurs. Our PlayStation Now numbers follow the same trend, while this exhaustive test by Battle(non)sense, also shows similar overhead across a bunch of cloud gaming systems. With all of that in mind, I do wonder how much overhead can be removed and whether the 60ms latency barrier can be breached.
Of course, this is all speculative at the moment - and that applies to impressions on game performance too. It's extremely unlikely that Google would be showing its hand in terms of system capabilities this early (so don't read too much into AC Odyssey's 30fps cap) and as mentioned previously, we still have no idea if the Odyssey demo is using the system spec shared with developers working on the Yeti platform. If it is, all we can really take away from this is that Ubisoft has been able to accurately port one of its most advanced titles from Windows and DX11 across to Linux and Vulkan. Some minor quality presets have been tweaked, but this is a fully working cloud version of Ubisoft's hit, which happily runs on a browser.
There's still the sense here that we're one technological breakthrough away from cloud systems becoming fully viable. Image quality will be improved still further in the fullness of time simply through the delivery of more efficient compression and more bandwidth. The latencies seen today are certainly a massive improvement over the general OnLive and Gaikai experience from back in the day, but driving latency down still further is still the key for making these systems work for more than just the casual user. All eyes are on Google, Microsoft - and all the other cloud players working away on their systems - to deliver that final breakthrough.