Nvidia announces DLSS 3.5 with ray reconstruction, boosting RT quality with an AI-trained denoiser
Plus: Half Life 2 RTX, Alan Wake 2 gets path tracing and DLSS 3 comes to Call of Duty.
Gamescom 2023 has kicked off - as you might have noticed - and Nvidia has unveiled DLSS 3.5, a new version of DLSS that adds 'ray reconstruction' - essentially AI-tuned denoising that boosts the quality of ray tracing effects on RTX graphics cards. The firm also announced a new community remaster of the legendary Half-Life 2, new DLSS 3 and Reflex titles and more.
Let's cover the DLSS 3.5 news first, as this is fascinating. The critical information here is that real-time ray tracing relies on a relatively small number of rays that are cast - certainly far fewer than one ray per pixel - so we don't have a complete picture of the 'ground truth', we have an image with both signal and noise. That means a denoising step is required to fill in the missing pixels, which uses techniques like temporal accumulation and spatial interpolation to generate a higher-quality final image.
So far, so standard. Normally these denoisers are hand-tuned, but the innovation is that Nvidia has trained an AI network (on five times as much data as they used to train DLSS 3) that can achieve better results. It can use different methods with different ray-traced effects, recognise good pixels and discard bad pixels, and retain the high frequency detail that would normally be smoothed away by upscaling.
As well as improving fidelity, Nvidia says that replacing multiple denoising steps with a single AI-trained model can also lead to a slight performance advantage. They showed an example (in the video above) where Cyberpunk 2077 jumped from ~104 to ~112fps with ray reconstruction enabled. However, in general we should expect performance that's "about the same" with the feature enabled.
Notably, this new denoiser requires additional information from the game to operate, so it needs to be added by developers on a per-title basis rather than being a universal upgrade for all existing RT titles.
However, ray reconstruction runs on the Tensor cores on Nvidia graphics cards, so unlike frame generation, this feature will be available on RTX 20-series and 30-series graphics cards as well as Nvidia's latest generation 40-series GPUs.
Ray reconstruction is being added to path-traced games first, as they show the most benefits given that they use entirely RT effects. That short list includes the Phantom Liberty expansion of Cyberpunk 2077, Portal with RTX and Alan Wake 2 - the latter of which we didn't know before now was a path-traced title. Of course, Remedy have fine form, with 2019's Control remaining one of the finest RT showcases, so seeing them embrace path tracing makes perfect sense. (An Alan Wake 2 DLSS 3.5 trailer is set to go live at 2PM UK time today with more footage of the game and its graphical technologies.)
The first DLSS 3.5 titles are set to arrive "this fall". In addition, three content creation tools - Chaos Vantage, D5 Render and Nvidia Omniverse - are also integrating ray reconstruction to improve preview images.
For more detail on how ray reconstruction (and ray tracing in general) works, it's worth reading Nvidia's DLSS 3.5 blog article.
Elsewhere in their presentation, Nvidia also announced Half-Life 2 RTX, a community project to use RTX Remix to remake the seminal Half-Life 2 with a laundry list of technologies including full ray tracing, DLSS 3, Reflex and RTX IO, on top of modern assets.
The game looks fantastic already, thanks to the works of several Half-Life modding teams folded into a new group called Orbifold Stuidos, but no doubt Nvidia hope that the announcement of the project causes more volunteers to get involved. If you're interested, you can head to Nvidia's Half-Life 2 RTX site.
Elsewhere, DLSS 3 frame generation was announced for Call of Duty: Modern Warfare 3, Payday 3 and Fortnite, while MW3 and Payday 3 are also set to feature Reflex to reduce latency. Nvidia's metrics suggest 90 percent of GeForce Experience users turn on Reflex when it's offered - though it is often on by default - thus making it a much more universally liked feature than ray tracing or even DLSS, which have more obvious drawbacks in terms of performance or image quality depending on the settings chosen.
Nvidia also stated that Xbox PC Game Pass titles would be available on GeForce Now from this Thursday (the 24th of August) and detailed more of its efforts in AI-powered NPCs, including 'NeMo SteerLM' which allows for characters to respond in more realistic ways - Nvidia slides show characters with sliders for creativity, humour and toxicity.
It'll be interesting to see the first DLSS 3.5 titles in the flesh, so stay tuned for some impressions and analysis when we've had a chance to check it out.
The quality improvements do look impressive, and it'll be interesting to see if we see the same AI-trained denoiser used in older RT releases - Control could look astonishing, for example...