In Theory: Could the next-gen Switch use Nvidia DLSS AI upscaling?
Preliminary experiments produce some extraordinary results.
That rumour didn't last long. Stories last week suggested that the long awaited Switch Pro would arrive in 2020, with Nintendo bringing the fight to the next-gen Microsoft and Sony console launches. It all sounded rather unlikely and Nintendo was quick to dismiss the story. The current Switch range continues to sell relentlessly and there's no substantial commercial reason for Nintendo to release a more powerful model quite yet - nor to go head-to-head with PS5 and Series X with what would almost certainly be less capable hardware. However, the firm's partnership with Nvidia is likely to continue and three years on from the Switch's release, plans must surely be afoot for a next-gen system. On top of that, looking at how Nvidia's tech is evolving presents some mouthwatering opportunities for a new, more powerful Nintendo console hybrid.
Expectations should be tempered if the plan is to produce a more powerful console and nothing more. Fundamentally, compute power in a mobile device is limited by the need to accommodate a relatively small processor running at relatively minimal clock speeds. If we get anything close to the kind of performance we've enjoyed from PlayStation 4 and Xbox One, we should consider that a minor miracle for a system derived from a processor designed primarily for mobile gaming. However, with that said, a new Switch based on more recent Nvidia technology opens the door to the firm's impressive AI upscaling techniques - and I decided to put them to the test in scenarios designed to more accurately represent Switch-level gaming.
I attacked this challenge on two fronts. First of all, AI upscaling technology is already available in a Tegra X1-based product - the newly revised Switch Android TV, a tubular revamp of the powerful streamer that Nvidia continues to support and upgrade. AI upscaling is exclusive to the new versions of the Shield, and works on any video content running on the machine - the only limitation being that anything above 30fps content is not supported. And this led me to wonder: what if I fed the Shield with Switch capture? How would that look?
I also thought about approaching this from the opposite direction. We've been quietly but cautiously excited by the DLSS AI upscaling tech found in recent games - it's a vastly improved proposition over some already impressive implementations in key PC games. As we've already showcased, Wolfenstein Youngblood performs brilliantly under DLSS and in some scenarios, the upscaled version looks cleaner than the native presentation. So what if we dropped resolution to the Switch version's max 720p and used DLSS performance mode on the lowest settings? How would this compare with the Switch version?
The reason I'm excited about DLSS is remarkably straightforward. We want to push visual features to the next level with technologies like real-time ray tracing, but we don't want to lose too much performance for the privilege. Fundamentally, why use GPU resources to paint every single pixel when AI upscaling can 'infer' a lot of those pixels instead? Wolfenstein Youngblood and other freshly baked DLSS titles are showing some remarkably impressive results and the tech has a lot of potential. If the performance uptick is substantial and the quality is there, it could also be deployed in a mobile device where compute resources are extremely limited.
Instead of using DLSS to mitigate a performance loss from high-end features like ray tracing, instead we'd be using it purely to improve image quality - and it's no secret that Switch's most ambitious games do tend to be rather blurry, a factor of the low resolutions required in many cases. Even first-party Nintendo titles are affected here - Splatoon 2 and Super Mario Odyssey use dynamic resolution scaling and often reside in 720p territory. Meanwhile, the UE4-powered Yoshi's Crafted World can even hit a minimum of 576p in docked mode. In the age of the 4K TV, docked play can look a little lacklustre in some cases.
The first order of business was to check out the new Shield's AI upscaling technology. As far as we know, the latest Tegra X1 doesn't feature the tensor cores designed for accelerated deep learning functions, so the process would likely be running on the X1's standard CUDA cores instead - which lie dormant otherwise during video playback. Full-fat DLSS reconstructs images based on a training algorithm derived from super-sampled images taken from each supported game. I'd imagine that Shield's AI upscaling is something very different - and this is borne out by the results I received.
If I had to guess, I'd suggest that the Shield AI upscaling blows up and attempts to intelligently sharpen the image, letting that extra 'detail' bleed out onto a 4K canvas. You may see from the Doom 3 and Dragon Quest Builders 2 images below (upscaled from native 1080p) that there is more detail resolved - but at the same time, I achieved very similar results from a Photoshop upscale and sharpen filter. I also tested the scaler by comparing the original Switch capture to the Shield's output, downscaled back to 1080p. Here, the differences were very difficult to notice. If there were any improvements they disappeared in the double-scaling process (1080p to 4K back down to 1080p again).
There is value to the Shield's AI upscaler (a quick google reveals that some people are genuinely amazed by the results) but it's clearly not designed to process game footage. Jaggies become jaggier when upscaled and sharpened to 4K, while there simply isn't enough core detail at low resolution games to deliver any kind of meaningful improvement. I don't think there is much relation to Nvidia's PC-based DLSS here as while quality varies on its older implementations, it always adds extra detail - something that isn't happening with the Shield's algorithm. There can be some big differences in TV and movie content, but I do feel that the sharpening halo artefacts are just too strong, even on the lowest setting.
However, I am excited about the potential for Nvidia's DLSS technology in a future mobile platform because the results from my testing with Wolfenstein Youngblood are quite remarkable. Previously, we've run the game maxed at high resolutions and found that without ray tracing features enabled, a £260/$299 RTX 2060 can run the game at DLSS 4K at 70 to 90 frames per second. The visual quality is just superb too to the point where I think that if DLSS is an option to you on this title, you'd be nuts not to use it.
But what happens when you run Youngblood at 720p on the lowest settings in DLSS performance mode? In effect, what if we try to make the PC version run as close to Switch settings as possible? This is a fascinating test because in this scenario, the PC version is rendering a framebuffer that's just 640x360 in size - a 50 per cent resolution scale and well into the kind of pixel-count ballpark employed by the Switch version. I'll let the screenshots directly below tell the story, but it's fascinating to see how well the DLSS version retains detail and easily outstrips the image quality delivered by the Switch version. DLSS upscaling isn't just about getting more from higher-end hardware - it could play an important role on a mobile chipset too.
By switching things up to 1080p DLSS and engaging medium settings, the GPU load obviously increases, but I thought it would be fascinating to see how a native 540p image with AI upscaling compares to the PlayStation 4 version. While 60fps might not be achievable on a prospective Switch Pro at these settings, the increase in resolution and a bump to console equivalent medium settings should be. The results are even more impressive, in my opinion, and further cements my opinion that incorporating hardware-accelerated deep learning features into a next-gen Switch could yield real benefits.
Nintendo itself has categorically ruled out a Switch Pro for a 2020 release and our understanding is that there are no references to any kind of hardware upgrade within current revisions of the Switch firmware - which is how we discovered the existence of the latest models based on the 'Mariko' Tegra X1. The Korean report on the new model talked about a processor using Nvidia's Volta technology - which would likely feature deep learning accelerated support. The firm already produces the 'Xavier' Tegra processor, but with a 350mm2 die size at 12nm, it's almost as large as the chip in Xbox One X and entirely unsuitable for a handheld.
With that in mind, it looks like it's the end of the line for 'off the shelf' Tegra parts making their way across to Switch, with future processors likely to be custom in nature and probably using a 7nm process technology - factors that make a Switch Pro more viable for 2021 (or perhaps even later). And even then, there may be other approaches Nintendo may prefer to take instead of creating new silicon. The new Mariko-based Tegra X1 in the new Switch models could potentially deliver around 60 to 70 per cent more performance on both CPU and GPU - Nintendo chose enhanced battery life and new form factors over performance upgrades with the new Switch and Switch Lite. Boosting clocks to max (or near to it) may not be ideal for a handheld, but it's certainly an interesting prospect for a Switch micro-console, perhaps.
In the here and now though, DLSS results on Wolfenstein Youngblood have proven rather impressive. For PC users who may have overlooked DLSS after some not-so-favourable press, I'd strongly recommend checking it out if you have the means. I was concerned that the results in Wolfenstein Youngblood may be a one-off, but having also sampled the same new revision of DLSS in Deliver Us The Moon, image quality is still extremely impressive - even at the lowest quality/highest performance setting (there are some sample captures in the embedded video above). We'll be following the progress of AI upscaling with much interest - it's clearly paying dividends in the PC space and could prove to be a key technology for the next generation of console gaming.