Carmack, Sweeney and Andersson Unplugged

So one of the key things we've been hinting a little bit at this is that games nowadays are giant and they are so variable and there are so many things we want to do and, to be honest, we don't fully have control over everything we are doing. It takes a massive effort to optimise a game for a fixed frame-rate and we're actually sacrificing things for it also in some areas, depending on how you actually control your games - and many games are actually lots of variation within them also you want to use what's the most appropriate there. Some parts you perhaps do at 30fps, some parts perhaps at a higher fps but you want to do that and have a great experience at all times so G-Sync really comes into play for this stuff. You're really adaptively finding what's [the] best fit for this specific game and this specific time and allowing game designers to really play freely with that and I think that's one really key aspect.

Yeah, I've been thinking a lot about this, you know. People have come nowadays to expect a lot more perfection from these entertainment experiences than they did in the past. The one thing that really motivated that was Apple coming out with the iPhone. This product is so polished that it really changes everyone's expectations. One thing you don't expect now is graphical glitches and yet, when you play a game, even a lot of the best games on Xbox 360 and PlayStation 3 - and we at Epic are guilty of this - you see tearing, you see jittery frame-rates and we've reached a point where technology enables us to move beyond that and gamers won't accept it any more, so we as the software developers, the hardware GPU makers, really have to work together to absolutely pound out all the remaining latency and jitteriness and visual artifacts from the entire pipeline so that we can come ever close to approaching the ideal for most of these games which is a fluid and realistic experience.

So I fought the crusade for 60fps locked frame-rate on the last generation and there are a lot of significant sacrifices that get made along the way. There were constant battles of "we can't do that particle effect here" and "we can't have more creatures there". The game would have been better if you could just say, all right, 90 per cent of it is 60fps but 10 per cent is going to be awesome and be a little bit lower frame-rate but we didn't really have that option because dropping down to the stuttering mess of going [from] 60 to 30 is a cliff you fall off of and we weren't willing to do that.

If you had G-Sync available on the targets that you're designing for you're free to do that. You can say mostly we want this to be silky smooth but a few places we just want to blow the doors off and have something where it's OK to chop down a little bit and that will just unequivocally make for better games. I mean, I have a history with kind of old biz-sim people where there was a crowd that used to believe that 60fps is all that matters and they would look down their nose derisively at PC titles but the bottom line is they were wrong.

I mean, it's a trade-off that you make where smoothness is important but very often it can be legitimate and correct and valuable to sacrifice frame-rate for some things. You know, there's a funny story maybe 13 years ago on the 3DFX technical advisory panel that I was on, I was kind of bemoaning all that time ago, you know that you have swap tear going on and one of the other guys from a more engineering background looked over somewhat accusingly and said it's your fault and he was basically saying that the benchmark wars surrounding Quake were what lead virtuous GPU vendors to give up on tear-free swapping because it was like, well, you get a better benchmark result when you tear and it was the right thing to do.

I mean, games were better when you could go between 20 and 30 frames per second if you care about the responsiveness but we weren't between 30 and 60 which is more of a no mans land. So, it was a valid trade but we lost sight of the cost that we have there that it is a bad thing and now people just kind of glaze their eyes and look past it and being able to address that is going to be really important.

So, one of the things that I've found interesting about G-Sync when we were designing it, we kind of postulated what the benefits would be; you get all the benefits of v-sync off without the tearing. You can intellectually grasp what that means but when you experience it it's surprisingly better than you thought. I think a lot of us didn't even realise how good it would be when we were designing it until we saw it. Can you think of an "ah-ha" moment over the last few years where something came out better than you thought or kind of surprised you in a way you didn't expect.

For me, again, the low persistence thing is something Valve did where they were working on one of their research head mounts and I had had conversations with them previously and my position had been 60Hz is clearly not good enough, so I thought if we get to 120Hz that should be good enough. They were quite positive that the persistence in a display was a very important thing and I was somewhat skeptical but when they had me set up there where you could literally just turn it on or off you can see that, ok, 120Hz is definitely much better than 60Hz but that's still 8ms of locked-on time to the pixels but you really want that to be more like 1-1.5ms or something. Within the space of just a couple seconds I was like, yep, you're right, you're absolutely right, this is a super important thing and I didn't expect it to be that significant and I was thrilled to be kind of proven wrong on what was most important.

Last generation, I would say, I think this solves the monitor problem as well as it can be solved and the future problem is in this stuff that John and Oculus and the others are working on. That's what you need to go beyond what's possible on a monitor.

Well, one more basic thing also is to actually get to that performance. You don't see many changes now actually shaping that out of the box with good settings at 100fps. Perhaps on the highest end graphics card or perhaps with multiples of them, yes, you'll be there but you want to be there at $150 or $200 - GPUs that people can actually afford. You want to have that experience for everyone to run your game as the standard thing with more than 100fps and a variable refresh rate depending on what you're actually doing.

But taking that leap forward in our games requires a lot more performance and we need to get that from the hardware vendors, from the operating systems, from the way we design our games, from the techniques that we're using but I think that's actually quite interesting for a fraction of gamers.

So I actually think that the pictures being made in modern games are really pretty damn good and, in many ways, more than adequate for what we're doing in the gaming entertainment side. We can always make things better and we will make things better, they're going to continue to improve over all these different axes, but my personal belief is that it's time to start pushing forward on these higher frame-rate, lower latency things.

You can still argue that it's kind of this connoisseur thing where you can just look at it and say, well, the picture is prettier here and I'd rather have this at 30fps and there will be different classes of consumers but, from my point of view, I think we're already past the knee in the curve... the cost benefit curve of doing more advanced graphics there and I'd like to see more effort spent towards reducing the latency and improving the all-round feedback loop. I think IO devices are also a thing, obviously with head mounted displays and various position tracking things, that will make larger differences than the next few orders of magnitudes spent just on making pixels because we know where the better pixels go, it goes to movie rendering and we have existence proofs showing we can spend another six orders of magnitude of graphics power and then we get the Avengers scenes rendered in real-time whatever and that'll be great, but I think that there's other things that we can do with that horsepower which is more valuable.

I think one of the interesting things that, if VR and head mounted displays are an important part of the future, if they're going to go 4K to 8K per eye, you're talking about 16 maybe 32 million pixels rendered at over 100Hz and you're probably going to walk around which is going to be a little harder carrying your three way SLI rig so you're going to want enough horsepower in a mobile platform display 32 million pixels at 100Hz with really nice pixels at super low latency. So that'll keep guys like Nvidia busy for a long time.

Let me just say, it sounds insane when you say it like that, but it's going to happen. It actually will happen. It sounds absurd but...

Let's just take a little step back, you guys remember Voodoo graphics? You know, in the dawn of 3D graphics that was 45 million pixels per second. We're talking about GPUs these days that have hundreds of pixels per clock at gigahertz clock speeds that are teraflops of floating point performance. John's right, I mean, it's been many orders of magnitude so, while it sounds nuts to think that you're going to have something more powerful than a three way Titan SLI rig attached to your face, it's not. I mean, it's inevitable which is crazy... but it's cool.

I think this is going to be pretty broadly picked up because this is not like it doubles the size of the die [GPU chip] to do this, it's not that expensive of a technology and, I have no idea what Nvidia's licensing costs are going to be on here, but this is something that should be broadly adopted. From a pure technology standpoint this is just the right thing and obviously in the early years this is just going to be a smoothness benefit for people but, in the space of five years or something, I would hope something like this is pretty much in everything.

Absolutely. There is a limited amount of things that monitor manufacturers can compete on. Once you have 4K...

This is actually a good thing!

At least remove the sharpening, edge detection and motion compensation and actually have this, and 4K pixels would be nice, and 120Hz also. I think that when people actually see what Nvidia's been doing, more and more people see that there will actually be a lot of movement in the industry to transition to something like this really quite quickly and that's what I'm looking forward to because then we can actually target games more specifically for this.

Yeah, it's actually that the rest of the industry needs to hardware-up! Every display in a device, every platform including the global platforms will need to adopt this kind of technology over the next few years because it's visually striking. Doesn't show up in a TV commercial but to any player playing a game it's incredibly apparent the difference in visual quality here and so it's going to be a major addition for everybody.

Actually, one wish I have there, you go to all these retail stores where they sell tons of TV displays and monitors and you see all these really nice monitors and really nice big screen TVs and all of them are set to a startup "showroom mode" that just cranks up all of the colours. I would really like to replace that. Remove that mode, showroom mode does not exist anymore instead, just have this variable refresh rate and you can actually see the difference on that. Those false colors - you don't want to have those anyway. Those aren't calibrated to anything we actually created, it just creates an artificial environment plus all of the other s*** that you do on there, support legacy content.

Well said.

I'll answer that from an industry perspective. Pretty much every game that ships already has that mode. They almost all have settings that when you turn everything on and you turn it up or turn on AA or TXAA and try to run it on a 4K monitor, even with Titans, it's not 60Hz. Most of these games already have settings that push the absolute limits of everything and then you get in a firefight with 64 guys shooting and explosions and it is not a 16ms frame. I guess to some degree that already exists but whether they turn it up to 11 - let's ask them.

There was a lot of pain from the artists not wanting their favourite particle effect to be cut and you had to be pretty brutal to hit 60fps. It is a little unclear how you would make the cuts if you're still trying to be a 60Hz game on next-gen consoles. Something like this would certainly let lower-end PCs play the same settings without falling apart like that but I'm not sure how much you'd throw in in regards to extra, additional things. It would be like optimising for a bit of a niche platform at the beginning but I think it's absolutely right that you can crank the settings in most games that cause them to want something like this.

Rage did suffer in some ways by not having those extremes moments and not having the things where we just think, let the frame-rate drop and probably, the game unquestionably would have been a richer experience if we were able to support some of those but I thought that was a slippery slope - we let those go in some place and all of the sudden we won't have the smooth frame-rate everywhere because those will be showing up everything. There are tough judgement calls to be made with anything like that but it will be nice if nobody has to fight through those same battles to do that. You can make the decision that you want a super-smooth game and not have to be quite so dictatorial and decisive about the things that get cut to hit that because the raster is a very harsh taskmaster there and you always have to hit it and this is clearly a better way for things to happen.

G-Sync.

Yeah, at this point we'll always be able to suck up more resolution but, right now, people migrate to 4K and they get much lower frame-rates and I think the trade-offs are already a little bit sub-optimal that people are making now and quadrupling down on that now, while G-Sync will make everything somewhat better.

So I have the whole issue of the low persistence displays that I have to deal with on the head mounted display systems but I think that for a conventional desktop monitor I really want to see this take off... It's a shame that those 4K monitors back there aren't G-Sync because you see them juddering and tearing and exactly the problems we have been talking about now. It's actually an object lesson there and it's a shame we don't have those G-Synced.

I see 4K as kind of a workstation application for the next few years. In that time frame we won't be using those monitors for building games because we won't have the huge complex tools open that photographers will be using for their work... videographers, lots of people will be using it but I don't think it's the right output resolution for rendering until your GPU is significantly faster. Once that 20 teraflops is in a single GPU and costing less than three hundred bucks, that's when you want to go 4K.

The environment where I work on my own desk, I essentially I have four 24-inch monitors and a 4K monitor next to that, putting them in a half circle around you and it's actually a really good working environment. So, having a single 4K screen that's curved and 50-inch something like that would actually replace four of my screens and make an amazing work environment. You can definitely play games in the big screen in an immersive way also and you're starting to see these curved OLED screens in that size, so, add G-Sync on it and refresh to 120Hz so it's perfect gaming and a perfect workspace.

While we're at it, I want spherical section screens so everything focuses properly on the headsets.