Call of Duty 4 Engine Analysis

As we're currently spending a good deal of time analysing the new Call of Duty and Quantum of Solace games for the regular Eurogamer face-offs, we thought we’d get to know the core technology better by unearthing our copies of the brilliant Call of Duty 4: Modern Warfare.

Both games run at a sub-720p resolution of 1024×600, blown up to 720p (or indeed 1080p) but the resolution loss is diminished thanks to the superb frame rate and the use of anti-aliasing to refine the edges - in short, it moves and scales smoothly. The console games look like-for-like, but the eagle-eyed can see marginally better special effects in the PS3 code; 360 for its part has occasionally better textures. By matching scripted sequences from the game engine, we can compare the performance of Modern Warfare on both Xbox 360 and PlayStation3.

We all know that the console versions of Call of Duty 4 aspire to 60FPS gameplay, but the question is, which version is more successful? Are the two games really as close in terms of refresh rate as commentators (including this fool) have said in the past? Perceptually, we’d say ‘yes’, just as we did almost a year ago to the day in that Eurogamer feature.

But technically, it’s a no, as the tests below demonstrate, and it’s also worth bearing in mind that there’s no hardcore combat in any of those tests where you would expect to see the gap widen. In the limited tests available on like-for-like video, PS3 game is more impacted in 12 out of 13 scenarios, but as the action is usually maintained at around 50fps, the human eye is usually fooled. In more intense gameplay, with lots of left to right panning movement, the difference will be more noticeable, but it’ll take a really heavy scene to actually affect response from the controls.

Onto the tests then. The methodology was pretty straightforward - to capture mostly scripted scenes from the game generated by the engine (as close to like-for-like video as you’ll get in consoledom), and then use the Digital Foundry frame analyser to check them out, to see what causes each platform to trip up, and to compare performance.

Downloads

The vids embedded below are for commentary context only. 30FPS streaming video is useless for accurate measurement. However, 720p60 clips are available for download. Go here for clips #1 to #6, and here for #7 to #13. The output of the frame rate detector for each clip on both platforms is available here.

Additional Note:

When examing the frame rate graphs below, it’s well worth pointing out that the graphs are plotted across an average number of frames. Therefore a single dropped frame will look like a ‘V’ on the graph, while several dropped frames in succession will adopt a more ‘\__/’ style of shape.

Not much to stress the engine here, both games exhibiting a solid frame rate.

• PlayStation 3: 59.94FPS (average)
• Xbox 360: 59.94FPS (average)

At the beginning of this clip, the PS3 engine renders at around 50FPS. The average rises as the engine returns to 59.94FPS. Xbox 360 is solid at 59.94FPS throughout - the 0.06fps loss from 60fps is not relevant, it’s really the case that HDTVs update at this frequency, not actually at 60FPS (a throwback to the NTSC drop-frame format that has inexplicably made its way into the HD standard). Note how the rain effect on the PS3 version is far more pronounced.

• PlayStation 3: 56.94FPS (average)
• Xbox 360: 59.94FPS (average)

Xbox 360 drops a total of 16 frames on the section where the player is looking into the control room as your colleagues shoot the seamen, the rest is a solid 59.94FPS. PS3 on the other hand, drops 68 frames - over a second of video - in the same timescale. Note how the frame rate drops on both versions at the same time, with the PS3 game being more impacted.

• PlayStation 3: 55.06FPS (average)
• Xbox 360: 58.79FPS (average)

Flawless playback on Xbox 360, with not a single frame dropped. For the most part, PS3 runs the same section at around 48FPS, recovering here and there to slightly increase the overall average. As there is little lateral left-right movement, the dropped frames are hard to notice.

• PlayStation 3: 51.48FPS (average)
• Xbox 360: 59.94FPS (average)

A chance to see how the CoD4 engine fares rendering game characters close-up at extreme detail levels. 360 drops a single frame rendering the final blast from the terrorist’s pistol. PS3 drops down to 50FPS rendering the red-bereted AK47-wielding terrorist, then drops further frames as the pistol is being handed to the executioner. The long length of the sequence keeps the average high.

• PlayStation 3: 58.762FPS (average)
• Xbox 360: 59.91FPS (average)

The intro to the first helicopter mission. The left-to-right movement makes frame loss and the resultant judder much easier to see. On 360, most of the frame loss occurs once the action moves in land, with a gradual decline in frame rate. PS3 appears to decline sooner, and faster. Once again, note how both versions struggle at exactly the same points. Also of interest is the sudden dip at the beginning of the clip - our guess is that the game’s caching data at this point - you’ll notice it happen in several of the other videos in this feature.

• PlayStation 3: 52.92FPS (average)
• Xbox 360: 56.88FPS (average)

The descent from the helicopter is the first real engine-stressing moment. Before reaching touchdown on terra firma, 360 runs at 50FPS, PS3 worse still at 40FPS. Again, a lot of left to right movement here emphasises the judder. Frame rate recovers once on the ground, skewing the average higher.

• PlayStation 3: 47.43FPS (average)
• Xbox 360: 53.74FPS (average)

After the initial data caching (?) glitch, this clip settles down nicely at 60fps on Xbox 360, while the equivalent video pushes the frame rate down to 50FPS on the PlayStation 3. As this is mostly a gameplay clip from a pretty advanced-looking level, it suggests that Infinity Ward is starting to push the CoD4 engine to its limits on the Sony platform.

• PlayStation 3: 51.43FPS (average)
• Xbox 360: 59.62FPS (average)

A single dropped frame on Xbox 360 as the SAM hits the helicopter (quite interesting to see how in terms of averages this skews the graph). In a 15 second clip, the PS3 code drops over a second’s worth of frames (68) but as the frame drops are sporadic and not even, they’re virtually invisible to the human eye. This is the essence of perceptual 60FPS.

• PlayStation 3: 55.48FPS (average)
• Xbox 360: 59.88FPS (average)

Pretty much a constant 60FPS on Xbox 360, with just two frames lost as the player is revived by Captain Price. The same section causes issues for PS3: 27 frames lost, bringing frame rate for that section down to around 40FPS The action returns to 60FPS for the rest of the clip, raising the average, but it’s interesting to see just how much more of an impact a single engine-stressing moment has on both platforms.

• PlayStation 3: 57.43FPS (average)
• Xbox 360: 59.76FPS (average)

Just like Test #8, the 360 code drops a handful of frames on the initial zoom-in, but is otherwise absolutely solid. PS3 stutters at the same point (again probably a data caching or retrieval issue), but the frame rate fluctuates more - most of the first half of the clip runs at 50-55FPS before recovering to even out the average. It once again indicates that a lot of action will cause the PS3 game more problems.

• PlayStation 3: 55.39FPS (average)
• Xbox 360: 59.03FPS (average)

Another helicopter clip, and more left to right movement that should effectively show judder/frame loss. Despite the length of this clip, 360 drops just three frames, all in succession (again, caching?). The whole game is actually pausing at this point, knocking the two captures slightly out of sync. PS3 has a similar pause a few seconds earlier, but has four seconds of action of sporadic frame loss not seen in the 360 version.

• PlayStation 3: 57.91FPS (average)
• Xbox 360: 59.03FPS (average)

In this final clip, we once again zero in on a moment of pretty non-taxing gameplay. A single frame dropped on Xbox 360 during the initial zoom-in. Likewise with PS3, which also drops a further 94 frames during the course of the clip.

• PlayStation 3: 55.64FPS (average)
• Xbox 360: 59.90FPS (average)

Conclusion

Call of Duty 4: Modern Warfare clearly runs smoother on the Xbox 360 than the PlayStation 3, but the performance defiency barely affects gameplay owing to the extreme frame rate the game runs at as an average - the human eye will have big trouble spotting an occasional dip to 50FPS in these clips. However, what is clear is that the tech is being pushed, and the new CoD game pushes it further. How much it is affected and what impact there’ll be on gameplay will be interesting to find out.

From what I’ve played so far of the new Call of Duty: World at War on 360 (still waiting for the PS3 one to turn up), the engine is being stressed and dropping more frames more of the time. Nothing I’ve seen indicates that the tech has been optimised that heavily, so I’m going to be very curious indeed to see how the game runs on the Sony platform.

What is worth remembering is that the core engine powering CoD4 is an enhanced version of the one that ran the brilliant Call of Duty 2 (still one of the best FPS games ever made). That in turn is derived from code from the Quake 1/2 era - geared for a single CPU and GPU combo - so in that sense it is not surprising that 360 sees a performance benefit, and indeed, it makes the PS3 code that much more impressive. Call it a rumour if you will, but I recently received a message from a development source that while the CoD engine hasn’t been massively improved for this year’s Treyarch games, next year’s Infinity Ward title will be running a more enhanced version, with more key optimisations geared towards parallelisation (good for all platforms); and expected to power more Activision games up to 2012, when the current generation of consoles will most likely end. As a commited advocate of 60FPS gameplay, if this information is on the money, I can’t wait to see it…

Update: A full compilation of the 13 clips above, encoded into superb quality streaming HD, is now available.