The numbers game

Amazon has been all over the headlines this week, courtesy of its intriguing $99 Fire TV capsule console. From hiring talent like Portal's Kim Swift and Far Cry 2's Clint Hocking, to buying Killer Instinct studio Double Helix and securing versions of The Walking Dead and Minecraft, as a tech company Amazon has been making all the right noises. It clearly takes gaming seriously.

Yet at the same time, as a retailer, it's just taken a giant leap backwards by quietly starting to roll out the use of Metacritic scores on its games listings. Those ubiquitous traffic-light numbers will now appear at unmissable size, right next to the "Add to Cart" button. If it was bad news for a specialist retail service like Steam, and it's even worse when a mainstream retailer the size of Amazon follows suit.

The slow and inexorable acceptance of Metacritic as some sort of ultimate truth is as predictable as it is depressing. We humans are driven to impose uniformity, to find a box for everything, to have things in the right order. This means we're also prone to something called apophenia, the tendency to see patterns and meaning in unconnected data.

Spoiler alert: that's all scores are. Unconnected data. There's no great objective truth behind the number at the end of a review, just a gut feeling expressed in an easy-to-digest format, a convenient summation of the paragraphs that precede it. And those paragraphs are a subjective opinion, hopefully informed by experience and insight, but ultimately no more mathematically quantifiable than whether you prefer chocolate or strawberry ice cream.

At best, a score should accurately reflect the text and it should adhere to a scale that is consistent across the site or magazine. That doesn't always happen, because it's a fallible human system, not a machine. A score is a clumsy but handy way of illustrating how a particular game succeeds or fails on its own terms, as perceived by the writer, and how it stacks up against its immediate peers. The idea that there should be some wider coherence, that every game ever made can be ranked in order of quality simply by crunching numbers, is nonsensical.

"The idea that there should be some wider coherence, that every game ever made can be ranked in order of quality simply by crunching numbers, is nonsensical."

It's here that gaming proves especially susceptible to the false comfort offered by a metascore. Metacritic doesn't just cover gaming, but film, music, TV - these are media that feel mercurial, alchemical. However they're delivered to us, we can see and hear the human hand in their construction. They spring from a creative place that we accept is impossible to truly pin down.

Metacritic ratings also appear on the Internet Movie Database, but nobody in the film industry really seems to care. Studios are not closed, actors do not have their pay docked, if a certain arbitrary metascore is not reached. Film has been around for too long, its cultural roots are too deep, for Metacritic to have any real impact. There are still arguments and differences of opinion, of course, but they tend not to be about whether a four-star movie review read like three stars, or whether gruelling documentary The Act of Killing is as good as Disney's Frozen because they both have the same score. Scores for movies are rightly treated as a sideshow, a not entirely accurate thermometer, rather than the main topic of conversation.

Relatively young as a medium and with the neatly ordered rankings of high-score tables and player statistics woven into their electronic DNA, games are much more vulnerable to the pernicious comforting lie of the metascore.

It's no surprise that gamers flock to Metacritic for reassurance when handing over their money, seeking the solid confidence of a single "true" score, and in this age of metrics, analytics and data-driven decision making you can hardly blame publishers for using Metacritic's pseudoscience as a guide rail for performance. Like all successful ideas, it's incredibly convenient. It's also incredibly flawed.

As a writer, there's personal ego at play whenever words are sidelined in favour of numbers, of course. It's always depressing to spend days writing a thousand carefully considered words, only for discussion to congeal around whether the figure at the end deviated by one point from the perceived 'correct' score. This isn't, however, an argument against scores in themselves. They serve a purpose, when taken individually and in context, but we've all - from writers to gamers, PR to retailers - allowed them to take on an autonomous significance far in excess of what they actually represent.

Crucially, we need to rid ourselves of the notion that the primary purpose of a games review is to act as a buyer's guide. A review can tell you whether you will like a game, but it does so primarily through description rather than simply imparting the writer's preferences. It's then your job to take all those words, along with your personal context, to decide if you think the game is interesting. You don't have to agree with a review in order for it to be useful or enlightening.

So while Metacritic's domination of the critical conversation is an annoyance, its determination to sit in Amazon's shop window and be a part of the commercial process of buying a game is more troubling. The metascore is a statistic built from the raw fibre of multiple incompatible subjective opinions, softened by the absence of context and eventually broken down by the invisible enzymes of Metacritic's top secret algorithms and statistical weighting. Yet the gaseous digits that squeak out at the end of this patently ridiculous process are too often treated as gospel by shoppers and publishers alike.

Good criticism should always be an invitation to further discussion, the start of the conversation rather than the end. In going from curious statistical experiment to globally accepted league table, Metacritic turns the open invitation of individual reviews into a full stop, drawing a line under critical thought. It even colour codes the results, reducing even its basic 100 point scale into an idiot-proof traffic signal: green means buy.

If just one gamer turns away from a potentially fascinating game on Amazon, purely because the metascore beacon flashed a cautionary yellow warning, that's one gamer too many.