Football Manager developer denies newspaper claim it edited its player database after accusations of racial profiling
As real-world football faces up to its racial bias.
The developer of Football Manager has insisted it did not change the way the game works in response to an allegation of racism.
An article published in The i newspaper this week alleged Sports Interactive altered the way the simulation's widely-used player database works following the publication of an online article in 2019 that accused Football Manager of having "a racism problem".
Football Manager's database represents real-world footballers by a number of stats (roughly 250 each) covering a variety of aspects from the physical (Pace, Strength, Shooting, Tackling etc) to the mental (Adaptability, Creativity, Decisions, Leadership etc). These stats are initially compiled by a network of more than 1000 researchers around the world, then refined and compiled into the final database by Sports Interactive's in-house team.
This database is used by professional clubs all over the world in their talent ID processes, and has been credited with predicting the success of a raft of superstar footballers, including Kylian Mbappé, Cesc Fabregas and Sergio Aguero.
In June 2019, Quartz published a report based upon an analysis of data from Football Manager's player database that found "players and staff with darker skin tend to have worse ratings on non-physical attributes".
Quartz examined the share of players and staff with each skin colour number who received a rating of 10 or above on Sportsmanship, Temperament, Professionalism, and Loyalty - aka, those who rank in the top half of scores for these non-physical attributes. The charts make for grim reading.
Additionally, Quartz analysed the Professionalism rating of more than 900 players in the English Premier League, and found lighter skin players tend to receive higher scores on non-physical attributes. Sports Interactive did not comment at the time.
It's worth explaining exactly how these "personality" attributes work. They are hidden, that is, players can only find them if they own Football Manager's in-game editor. Each player has a Personality, which is a descriptive word and visible to players. For example, a player's personality can be described as Determined, Fairly Determined, Ambitious, Professional, etc.
A player's Personality is determined by a combination of visible and hidden stats. A player's Determination and Teamwork, for instance, are visible stats. Hidden stats include Ambition, Consistency, Dirtiness, Injury Proneness, Controversy, and the four stats analysed by Quartz: Sportsmanship, Temperament, Professionalism, and Loyalty.
So, Quartz will have dug into the Football Manager editor to unearth these hidden stats for players, and used them as a basis for the study.
The Quartz article came up again early June 2020 amid the Black Lives Matter movement, prompting a Twitter response from Sports Interactive boss Miles Jacobson.
Jacobson said Sports Interactive "took the article seriously", and as a result "had a hard look" at its own numbers to see if there was some "unconscious bias".
Jacobson then accused the Quartz article of "a very disingenuous use of the numbers". Explaining that comment, Jacobson said:
"Firstly, these numbers are not simply made up; they are used as attributes for 'newgens' (i.e. fictional players which replace the real ones who retire) and are generated using averages of the players from the countries in question, as rated by researchers in those countries.
"At a casual glance, the numbers quoted in this piece would suggest a bias in certain areas against players from certain countries (note, not races, countries), but this is a very disingenuous use of the numbers.
"Looking at race, rather than nationality, players in the FM database are graded by 20 separate skin tones (for reason of their on-pitch representation in the 3D match models) with '1' as the lightest skin tone and '20' as the darkest.
"Using this skin tone metric, players with the darkest skin tones show a higher average rating for 'Ambition' and 'Temperament' and a lower rating for 'Controversy'. This paints a different picture than the one painted by the writer of the piece in question.
"People can come up with statistics to prove anything. Using our data, you could, for example, make an argument that Football Manager is biased against players from Iceland and, without too much of a struggle, find the numbers in our game to back this up.
"Essentially, it's always a mistake to start with your argument and then look for the figures to prove your point. Sports Interactive is strongly committed to the fight against discrimination and always has been. We absolutely refute the claims made in the Quartz article."
Fast forward to the end of June, and The i report, which claimed Sports Interactive edited the Football Manager database "after accusations of racial profiling".
"... i has learned that Sports Interactive has made changes to the game's database to eradicate any suggestion of profiling or negative stereotyping, having been made aware of the accusations," reads the article.
"While the company was confident that the existing system was not biased in any way, they have altered the process by which personality attributes of newgen players are created to reinforce a message of equality.
"Rather than using a template based on the averages of existing players of a certain nationality, all personality attributes of newgen players are now generated randomly for every nationality. So where before a Nigerian newgen would have personality attributes calculated according to every active Nigerian player in the game, that process no longer occurs."
However, Sports Interactive denied it made changes to Football Manager in response to the Quartz article, telling Eurogamer: "This simply isn't the case."
Sports Interactive communications director Ciarán Brennan told Eurogamer the game's newgen player stats were once derived from national averages - i.e. the average rating for every player from a specific country was calculated and used as the basis for the attributes of any newgen from that country. For example, if the average acceleration for a Scottish player during the 2010/11 season had been 11.5, every Scottish newgen's speed rating would have been tied to this figure.
But the system changed during the development cycle for Football Manager 2018, "because we'd decided that the results it produced weren't good enough."
The new system Sports Interactive moved to abandoned the nation-based averages and replaced them with a system that gives newgens a random set of attributes based on their role and position, Brennan said. And, crucially, Football Manager 18 was released in November 2017, and the Quartz article was published in June 2019.
"We utterly refute the allegations made in the original Quartz piece and also the inference made in The I's piece that we changed the game as a result of those allegations," Brennan said.
If Sports Interactive changed Football Manager's player stats system to a random number generator before Football Manager 18 came out, why does Quartz' study, published in June 2019, show evidence of racial bias? As Jacobson says, at a casual glance, the numbers quoted in the Quartz piece do indeed suggest racial profiling in Football Manager. But according to Brennan, the numbers referred to in the Quartz article were no longer being used in Football Manager by the time the article was published in June 2019. However, they were still present as "legacy data" in the game's editor - data that was based upon Sports Interactive's old system of generating newgen player stats from templates derived from national averages. This legacy data is no longer in the Football Manager editor, Brennan confirmed.
This leads on to The I's report, which, remember, claims Sports Interactive changed Football Manager's player stats system in response to the Quartz article. Daniel Storey, football writer, broadcaster and author of The I's article, told Eurogamer:
"I was reminded of the 2019 story when the study on the use of language surrounding Black players was published earlier this week. I sought to gain clarification from SI that changes had been made to alter the process of personality attribute calculation of newgen players, and they confirmed that to be the case."
The study referenced by Storey was published on Monday by Danish company RunRepeat in association with the Professional Footballers' Association. It found "evident bias" in some football commentary relating to the skin tone of players. In 80 televised games analysed across four European leagues, including the Premier League, players with a lighter skin tone were praised more often for their intelligence and work ethic. And those with darker skin tones were "significantly" more likely to be "reduced to their physical characteristics or athletic ability", such as their pace and power.