"Smart" vs "Athletic": how AI reveals the unconscious racial bias in our language

It's easy to imagine sports commentators have moved on from the sort of offensive comments that received widespread attention in the 1980s. But AI analysis shows how an old bias endures in language differences

January 28, 2020
The way commentators talk about players of different races has been the subject of AI investigation. Photo: PA/Prospect composite
The way commentators talk about players of different races has been the subject of AI investigation. Photo: PA/Prospect composite

There is a popular sentiment that “fairness” is inexorably subjective and incapable of being determined by objective standards.

Similarly, when it comes to sports commentary, one would think that the protagonists’ analysis will be based purely on the state of play taking place on the pitch, field or court—but that’s simply not the case.

Sports pundits have been known to exert racial bias and its impact is very damaging. Racial bias in sport can cause a negative picture to be painted of a particular athlete and may mean they don’t get selected to play for certain teams.

In 1988, Jimmy “The Greek” Snyder, a CBS lead pundit, famously attributed a black athlete’s “superior” talents to the legacy of American slavery:

[su_quote]The black is a better athlete to begin with, because he's been bred to be that way. Because of his high thighs and big thighs that goes up into his back. And they can jump higher and run faster because of their bigger thighs. And he's bred to be the better athlete because this goes back all the way to the Civil War, when, during the slave trading, the owner, the slave owner would breed his big black man to his big woman so that he could have uh big black kid, see. That's where it all started.[/su_quote]

Snyder believed that black people were superior when it comes to sports. Physically they overpowered their non-black teammates and competitors. “They’ve [blacks] got everything,” he said. “If they take over coaching like everybody wants them to, there’s not going to be anything left for white people.”

This controversy forced CBS to sever ties with Snyder—and also put pressure on other sports commentators to choose their words wisely and try to comment without bias.

A new era?

Snyder made his remarks over 30 years ago and since then, the world of sports has evolved somewhat. But stereotyping and bias are still very much at large—and research proves it.

We know that technology such as facial recognition and Artificial Intelligence (AI) has the potential to be racially motivated, primarily down to the algorithms that are inputted into the system. As computer scientist Aylin Caliskan said, “The machines learn from their creators—us.”

But how about those same systems being used to prove the racial bias of their users?

Mohair Iyyer, an assistant professor of computer science at the University of Massachusetts Amherst, used technology to prove that sports commentators were still racially biased in their commentary, some 20 years on from Snyder’s comments. Iyyer used Artificial Intelligence and “big data” analytics to help answer the question of whether racial bias is perpetuated within sports commentary.

The answer was a resounding yes.

In 2019, using transcripts from 1,455 NFL and NCAA football games, as well as YouTube videos, Iyyer and his colleague Jack Merullo used data mining to assess whether their hypothesis of bias in sports commentary was in fact true. The data they looked at covered from 1960 to 2019.

Breaking down the data

Once the data was collected they began analysing the information. Each time a player was mentioned they linked it with their race. For example, if they heard, “Von Miller” or “Miller” they assigned that player a racial identifier (“white” or “non-white”). As a result of this, patterns started to emerge that showed a link between the player’s race, the position they played and how they were referred to. One key finding was that commentators referred to non-white quarterbacks by their first name more often than white players (18.1 per cent for non-white vs. 8.3 per cent for white).

Iyyer's research showed that there was a consistency with how commentators discussed players that tallied with their race—thus showing that there is an unconscious bias in the way that pundits talk about athletes.

So how did they reach their result?

Iyyer examined this all by using natural language processing algorithms and carried out a sentiment analysis—which is when contextual mining of text identifies and extracts subjective information in the source material. This means it examines the positive or negative words within a sentence describing something to draw out the overall sentiment of that description.

So, if a person is described using a variety of negative or positive words, sentiment analysis will give you a good analysis of the overall point of view surrounding that individual.

In 1988, Raymond E Rainville and Edward McCormick examined the relationship between commentator sentiment and player race, in a study entitled Extent of Covert Racial Prejudice in Pro Football Announcers' Speech. This study concluded that white players receive more positive coverage versus black players, where it was more negative.

Ability versus safety

To examine their patterns, Iyyer and Merullo assigned a binary sentiment label to contextualized terms (i.e., a window of words around a player).

The words most associated with white quarterbacks were: cool, smart, favourite, safe, spectacular, excellent, class, fantastic, good, interesting. 

The positive words most associated with non-white quarterbacks were: ability, athletic, brilliant, awareness, quiet, highest, speed, wow, excited, wonderful.

The words associated with non-white players focused much more on ability—and the words associated with white players looked at the character and performance of the person.

The study confirmed that non-white players are much more frequently praised for physical ability than white players, who are praised for personality and intelligence.

Racial bias is a common occurrence in sports and it can be a very difficult thing to manage.

“It’s really hard to get sports journalists to consider the words, the framing and the context that their stories are put in,” says Dr Cynthia Frisby, a professor at the University of Missouri’s School of Journalism.

In 2015, Frisby published a study that showed how black male athletes received much more negative coverage and commentary, compared to their white counterparts who were seen as heroes.

Ultimately, sports are a reflection of the environment and society that we live in—and as a result, it’s no surprise that there is racial bias in sports commentary. When racism runs rampant in our daily life, it will obviously enter into the very systems and institutions we find joy, happiness and freedom in.

Iyyer's analysis includes over 260,000 mentions of 4,668 unique players, and at least two-thirds of them are non-white. Analysing this data was not easy. But ultimately, using AI, analysing ‘big data’ sets and algorithms can paint a solid picture of how sometimes unconscious bias can play out—and help address patterns we may otherwise hardly even notice.