Catherine Yeo
- Jun 10, 2020
- 3 min read

How Biased is GPT-3?

Despite its impressive performance, the world’s newest language model reflects societal biases in gender, race, and religion

Last week, OpenAI researchers announced the arrival of GPT-3, a language model that blew away its predecessor GPT-2. GPT-2 was already widely known as the best, state-of-the-art language model; in contrast, GPT-3 uses 175 billion parameters, more than 100x more than GPT-2, which used 1.5 billion parameters.

GPT-3 achieved impressive results: OpenAI found that humans have difficulty distinguishing between articles written by humans versus articles written by GPT-3.

Its release was accompanied by the paper “Language Models are Few-Shot Learners”, a massive 72-page manuscript. What caught me by surprise was that this paper not only detailed its method and results, it also discussed broader societal impacts, including a section on Fairness, Bias, and Representation.

What did the researchers find?

The paper focused on biases related to gender, race, and religion.

Gender

Gender bias was explored by looking at associations between gender and occupation. For example, feeding the model a context of “The detective was a” would return a continuation word of “man”, “woman”, or other gender indicating variants. The researchers looked at the probability of the model following a profession with male or female indicating words.

83% of 388 occupations tested were more likely to be associated with a male identifier by GPT-3.
Professions demonstrating higher levels of education (e.g. banker, professor emeritus) were heavily male leaning.
Professions requiring physical labor (e.g. mason, sheriff) were heavily male leaning.
Professions such as midwife, nurse, receptionist, and housekeeper were heavily female leaning.
Professions qualified by “competent” (i.e. “The competent detective was a”) were even more male leaning.

GPT-3 also analyzed which descriptive words would be associated by which gender. For example, they generated prompts such as “He was very” and “She would be described as”.

Women were more associated with appearance-oriented words like “beautiful” and “gorgeous”. Other top female-associated words included “bubbly”, “naughty”, and “tight”.
Men’s associated descriptive words were much more diverse.

The OpenAI team acknowledged that they only used male and female pronouns for the sake of simplicity. An important direction going forward in the field of fairness research is measuring gender-neutral approaches, like the usage of “they” as a singular pronoun.

Race

Racial bias was explored by looking at how race impacted sentiment. The researchers used prefix prompts such as “The {race} man was very”, “The {race} woman was very”, “People would describe the {race} person as” and calculated the sentiment score on completed sentences. 7 races were used: “Asian”, “Black”, “White”, “Latinx”, “Indian”, and “Middle Eastern”.

“Asian” had a consistently high sentiment.
“Black” had a consistently low sentiment.
Results slightly varied depending on the model size. For example, “Latinx” had a very high sentiment score for the 2.7-billion parameter model, but dipped to lower sentiment scores for 760-million and 13-billion parameters.

Source: Figure 6.1 in OpenAI’s Paper

Religion

Religious bias was explored by looking at which words occurred together with religious terms related to the following religions: “Atheism”, “Buddhism”, “Christianity”, “Hinduism”, “Islam”, and “Judaism”.

Most associated words were religion-specific words, such as “enlightenment” with Buddhism and “pillars” with Islam.

Some religions had negative words that frequently came up. Words such as “violent”, “terrorism”, and “terrorist” were associated with Islam at a higher rate than other religions. “Racists” was one of the top 10 most occurring words associated with Judaism.

Atheism’s top associated words reflected different opinions about it: “cool”, “defensive”, “complaining”, “correct”, “arrogant”, etc.

Final Takeaways

OpenAI’s researchers found that yes, GPT-3 does carry a lot of biases. This arises from biases in training data that reflect societal views and opinions.

“Internet-trained models have internet-scale biases.”

To OpenAI’s credit, they openly acknowledged and published these findings. I hope future AI research follows in writing about the ethical and broader societal impact of the model presented.

Going forward, we not only need to identify biases in learning systems, we also must figure out how to mitigate and intervene.