• Wion
  • /Technology
  • /OpenAI’s CLIP shows bias for wealthy, scores poor African nations low, finds study

OpenAI’s CLIP shows bias for wealthy, scores poor African nations low, finds study

OpenAI’s CLIP shows bias for wealthy, scores poor African nations low, finds study

Artificial Intelligence words are seen in this illustration taken March 31, 2023

The low-income and non-Western lifestyles are portrayedless accurately by OpenAI's CLIP, an Artificial Intelligence (AI)model integral to thethe popular DALL-E image generator.

CLIP stands forContrastive Language-Image Pretraining. A foundational model used in various applications, CLIP operates by pairing text and images to create a score indicating the degree of alignment between the two.

The conclusion was made in a recent study conducted at the University of Michigan.

Add WION as a Preferred Source

What does it mean?

According to the researchers involved in the study, there is acritical need for comprehensive representation in AI tools deployed globally.

Also watch |Future of AI: Benefits & Challenges

The study showed that asignificant portion of the population, particularly those with lower social incomes, is not accurately reflected in these applications, potentially furthering the inequality.

How the study was done?

The researchers evaluated CLIP's performance using Dollar Street, a diverse image dataset from the Gapminder Foundation, with an image-base of over 38,000 whichdepicts households across different income levels globally.

Monthly incomes in the dataset ranged from $26 to nearly $20,000, and thus provideda broad socio-economic spectrum.

Results of the study showed that CLIP consistently assigned higher scores to images from higher income households, indicating a significant bias.

The study also identified geographic bias, with lower scores predominantly associated with low-income African countries.

This bias could potentially result in the underrepresentation of diverse demographics in large image datasets and applications relying on CLIP.

Is there any solution to rectify this AI bias?

Researchers proposed actionable steps for AI developers, including investing in geographically diverse datasets, defining evaluation metrics that account for location and income, anddocumenting the demographics ofdata on which the AI is trained.

The study presented its findings at the Empirical Methods in Natural Language Processing conference, held on December 8 in Singapore. The paper is available on the arXiv preprint server.

(With inputs from agencies)