New Call-to-action
What Is Bias in AI—and How Do You Prevent It?
Blog Feature

By: Mike Kaput

Print this Page

May 7th, 2021

What Is Bias in AI—and How Do You Prevent It?

Can artificial intelligence be biased?

You bet.

Despite AI's obvious benefits, it can end up harming consumers, brands, and industries through bias.

What do we mean when we say AI can be biased? What is bias in AI?

Put simply, bias in AI is when an AI system produces an unexpected, undesirable output.

Bias happens for two big reasons:

  • Human blindspots. Humans inject their own biases, consciously or unconsciously, into data used by AI or the AI system's design. These could include direct or indirect discrimination based on age, gender, sex, race, or other characteristics.
  • Incomplete data. Data used to train AI can also create bias when it's incomplete. AI is only as good as its training data. So, if that data lacks adequate diversity or comprehensiveness, the gaps cause issues.

Why can bias in AI be harmful? 

First is the obvious harm: AI can promote blatant bigotry.

An example of this is an AI bot that Microsoft created to post to Twitter.

The bot learned from conversations on Twitter how to tweet like a person. Unfortunately, much of the language used to train it was profane and bigoted.

As a result, the bot tweeted seriously inappropriate language. Microsoft shut the test down quickly.

AI can't invent bigotry on its own. It learns it from humans that display it in data sets used for training.

If Microsoft hadn't trained its bot on a dataset that included bigoted language, it wouldn't have been bigoted. It was a mistake, not a malicious action. The company didn't anticipate the consequences of using all of Twitter as a dataset.

Yet, it still harmed people and the company's image. The result was the same as if the company had intentionally programmed the bot to be biased.

That's why bias in AI is so dangerous.

Second, is the more common, but less obvious, harm: AI can become unintentionally biased due to incomplete data.

An example of this is the Apple Card.

In 2019, Apple released a credit card product. AI used by the company automatically gave applicants a credit line based on many characteristics, like spending, credit score, earnings, etc.

However, Apple took a massive amount of flak when it turned out that their AI gave women a smaller credit line than men, even when controlling for other factors.

It happened because the AI system was using incomplete data, which didn't account for a range of gender-related factors related to income and pay.

As a result, it concluded that women deserved less credit than men, even when financials were equal.

So, how do you address bias in AI?

Once you've built a product or system, it's usually too late.

You need to address bias at every step of the process that leads to the adoption of AI in products and operations.

The examples of Microsoft and Apple prove this. Both companies are adept at AI. Both companies have world-class engineering talent. Yet, they were both still caught by surprise by bias in AI. And when they discovered it, it was too late.

The technology was sound. But the bias considerations were not.

That's because fixing bias in AI isn't just a technology problem.

Sure, you need to be completely confident your data is comprehensive, accurate, and clean.

But you also need to have people and processes in place across every business function, not just engineering, to assess bias risks. It's a holistic effort, and it takes time.

An excellent place to start is to draft an AI ethics policy for your organization, whether you build AI technology or use it in your work.

An AI ethics statement is a formal document posted in public that outlines your company's position on AI.

It provides specifics on how your company will and won't use AI.

And it details what steps you take (or will take) to make sure ethical issues and bias don't affect AI you build or use.

New call-to-action

About Mike Kaput

Mike Kaput is Chief Content Officer at Marketing AI Institute and a senior consultant at PR 20/20. He writes and speaks about how marketers can understand, adopt, and pilot artificial intelligence to increase revenue and reduce costs. Full bio.

  • Connect with Mike Kaput
Disclosure: Marketing AI Institute writes about and recommends AI-powered marketing and sales technology. In all cases, content and recommendations are independent and objective. In some cases, Marketing AI Institute may have business relationships with companies mentioned, which may include financial compensation, affiliate compensation, or payment in kind for products or services. View a list of Institute partners here and MAICON sponsors here.