<img height="1" width="1" style="display:none" src="https://www.facebook.com/tr?id=2006193252832260&amp;ev=PageView&amp;noscript=1">

4 Min Read

The Hidden Threat to More Intelligent Marketing

Featured Image

Wondering how to get started with AI? Take our on-demand Piloting AI for Marketers Series.

Learn More

“We all know women deserve less money than men, right?”

If machines could talk, that might be a direct quote from the algorithm that determines credit limits for the Apple Card, a new financial product from the tech giant.

In late 2019, entrepreneur David Heinemeier Hansson (cofounder of Basecamp) tweeted about how he had been approved for 20 times more credit than his wife when they both applied for the Apple Card, due to the algorithm that regulates how credit limits are determined.

The entrepreneur took to Twitter, where his story went viral—prompting a response from Apple.

What happened next has serious implications for business leaders in the age of artificial intelligence.

When asked, multiple Apple reps didn’t know how the algorithm worked, Hansson reported on Twitter. And, while they were respectful of concerns around discrimination, they blamed the algorithm for the issue.

As Hansson’s complaint moved up Apple’s chain of command, it appeared management didn’t have any visibility into how the algorithm worked either. Nor did they know how to fix the problem.

All the while, Hansson torched the brand online.

The issue became such a big deal that Apple cofounder Steve Wozniak chimed in, saying the same issue happened to him and his wife.

It turned out, Apple didn’t even build the algorithm, either.

Apple isn’t a bank, so it partnered with Goldman Sachs to launch the Apple Card. Goldman was responsible for creating the algorithm.

So, Goldman Sachs got put on blast, too. In response, Goldman released a short statement saying it didn’t endorse gender bias. The ineffectual statement didn’t go over...well.

Oh, Goldman is also now subject to a probe by the New York Department of Financial Services over the issue.

The final straw came when Wozniak also made it clear that Apple, the company he cofounded, bore some blame for the incident.

And this all played out in a massively public conversation.

Is your head spinning yet?

A single instance of algorithmic bias caused:

  1. Serious moral issues by intentionally or accidentally enabling financial discrimination against women.
  2. Major damage to brand equity for Apple (which is famed for its brand strength) — over an algorithm it didn’t create and didn’t understand.
  3. Possible legal repercussions for Goldman Sachs — over an algorithm it did create and didn’t understand.

If you’re a business leader who doesn’t think AI is worth paying attention to, then you’re not paying attention.

If you’re not using AI right now, you may be considering a strategy and adoption plan. And, even if you’re not planning for AI, I guarantee you at least one of the products or services your business works with uses algorithms.

Which means bias in artificial intelligence presents a range of complex challenges for you and your company. Challenges you’re probably not prepared for right now.

Bias in AI usually happens because of the data used by the AI system, not the system itself.

In the case described here, the data might intentionally or accidentally reflect human biases about different genders, races, or groups. For whatever reason, the data processed by the algorithm is skewed in a way that results in women being perceived as more of a credit risk than men.

But the data can also be biased in more innocent ways. 

Say you purchase an AI system trained to write highly engaging social media posts. The technology looks great: it’s been trained on over two million successful posts and is going to up your social engagement while cutting time spent on social by 90%.

What’s not to like?

Do you know anything about the two million posts the AI tool was trained on? Were the posts from businesses that are like yours? Were the posts in the language you do business in? What platforms were the posts on? 

Good luck finding out. And you better pray it works for you after your name goes on the invoice.

The reality is this:

If your company’s products use AI…

If you are actively investigating AI for your brand…

Or if you partner with anyone using AI…

You’re in the exact same position that Hansson and his wife were in.

You don’t know if the data used by algorithms that affect your brand is biased.

If it is biased, you don’t have a full picture of how or why the data is biased.

And you probably don’t have a strategy for what to do and say if something goes wrong because of bias.

You’re not alone. 

This is a really hard problem that even the people building the world’s most sophisticated algorithms haven’t figured out.

No one has the answers yet. 

But it is time you start asking questions to better understand how your brand interacts with algorithms.

Because if your brand gets entangled with algorithms that have serious bias problems, it’s going to come back on you — likely at lightning speed, in a public forum, where all your customers can see.

And when you’re asked about who bears responsibility for a sexist, racist, bigoted, or just plain wrong machine-assisted outcome…

Saying “It’s just the algorithm” isn’t going to cut it.

Related Posts

How Are Natural Language Applications Solving Real Marketing Challenges?

Neil Yager, PhD | November 18, 2019

AI is not magic. It is a scientific answer to real-world business challenges. Here's how to use it for real marketing problems.

How Machine Learning Algorithms Can Spark Creativity in Your Company

Bethany Eierdam | May 19, 2021

Machine learning algorithms can help your content marketing stand out among your competitors. Learn how to use this AI to improve your creativity.

McKinsey Predicts AI Will be as Impactful as the Steam Engine

Ashley Sams | September 12, 2018

Our team reads tons of AI and machine learning news every week so we can share the top stories with you. Machine learning articles worth reading this week include McKinsey's AI predictions, Google's new What-IF machine learning tool, and some of the most notable machine learning use cases.