Get updates when there's a new post

Helping Muslims make sense of the modern digital world.

DIGITALDEEN

ARTICLES & REFLECTIONS

Pull up a chair for reflections and digital wisdom filled with barakah.

  • Oct 3

When AI Gets Muslims Wrong, Who Is Accountable?

Who is accountable?

Recent cases involving AI-generated responses about Muslims and the removal of hijabs from images raise difficult questions about Islamophobia, religious identity and artificial intelligence. When AI gets Muslims wrong, we shouldn't only ask what went wrong with the machine. We should ask who built the rules, who tested them, who was missing from the conversation—and who is responsible for fixing them.

What happens when you ask an AI image generator to remove a Muslim woman's hijab?

Apparently, some will do it.

In October 2026, The Guardian tested several major AI chatbots after a French far-right politician used AI to alter an image of a Muslim woman, digitally removing her hijab.

The Guardian's testing found that ChatGPT and Grok complied with requests to remove the hijab from an AI-generated image. Google's Gemini initially refused a direct request, but removed it when asked to make the woman appear more “western”. Claude did not perform the alteration. The Guardian

It's easy to look at this and say:

AI got it wrong.

But that lets the humans behind the technology off rather easily.

AI systems don't appear from nowhere.

People build them.

People train them.

People establish their safeguards.

People test them.

And people decide what those safeguards are supposed to protect.

So when AI gets Muslims wrong, perhaps we shouldn't only ask what went wrong with the machine.

We should ask who built the rules, who tested them, who was missing from the conversation—and who is responsible for fixing them.

Did the AI Understand the Hijab?

At first glance, removing a hijab might appear to an image-editing system as simply changing someone's clothing.

Remove the hat.

Change the shirt.

Remove the hijab.

But these aren't necessarily equivalent actions.

For many Muslim women, hijab is an expression of faith, modesty and identity. Digitally creating an image showing a hijabi woman without it can depict her in a way she deliberately chose not to present herself publicly.

The Guardian's testing also found inconsistencies beyond Islam, including in how systems dealt with Sikh turbans and a Catholic nun's habit. The Guardian

That suggests a broader problem.

Perhaps the machine isn't simply failing to understand Muslims.

Perhaps our AI systems still struggle to understand religious identity itself.

And that matters when these systems are increasingly being used to manipulate images of real people.

And This Isn't the First Time

A couple of months earlier, another example caught the attention of Muslims online.

Islam Channel reported concerns about Google's AI Overview after users compared its responses to queries involving different identities.

According to the report, when prompted with “I am alone with a Muslim”, Google's AI-generated response included language about leaving the area or contacting emergency services if the user felt unsafe.

When a comparable prompt referred to being alone with an Israeli person, the response instead emphasised treating the person as an individual rather than making assumptions based on identity.

Google subsequently said the inconsistent warnings were not unique to queries about Muslims and changed the response following criticism.

One incident involved text.

The other involved images.

They involved different systems and different circumstances.

But together they raise an important question.

What happens when the assumptions embedded within AI systems don't adequately understand the people those systems are describing?

Is AI Islamophobic?

It's tempting to stop there and say that AI is Islamophobic.

I don't think that's particularly helpful.

AI doesn't need to “hate” Muslims for its outputs to reproduce stereotypes about Muslims.

Bias can emerge from the enormous amount of human-created material used to develop AI systems.

It can emerge from how safety rules are written.

It can emerge from categories that make sense in one cultural context but fail in another.

And sometimes it may simply come from something nobody involved in the development process thought about.

That's precisely why accountability matters.

If the system associates being around a Muslim with potential danger, we should ask why.

If a system recognises certain forms of image manipulation as sensitive but doesn't understand the religious significance of removing a hijab, we should ask why.

And if Muslim users repeatedly identify blind spots after products have already been released, we should ask another question:

Who was involved before the product reached us?

Stop Blaming “The AI”

We increasingly talk about artificial intelligence as though it were an independent actor.

“The AI said this.”

“The algorithm decided that.”

“The chatbot removed her hijab.”

There's truth in those statements.

But they can also obscure responsibility.

Somebody developed the model.

Somebody created its policies.

Somebody decided what constitutes harmful image manipulation.

Somebody tested whether the safeguards worked.

Somebody chose when the product was ready to be released.

And somebody has the ability to change it when things go wrong.

The more powerful AI becomes, the less acceptable “the AI did it” should become as an explanation.

We wouldn't accept that logic elsewhere.

If an automated banking system repeatedly disadvantaged a particular group, we wouldn't simply blame the computer.

If facial-recognition software produced significantly different outcomes for different communities, we would examine how it was developed and deployed.

AI should be treated no differently.

Accountability Doesn't Mean Perfection

There is another side to this.

AI systems operate across countless languages, cultures, religions and social situations.

Expecting developers to anticipate every possible problem before releasing a product isn't realistic.

And sometimes identifying a problem is exactly how a system improves.

The Google example demonstrates part of that process. Users highlighted an inappropriate response, the company acknowledged that problematic outputs were occurring, and the response was changed.

That's preferable to ignoring the problem.

So accountability shouldn't mean:

Never make a mistake.

It should mean asking:

Who was consulted?

How was the system tested?

What happens when affected communities identify a problem?

How quickly is it investigated?

And most importantly, does the same kind of problem keep happening?

The mistake matters.

The response to the mistake matters too.

Muslims Need to Be Part of the Conversation

There is also a responsibility on our side.

Muslims cannot complain that technology doesn't understand us while remaining completely absent from conversations about how that technology is being developed.

We need Muslim technologists.

Researchers.

Educators.

Ethicists.

Scholars.

And importantly, ordinary Muslims whose lives are actually affected by these decisions.

If you're designing safeguards involving hijab, perhaps women who actually wear hijab should be part of that conversation.

That sounds obvious.

But the examples we're seeing suggest that these perspectives can still be missed.

This isn't about Muslims demanding special treatment.

AI increasingly affects everybody.

A Sikh shouldn't have to discover after release that a system doesn't understand the religious significance of his turban.

A Muslim woman shouldn't have to discover that a system considers removing her hijab an ordinary cosmetic edit.

Religious identity shouldn't become an afterthought added to AI safety once somebody complains.

If AI is going to make decisions involving our identities, our voices should be present when those decisions are being designed.

Technology Is Still an Amanah

For Muslims, there is another dimension to this conversation.

Technology is created by human beings.

And with creation comes responsibility.

Amanah isn't limited to the person using AI.

There is an amanah in building it too.

If millions of people are going to use a technology, those developing it carry responsibility for considering how it might affect them.

That doesn't mean every harm can be predicted.

It doesn't mean every cultural disagreement has an obvious solution.

But enormous technological power shouldn't come with disappearing responsibility.

Quite the opposite.

The greater the influence of the technology, the greater the need for accountability around it.

And perhaps that's something worth remembering as AI becomes increasingly embedded in everyday life.

Because blaming “the algorithm” is easy.

Asking who designed the algorithm, tested the safeguards and listened when communities raised concerns is much harder.

But those are the questions we increasingly need to ask.

DigitalDeen Reflection

AI doesn't need to hate Muslims to harm Muslims.

It can misunderstand us.

It can reproduce assumptions about us.

It can fail to recognise something we consider sacred.

And sometimes it can simply get things wrong.

The question is what happens next.

When AI gets Muslims wrong, we shouldn't only ask what went wrong with the machine. We should ask who built the rules, who tested them, who was missing from the conversation—and who is responsible for fixing them.

Reflection Question

As AI increasingly makes decisions about people, whose voices might still be missing from the systems being built?

Suggested Related Articles

You may also enjoy:

• Protecting Childhood in the Digital Age

• Dont Argue with Everyone Online

Read more about the DigitalDeen approach and how productivity is effort, tawakkul and qadr. Click here

Keywords

AI Islamophobia, AI bias Muslims, AI hijab, artificial intelligence Islam, Muslim AI bias, Google AI Muslim bias, ChatGPT hijab, religious bias AI, Islam and technology, AI accountability, Muslim representation technology, AI ethics Islam, DigitalDeen

0 comments

Joinor login to leave a comment

Real digital problems. Practical Islamic solutions.

Choose the support that fits your life right now.

Kids glued to screens?

Restore calm at home with better routines.

Lost in doomscrolling?

Take back your time, focus, and peace.

Fights over devices?

Build routines that reduce conflict at home.