Dr. Kumud R. Jha · Singapore · Doctorate in AI · US Patent Holder View LinkedIn Profile
Let me start with a question about AI bias that sounds simple but isn’t:
if a mirror shows you something ugly, do you blame the mirror?
AI systems are, at their core, mirrors. They reflect the data we feed them — which means they reflect us. Our history. Our prejudices. Our blind spots. The inequalities we encoded into institutions long before any algorithm existed.
The problem is not that AI is biased. The problem is that we are tempted to treat its output as objective — and that temptation is dangerous.
Here is what that looks like in practice.
In 2018, Amazon scrapped an AI recruitment tool it had been building for years. The system had been trained on a decade of CVs submitted to the company — which, like most tech companies, had historically hired predominantly men. The AI learned the pattern. It began downgrading CVs that included the word “women’s” — as in women’s chess club, women’s debate society. It penalised graduates of all-women’s colleges. Amazon shut it down. But the lesson is not that Amazon built a bad AI. The lesson is that it built an AI that was very good at learning from a very biased dataset. Much like AI’s role in job displacement, the technology isn’t neutral — it reflects the choices of the people who built and deployed it.
In the United States, a risk assessment tool called COMPAS was used by courts to predict the likelihood of a defendant reoffending — informing decisions about bail, sentencing, and parole. An investigation by ProPublica found that the system labelled Black defendants as high risk at nearly twice the rate of white defendants with comparable records. The algorithm had no malicious intent. It had learned from historical criminal justice data — data generated by a system with its own well-documented racial inequities. The mirror, again, reflecting faithfully.
In healthcare, an algorithm used by US hospitals to identify patients who needed additional care was found to systematically underestimate the needs of Black patients. The reason was subtle: the algorithm used historical healthcare costs as a proxy for health needs. But Black patients had historically spent less on healthcare — not because they were healthier, but because they had less access to it. The proxy seemed neutral. The effect was not.
In the UK, the A-level grading algorithm used during the pandemic downgraded thousands of students from state schools relative to their predicted grades — while students from independent schools fared better. The system had been trained on historical school performance data. It reproduced historical advantage. Students who had never had a fair shot were told, by an algorithm, that their grades confirmed it.
And then there is the less visible bias — the kind embedded in who is not included at all. Facial recognition systems, trained predominantly on lighter-skinned faces, consistently show higher error rates on darker skin tones. In one landmark study, the error rate for darker-skinned women was 34% — compared to 0.8% for lighter-skinned men. These are not edge cases. These are the people the system was simply not designed to see.
Now here is the more uncomfortable argument.
AI reflects society. That is, in one sense, precisely what it is designed to do — learn patterns from human-generated data. The bias is not a bug introduced by malicious engineers. It is, in most cases, the faithful reproduction of structures that predate the technology by decades or centuries.
But here is where I want to push back against the easy conclusion: the fact that AI reflects society is not an excuse. It is the indictment.
We should not accept a technology that bakes historical injustice into automated, scalable, seemingly objective decisions — and then hides it behind algorithmic complexity that most people cannot interrogate. A biased human decision-maker can be questioned, challenged, overruled. A biased algorithm, deployed at scale across millions of decisions, compounds inequality silently and at speed.
AI should not be a mirror. It should be a lens — one that allows us to see our biases clearly, and then make a deliberate choice to do better.
The good news is that this is technically possible. Bias can be measured. Models can be audited. Representative datasets can be built. Diverse teams catch blind spots that homogeneous ones miss. The EU AI Act now classifies high-risk AI systems — including those used in hiring, credit, healthcare, and criminal justice — and mandates human oversight, transparency, and regular auditing.
These are not perfect solutions. But they are the right frame: AI bias is not a technology problem. It is a governance and accountability problem wearing a technology mask.
The question for all of us — whether we build AI, procure it, regulate it, or are simply subject to its decisions — is the same:
Who decided what this system should learn? And who is checking whether what it learned is just?
