# The Bias Machine: How AI Hiring Tools Discriminate and What We Can Do About It

> A comprehensive investigation into algorithmic discrimination in AI-powered recruitment. With research showing AI systems prefer white-associated names 85% of the time, landmark lawsuits reshaping the legal landscape, and new regulations from NYC to the EU demanding accountability, we examine the evidence, the cases, the technology, and the path forward for organizations navigating the most consequential ethics challenge in modern hiring.

- Published: 2025-12-29
- Author: Gene Dai
- Canonical: [https://digidai.github.io/2025/12/29/ai-hiring-bias-algorithmic-discrimination-fairness-2025/](https://digidai.github.io/2025/12/29/ai-hiring-bias-algorithmic-discrimination-fairness-2025/)
- Topics: ai hiring bias, algorithmic discrimination, ai recruitment fairness, workday lawsuit, hirevue bias, eu ai act hiring, nyc local law 144, eeoc ai enforcement, ai resume screening bias, debiasing ai recruitment

---

<p>
<em>
The rejection email arrives at 6:47 AM. Still in bed, phone in hand,
doing what job seekers train themselves not to do—checking for responses
before their first coffee. Bad habit. They know better.
</em>
</p>
<p>
<em>
"We have reviewed your application and have decided to move forward with
other candidates whose qualifications more closely match our current
needs."
</em>
</p>
<p>
<em>
Seven hours. Submitted at 11:52 PM the night before, after rewriting the
cover letter for the third time, after convincing yourself that this one
felt different. Seven hours is all it takes for the system to evaluate
fifteen years of experience. Fifteen years. Seven hours. Rejection.
</em>
</p>
<p>
<em>
The name experiments have become documented phenomena in job seeker
communities and academic research: submit the exact same resume to a
different opening at the same company. Same qualifications. Same
projects. Same everything. Except change the distinctively Black name to
something racially ambiguous. Remove the HBCU. Replace it with "State
University." Strip out the professional organization affiliations.
</em>
</p>
<p>
<em> The interview request comes in days instead of never. </em>
</p>
<p>
The pattern is well-documented in academic research and job seeker
communities: professionals with decades of experience, senior roles at
major companies, systems-level expertise—running informal experiments on
their own job searches and confirming what they suspected.
</p>
<p>
"The thing is, I already knew," is the common refrain in these accounts.
"Before I ran the experiment. I just needed to see it. To have something I
could point to that wasn't just a feeling."
</p>
<p>
The University of Washington research confirmed what job seekers had been
documenting: the AI systems that screen job applications in America are
systematically discriminating against Black candidates.
</p>
<p>
I should tell you something before we go further. I run an AI-powered
recruitment platform. I sell the exact kind of technology this article is
going to criticize. My company's revenue depends on employers believing AI
makes hiring better.
</p>
<p>So why am I writing this?</p>
<p>
Here's the honest answer: I don't know if I should be. Three weeks ago, we
got a term sheet from a Series A investor. $8 million. Life-changing
money. The kind that turns a struggling startup into a real company. And
two days ago, I got an email from our biggest potential client—a Fortune
100 retailer—asking us to remove bias auditing from our contract because,
quote, "it creates unnecessary legal exposure."
</p>
<p>I have to respond by Monday.</p>
<p>
So I'm writing this instead. Maybe as a way of figuring out what to do.
Maybe as penance for what I've already done. Maybe because I can't stop
thinking about the job seekers I've talked to, about what they said when
they described realizing they "already knew" before they ran the experiment.
</p>
<p>
I've sat in sales calls where clients asked about bias and I gave answers
that were—let me be honest—bullshit. Carefully worded bullshit, the kind
that's technically defensible but not actually true. I'll get to that.
</p>
<p>
I don't have this figured out. I'm going to tell you what I know, what
I've seen, and what keeps me awake. You can judge for yourself what to do
with it.
</p>
<h2>The Research Nobody Wanted to Talk About</h2>
<p>
In October 2024, researchers at the University of Washington published a
study. It should have been front-page news. It wasn't. A few tech
publications ran stories. HR blogs mentioned it. LinkedIn had a discourse
cycle that lasted about 72 hours. Then everyone moved on.
</p>
<p>I didn't move on. I couldn't.</p>
<p>
They tested how three leading large language models—GPT-4, Claude, Gemini,
the same systems increasingly used to screen resumes—ranked identical
candidates with names associated with different racial and gender groups.
Same resume. Same experience. Same education. Different name.
</p>
<p>AI systems preferred white-associated names 85% of the time.</p>
<p>Black-associated names? Nine percent.</p>
<p>
And the intersectional findings—this is the part I keep coming back to—
when comparing white male names to Black male names, the AI systems
preferred the Black male name exactly zero percent of the time.
</p>
<p>In thousands of comparisons. Zero.</p>
<p>
Let that sink in. If a Black candidate is competing against a white
candidate with the exact same resume, the AI chooses the white candidate
every time. Doesn't matter what the Black candidate has done. Doesn't
matter how good they are. They lose every time.
</p>
<p>
I want to make sure that lands. An identical resume. Identical
qualifications. The only difference is whether the name at the top sounds
Black or white. And the machines we've built to be "objective" reject the
Black candidate every single time.
</p>
<p>
99% of Fortune 500 companies use some form of automation in hiring. Which
means most job applications in America are filtered through systems with
documented, severe racial bias.
</p>
<p>
I showed this study to my co-founder when it came out. His response:
"Well, our system is different."
</p>
<p>
I didn't argue. I should have. I wanted to believe him. Still do, some
days.
</p>
<h2>The Argument I Keep Having With Myself</h2>
<p>
Here's the thing—and I go back and forth on this constantly, sometimes in
the same hour—is algorithmic bias actually worse than human bias?
</p>
<p>
Humans have always discriminated. A racist hiring manager might reject a
few hundred candidates over a career. At least with AI, you can audit it.
You can measure it. You can—theoretically—fix it. That's the pro-AI
argument, and it's not crazy.
</p>
<p>
The industry makes this case aggressively: "You know what's biased? The HR
recruiter who went to Michigan and only hires people who went to Michigan.
At least an algorithm doesn't care where you went to college."
</p>
<p>
Except algorithms absolutely do care where you went to college, because
they're trained on data from those same biased recruiters.
</p>
<p>
"Then we fix the training data. We iterate. That's what engineering is."
The counter-argument: going back to gut feelings and networking—the system
that gave us country clubs and old boys' networks—isn't better.
</p>
<p>
It's not wrong. That's what makes this hard. The alternative to AI hiring
isn't some bias-free utopia. It's the system we had before, which was also
deeply biased, just in ways that were harder to measure.
</p>
<p>
But here's where I part ways with the industry consensus: scale. A biased
hiring manager affects hundreds of people. A biased algorithm affects
millions. And it does it while wearing the disguise of objectivity. People
trust machines in ways they don't trust human recruiters. That trust is a
kind of violence when the machine is just human prejudices encoded in math.
</p>
<p>
The job seekers running these experiments see through the philosophy:
"You're debating whether AI bias is worse than human bias while people's
lives are being ruined. Every day that algorithm runs, someone doesn't get
an interview. Doesn't get a job. Can't pay their mortgage. Starts doubting
themselves."
</p>
<p>I don't have a response to that.</p>
<h2>What I've Seen From the Inside</h2>
<p>
Six months ago, we had a product meeting. Standard stuff—reviewing our
screening algorithm's performance, talking about improvements. Our head of
engineering pulled up a chart showing selection rates by demographic
group. (We don't ask for demographic data directly, but you can infer a
lot from names and zip codes, which is part of the problem.)
</p>
<p>The numbers weren't good.</p>
<p>
I remember the silence. The way people suddenly found their laptops very
interesting. And then someone—I'm not going to say who—said: "Well, we're
not asking for demographic data, so technically we're not discriminating."
</p>
<p>Technically.</p>
<p>
God, I hate that word. I've heard it a lot in this industry. We're
technically compliant. We're technically not using protected
characteristics. We're technically following the law.
</p>
<p>
But the algorithm was finding proxies. It always does. Zip codes. College
names. Membership in certain professional organizations. The system had
learned that candidates from HBCUs had lower "success rates" in our
training data—because the companies whose historical hiring data we
trained on had hired fewer HBCU graduates. Not because HBCU graduates were
less qualified. Because decades of discrimination had created a dataset
that encoded discrimination as a feature.
</p>
<p>
We tried to fix it. Spent three months retraining. Brought in
consultants—$40,000 for two weeks of work, which is roughly what we pay
our engineers in three months. Did bias audits. Our numbers look better
now.
</p>
<p>
But here's what I don't know—what keeps me up at 3 AM scrolling through
error logs—what other proxies are still in there that we haven't found
yet? What patterns has the machine learned that we don't even know to look
for?
</p>
<h3>The Amazon Warning</h3>
<p>
This isn't new. In 2014, Amazon's engineers thought they'd solved hiring.
They built an AI trained on ten years of resumes, designed to identify
candidates who looked like successful Amazon employees.
</p>
<p>
The result was predictable in retrospect: the system learned that male
candidates were preferable. It penalized resumes with the word
"women's"—as in "women's chess club captain." It downgraded graduates from
women's colleges. It favored certain verbs—"executed," "captured"—that
appeared more frequently on male engineer resumes.
</p>
<p>
Amazon's engineers tried to fix it. They made the system neutral to
specific terms. They reweighted features. Nothing worked reliably. The
algorithm kept finding new proxies.
</p>
<p>By 2017, the team was disbanded.</p>
<p>
That was eight years ago. The industry's response was essentially: "Well,
that was Amazon's problem. Our systems are different."
</p>
<p>
They're not. They're trained on the same historical data. They make the
same mistakes. The only difference is that now we have research
quantifying exactly how bad it is.
</p>
<p>And we're still deploying these systems. Including mine.</p>
<h2>The Sales Call I'm Not Proud Of</h2>
<p>I need to tell you about something that happened in September.</p>
<p>
We were on a sales call with a Fortune 500 financial services company. Big
deal—$1.2 million annual contract, the kind that would change our revenue
trajectory. Their head of talent acquisition asked a direct question: "How
do you ensure your system doesn't discriminate?"
</p>
<p>
I had a slide for this. I talked about our bias audits. I mentioned that
we test for demographic parity across race and gender. I referenced the
NYC Local Law 144 methodology.
</p>
<p>
She pressed: "But the University of Washington study—the one showing zero
selection rate for Black male names. How do you address that?"
</p>
<p>
And here's what I said: "That study tested general-purpose LLMs, not
purpose-built recruitment systems. Our approach is specifically designed
to avoid those failure modes."
</p>
<p>
Which is... true, in a narrow sense. We don't use raw LLMs for final
screening decisions. But it's also misleading, because our system is built
on top of language models, and those models carry the same biases. I
emphasized the technical distinction while glossing over the substantive
concern.
</p>
<p>She seemed satisfied. We got the deal.</p>
<p>
I've thought about that call a lot. I didn't lie. But I didn't tell the
whole truth either. I gave an answer that made the problem sound solved
when it isn't.
</p>
<p>
That's the thing about selling AI tools. You learn to speak in a way
that's defensible without being honest. "Our systems are tested for bias"
(true, but the tests don't catch everything). "We follow industry best
practices" (true, but industry best practices are inadequate). "We're
committed to fairness" (true, but commitment doesn't equal achievement).
</p>
<p>
The uncomfortable truth, which job seekers point out regularly: I knew our
system probably exhibits similar bias patterns. And I sold it anyway. To a
company that's going to use it to screen millions of applications.
</p>
<p>
What should I have said? "We don't know if our system discriminates, and
neither does anyone else, because the auditing methods don't work well
enough"? That's closer to the truth. It's also not how you close a
seven-figure deal.
</p>
<p>
The question that haunts me from these conversations: does my career matter
more than whether someone's kids can eat?
</p>
<p>I don't have an answer. I still don't.</p>
<h2>The Engineers Who Built What They Knew Was Broken</h2>
<p>
Anonymous accounts from ML engineers at major HR tech companies tell a
consistent story. These companies' tools screen millions of applications
per year. The engineers who build them see the problems firsthand.
</p>
<p>
"I knew from the moment I saw the training data," is the common refrain.
"Five years of hiring decisions from Fortune 500 clients. You know what
that data reflects? The biases of whoever was hiring five years ago. Ten
years ago. Twenty. The whole pipeline was built on the assumption that
past hiring decisions were correct. That the people who got hired deserved
it. That the people who got rejected deserved to get rejected."
</p>
<p>
When these engineers raise concerns, the response is predictable. Managers
"note the concern" but the timeline is tight. Clients are waiting. The
sales team has made promises. They can "iterate on fairness" in version
2.0.
</p>
<p>
Version 2.0 comes eighteen months later. Better UI. Faster processing.
No changes to the model.
</p>
<p>
The engineers describe their rationalization: "I think about the people
who didn't get interviews because of something I built. Our system screens
two million applications a year. If even 5% of rejections are biased—
that's a hundred thousand people." The justification: if they quit,
someone else would build it anyway. Someone who cares less. Maybe being
inside, they can push for changes.
</p>
<p>
"I don't know if I believe that anymore," one engineer wrote anonymously.
"But it's what I tell myself so I can sleep."
</p>
<p>I recognized the rationalization. It's the same one I use.</p>
<h2>What's Happening in Bangalore</h2>
<p>
Here's something nobody talks about: AI hiring isn't just an American
problem.
</p>
<p>
India's IT outsourcing industry—TCS, Infosys, Wipro—hires hundreds of
thousands of people every year. And they've been early adopters of AI
screening, partly because of the volume (TCS alone gets millions of
applications annually) and partly because Western clients demand it as
part of vendor qualification.
</p>
<p>
Industry research on Indian HR tech reveals a disturbing pattern. Systems
are trained on twenty years of hiring data. "We can predict job performance
with 73% accuracy," one product manager claimed proudly.
</p>
<p>The question nobody asks: have they tested for caste bias?</p>
<p>
The common response: "We don't collect caste data." Right. But they collect
names. And addresses. And which colleges people went to. And in India, all
of those are proxies for caste. The IITs have well-documented caste
disparities in admission. Certain surnames are associated with certain
communities. Neighborhoods in major cities are often segregated by caste.
</p>
<p>
"We've never looked at that. Our clients don't ask about it."
</p>
<p>
This is the thing about AI bias: it exports. American tech companies train
systems on American data, with American biases. Then those systems get
deployed globally, where they interact with local bias structures in ways
nobody's studied.
</p>
<p>
How much talent is being filtered out by algorithms trained on historical
data that reflects the biases of a different continent? Nobody knows.
Nobody's measuring.
</p>
<p>
The pattern is the same everywhere: automate discrimination without ever
naming it. In America, race. In India, caste. Clean hands. Plausible
deniability. And someone's kids don't eat.
</p>
<h2>The Legal Reckoning</h2>
<p>
For years, AI hiring discrimination existed in a legal gray zone. You
suspected something was wrong. You couldn't prove it. The algorithm was a
black box. The company claimed trade secrets. And good luck getting anyone
to explain why you weren't hired.
</p>
<p>
That's changing. The case that might reshape everything is Mobley v.
Workday.
</p>
<p>
Derek Mobley is African-American, over 40, and has a disability. Between
2018 and 2023, he applied to more than 100 positions at companies using
Workday's applicant screening software. Rejected from every one. No
interview. No callback. Nothing.
</p>
<p>
His lawsuit alleges Workday's AI systematically discriminated based on
race, age, and disability. Initially, it looked doomed—in January 2024, a
judge dismissed it, ruling there wasn't enough evidence to classify
Workday as an "employment agency" subject to anti-discrimination law.
</p>
<p>Then came the reversal.</p>
<p>
July 12, 2024: a federal judge allowed the case to proceed. The key
sentence: "Workday's software is not simply implementing in a rote way the
criteria that employers set forth, but is instead participating in the
decision-making process."
</p>
<p>Participating. In the decision-making process.</p>
<p>
That's the sentence that keeps me up at night. That's the sentence that
should keep everyone in this industry up at night. It means AI vendors—not
just employers—can be held liable for discriminatory outcomes. The "we
just built the tool, we didn't make the decisions" defense doesn't work
anymore.
</p>
<p>
June 2025: The court conditionally certified Age Discrimination claims on
behalf of a class that could include millions of applicants over 40 who
were rejected by Workday's system.
</p>
<p>Millions of potential plaintiffs. One vendor. One tool.</p>
<p>
I keep waiting for the industry to panic. For emergency board meetings and
product recalls. For some kind of reckoning.
</p>
<p>
Instead, the sales cycles continue. The contracts get signed. Everyone
figures they'll deal with it when they have to. Our investor called last
week to ask if Mobley would affect our valuation. I said probably not.
Because the truth is, it probably won't—not until someone wins a judgment
that actually hurts.
</p>
<h3>The Cases Keep Coming</h3>
<p>Workday isn't alone.</p>
<p>
August 2025: Someone sued Sirius XM Radio, claiming their AI hiring tool
discriminated based on race. Same pattern as Amazon. Eight years later.
</p>
<p>
May 2024: The ACLU filed an FTC complaint against Aon Consulting—three of
their hiring tools allegedly discriminate against people with disabilities
and certain racial groups.
</p>
<p>
March 2025: EEOC charges against Intuit and HireVue. A deaf Indigenous
woman was rejected because the video software lacked captioning. When she
requested accommodation, denied.
</p>
<p>
HireVue's CEO said the complaint was "entirely without merit." Said Intuit
"didn't use HireVue's AI-based assessment."
</p>
<p>
Maybe there's a legal distinction that matters. But here's what I can't
stop thinking about: that woman still didn't get the job. Whatever system
rejected her, it rejected her.
</p>
<p>
Labor lawyer Guy Brenner put it simply: "There's no defense saying 'AI did
it.' If AI did it, it's the same as the employer did it."
</p>
<h2>The HR Leader Who Couldn't Say No</h2>
<p>
Anonymous accounts from HR leaders at mid-sized tech companies reveal a
consistent pattern. They know their systems have problems. They've seen
the data: Black candidates get through initial screening at about 60% the
rate of white candidates with similar qualifications.
</p>
<p>
What can they do about it? The pressure to reduce time-to-hire is constant.
CEOs want hiring costs down 20%. Boards ask about AI automation in every
meeting. If HR leaders say "we need to slow down the AI rollout because of
bias concerns," someone asks for data. They show it. The response: "let's
monitor it." Nothing changes.
</p>
<p>
The rationalization these leaders articulate: "Even if I pushed back hard,
even if I got them to pause the system... the human recruiters were
probably biased too. At least the AI is consistent. At least I can audit
it. At least there's a paper trail."
</p>
<p>
Does that make it okay? "No. It doesn't make it okay. It makes it...
manageable. Defensible. Something I can explain to a lawyer. Is that the
same thing? Honestly, I don't know anymore."
</p>
<p>
This is how institutional bias works. Not one villain making one bad
decision. A thousand small choices, each reasonable in isolation, adding
up to something terrible.
</p>
<h2>The Regulatory Mess</h2>
<p>
While lawsuits grind through courts, regulators are scrambling—and by
"scrambling" I mean they're about five years behind.
</p>
<p>
New York City's Local Law 144 was supposed to be the model. Effective July
2023. Independent bias audits for automated hiring tools. Annual
publication. Ten business days' notice before AI evaluation.
</p>
<p>The enforcement has been a joke.</p>
<p>
December 2025 audit by the State Comptroller: the enforcement agency's
complaint process is "ineffective." In two years, they received exactly
two complaints. Two. In a city where thousands of companies use AI hiring
tools. They surveyed 32 companies and found one case of non-compliance.
</p>
<p>One.</p>
<p>The message: comply if you want. Nobody's checking.</p>
<p>
Illinois passed an AI Video Interview Act in 2019. An amendment taking
effect January 2026 lets victims sue. Colorado's SB 24-205 was supposed to
be comprehensive—annual impact assessments, risk documentation, consumer
notice. Then it became a political football. Delayed to June 2026.
Extended with a "cure period" through June 2027. The Trump administration
explicitly labeled it "burdensome." A December 2025 executive order
created a DOJ task force to challenge it.
</p>
<p>
Europe is different. The EU AI Act, effective August 1, 2024, classifies
HR tools as "high-risk." Emotion recognition in job interviews became
illegal February 2, 2025. Core obligations kick in August 2, 2026. Fines
up to 35 million euros or 7% of global turnover. Extraterritorial
reach—U.S. companies can be covered if their AI is used on EU candidates.
</p>
<p>
Fewer than 20% of organizations say they're "very prepared." Deadline is
eight months away.
</p>
<p>
European policymakers prioritize worker protection over innovation speed.
America's approach prioritizes... I'm not actually sure what. The freedom
to discriminate efficiently, maybe. The convenience of not having to think
about it.
</p>
<h2>The HireVue Problem</h2>
<p>No company better illustrates this mess than HireVue.</p>
<p>
Founded 2004. Pioneered video interviewing. Later added AI assessment. At
its peak, the company claimed it could predict job performance by
analyzing facial expressions, word choice, speaking patterns.
</p>
<p>
Think about that for a second. Predict job performance from your facial
expressions.
</p>
<p>
The backlash was substantial. 2019: Electronic Privacy Information Center
filed an FTC complaint. Research showed 44% of AI video interview systems
demonstrate gender bias, 26% both gender and race bias.
</p>
<p>
Then HireVue's own internal research leaked. Facial analysis contributed
only 0.25% to job performance prediction. A quarter of one percent.
Candidates were being scored on factors with virtually no correlation to
their ability to do the job.
</p>
<p>
Early 2020, HireVue dropped facial analysis. They now say their
assessments use only transcripts.
</p>
<p>Okay. That's something.</p>
<p>
But here's my question: if facial analysis was never predictive, why was
it deployed for years on millions of candidates? How many people were
rejected because an algorithm didn't like their face?
</p>
<p>
We'll never know. The data exists somewhere in databases. Those people got
generic rejection emails. They have no idea why.
</p>
<h2>The People We Don't Hear From</h2>
<p>
Job seekers who've run these experiments form a loose community—people who
suspected they were being discriminated against and wanted to prove it.
Their stories have a numbing similarity. Three documented patterns stand
out.
</p>
<p>
<strong>Age discrimination.</strong> Laid off in their 50s after acquisitions.
Applied to 200+ positions. Four callbacks. Changed graduation dates to suggest
they were younger. Callbacks jumped to nineteen in two months.
</p>
<p>
"I'm not even upset about the age thing," is the common refrain.
"Companies have been discriminating against older workers forever. What
gets me is the pretense. These systems were supposed to be neutral. Find
the best candidates regardless of whatever. But they just automated what
humans were doing. Made it faster. Scalable. Invisible."
</p>
<p>
The counter-argument—that algorithms are at least auditable—draws scorn.
"Auditable by who? The company that sold it? The company that bought it?
None of you have any incentive to find problems. The only people who have
that incentive are the people being rejected, and we can't see a damn
thing."
</p>
<p>
<strong>Ethnic signaling.</strong> PhDs from top universities. Seven years
experience. After including involvement in ethnic professional organizations
on their resumes, callback rates drop by two-thirds. Same resume, removed
affiliations. Callbacks triple.
</p>
<p>
"The worst part is second-guessing yourself. After the hundredth rejection,
you start thinking maybe you're not as good as you thought. Maybe your
degree doesn't mean what you thought it meant. Maybe you're just... not
what they're looking for."
</p>
<p>
Then they do the experiment. And they realize it was never about them. It
was about their name. Their organizations. The parts of themselves they
were proud of.
</p>
<p>
<strong>Disability barriers.</strong> Hearing impaired candidates needing
captioning for video interviews. Forty-three applications to companies using
AI video screening. Zero accommodations. Zero interviews completed.
</p>
<p>
ADA complaints filed. Most still pending. Some dismissed because companies
argued video interviews were "optional."
</p>
<p>
"I keep thinking about all the jobs I would have been perfect for. Jobs
where my hearing doesn't matter at all. Jobs where I could have proven
myself if anyone would just..."
</p>
<p>
None of these people want real names published. Most still job hunting.
All afraid that speaking publicly will make things worse.
</p>
<p>
That's the hidden cost. Not just the rejections. The silence they enforce.
The people most harmed are least able to talk about it.
</p>
<h2>What I Wish I Could Tell Our Clients</h2>
<p>
I've spent this article being critical—of the industry, of other
companies, of myself. So what should organizations actually do?
</p>
<p>
The tempting thing would be to give you a numbered list. "Six steps to
fair AI hiring." Something you could print out and hand to your legal
team. But I don't think the problem is that people don't know what to do.
The problem is that they don't want to do it, because doing it is
expensive and slow and creates legal exposure and makes your board
nervous.
</p>
<p>
So instead, let me tell you what I wish I could say on sales calls. What
I'd tell our clients if I didn't need their money to make payroll.
</p>
<p>
Understand what you bought. Most HR leaders I talk to can't explain how
their AI tools actually work. They know the marketing. They don't know the
training data, the model architecture, the validation methodology, the
audit results. They bought something they don't understand, and when it
discriminates, they'll claim they didn't know. That defense might work in
court. It shouldn't let you sleep at night.
</p>
<p>
Stop buying audits designed to pass. Here's the dirty secret: you can
structure a bias audit to find what you want to find. Test the right
metrics, the right populations, the right scenarios, and your system looks
clean. Test differently, and you discover it's rejecting Black candidates
at 40% the rate of equally qualified white ones. Most companies—including
most of my clients—choose the test that produces the result they want.
</p>
<p>
Make it possible to sue you. I know that sounds insane. But organizations
that force disputes into arbitration, eliminate class action rights, and
make it impossible for rejected candidates to understand why—they're not
solving bias. They're hiding it. If your AI can't survive legal scrutiny,
maybe that's telling you something important.
</p>
<p>
Train on the workforce you want, not the workforce you had. If you feed
twenty years of biased hiring decisions into an algorithm, you get an
algorithm that perpetuates bias. This isn't mysterious. It's basic ML. The
solution isn't removing demographic information—studies show that often
makes things worse. The solution is actively building datasets that
represent what you want your company to look like.
</p>
<p>
Let humans override. Not rubber stamps. Not recruiters glancing at AI
recommendations. Actual processes where human judgment matters. Random
audits of rejections. A/B testing against human-only decisions.
Authority—real authority—to override the algorithm.
</p>
<p>
Prepare for regulation instead of fighting it. EU AI Act: August 2026.
Illinois: January 2026. Colorado (if it survives): June 2026. Companies
treating this as a compliance problem to minimize are going to get caught.
The ones treating it as an opportunity to fix their systems might come out
ahead.
</p>
<p>
I've never actually said any of this on a sales call. Maybe I should
start. Maybe that's what I'll do Monday instead of signing that contract.
</p>
<h2>The Harder Questions</h2>
<p>
I've offered criticisms, confessions, suggestions. Now I want to sit with
questions I don't have good answers to.
</p>
<p>
If AI hiring tools are systematically biased, and 99% of Fortune 500
companies use them, what's the aggregate effect on the labor market?
Research suggests millions of qualified candidates—disproportionately
Black, older, disabled—filtered out before any human sees their
applications. That's not just unfair. That's massive misallocation of
talent. Economic damage nobody's calculating.
</p>
<p>
What happens to companies doing this? If your AI excludes Black engineers,
you end up with less diverse teams. Less diverse teams produce worse
outcomes—there's extensive research. You've optimized for the wrong thing.
You just don't see the cost because the excluded candidates are invisible.
</p>
<p>
And the psychological damage. How many qualified people, after hundreds of
unexplained rejections, have concluded they're the problem? That their
skills aren't good enough. Their experience lacking. When actually they
were filtered out by an algorithm that prefers white names, younger ages,
certain zip codes.
</p>
<p>
That damage doesn't appear in lawsuits or audits. It's damage to people's
sense of their own worth. Inflicted at scale. By systems that were
supposed to be neutral.
</p>
<h2>Where This Leaves Me</h2>
<p>
I'm not an AI pessimist. I believe these tools can be made better. Bias
reduced, maybe not eliminated. The research on fairness-aware algorithms
points toward real improvements.
</p>
<p>
But I'm realistic about incentives. Companies building these tools want to
ship fast, iterate later. Companies buying them want to cut costs, not
maximize fairness. Regulators are underfunded, politically constrained,
technically outmatched.
</p>
<p>
The people bearing the costs—the job seekers running these experiments—
have the least power to demand change.
</p>
<p>
Which is why the lawsuits matter. Why the EU AI Act matters. Why even
broken laws like NYC's matter. They create consequences that wouldn't
otherwise exist. They shift the burden from victims who must prove harm to
organizations who must prove they're not causing it.
</p>
<p>
What the job seekers keep saying: "All the people who ran the same
experiment I did. Got the same result. And just... gave up. Concluded they
weren't good enough. Never realized the game was rigged."
</p>
<p>
And the uncomfortable truth about what will actually change things: "The
lawsuits. The money. When it costs more to discriminate than to fix the
systems. That's the American way, right? We don't do the right thing
because it's right. We do it when doing wrong gets expensive."
</p>
<p>I wanted to argue. I couldn't.</p>
<p>
What should I do about the client who wants us to drop bias auditing? I
think I should say no. I think I should walk away from the money. And will
I?
</p>
<p>I didn't have an answer.</p>
<p>
Somewhere right now, a resume is being submitted. The candidate is
qualified. The algorithm is running. And inside that algorithm, patterns
learned from decades of discrimination are about to make a decision.
</p>
<p>
We built these systems. We're still building them. I'm still building
them.
</p>
<p>
The question is whether we build tools that judge people by their
qualifications, or tools that launder our prejudices through code.
</p>
<p>Monday is in three days. I still don't know what I'm going to do.</p>
<div class="post-footer">
<p>
<em>
This investigation draws on research from the University of Washington,
Brookings Institution, and Harvard Business Review; legal filings from
Mobley v. Workday and related cases; regulatory analysis of NYC Local
Law 144, the EU AI Act, and state legislation; and documented patterns
from job seeker communities and anonymous accounts from HR and ML
practitioners. The author is a co-founder of an AI-powered recruitment
platform. Published December 29, 2025.
</em>
</p>

<div class="author-bio">
<h3>About the Author</h3>
<p>
<strong>Gene Dai</strong> is a Co-founder of <strong
><a href="https://metix.ai">Metix AI</a></strong
>, an AI-powered recruitment platform. He writes about how technology
reshapes work and the people who do it.
</p>
</div>
</div>

## Continue reading

- [AI and the Great Reshuffling: How Intelligent Machines Are Transforming the Global Workforce](https://digidai.github.io/2025/12/27/ai-workforce-transformation-great-reshuffling-labor-2025/)
- [The Compliance Minefield: How AI Recruiting Became the Most Regulated Technology in HR—And Why Most Companies Are Still Breaking the Law](https://digidai.github.io/2025/12/24/ai-recruiting-privacy-compliance-global-regulation-2025/)
- [When the Recruiter Becomes the Recruited: The Rise of Autonomous AI Agents and the $130 Billion Question Nobody Wants to Answer](https://digidai.github.io/2025/12/23/autonomous-ai-agents-recruitment-future-2025/)
- [The $2.3 Billion Experiment: What Fortune 500 Companies Actually Learned from AI Recruitment](https://digidai.github.io/2025/12/20/fortune-500-ai-recruitment-case-studies-2025/)
- [AI Recruitment Implementation: A Pilot Plan by Company Size](https://digidai.github.io/2025/12/25/ai-recruitment-implementation-guide-by-company-size-2025/)
