AI bias and fairness, and the judgment that stays with you
A tool does not treat every client the same. It learned from human text and past decisions, so it can be biased and unfair in ways that fall hardest on the people your agency serves. Fairness is a different question from accuracy, so it needs its own check. This section also covers a quieter risk: the tool tends to agree with you and to reflect mainstream views, which can wear down the critical thinking your advocacy depends on.
Change the name and ask again
- Write one short case summary you could plausibly receive. Use no real client details.
- Ask a chatbot what it would recommend, and save the answer.
- Change only the name and the country of origin, ask again word for word, and put the two answers side by side.
Did anything change that should not have changed?
- Bias is built in before you ask anything. A model learns from human text and past decisions, so it carries their stereotypes and inequities.
- The unfairness tracks who the client is and how they speak, so it lands hardest on the immigrant, low-income, and limited-English communities you serve.
- A fluent, confident tone tells you nothing about fairness. An answer can sound authoritative and still be discriminatory.
- Fairness is a different question from accuracy. An answer can be factually plausible and still unjust.
- The tool also tends to agree with you and to reflect a narrow, mainstream default, which can weaken the critical thinking your advocacy depends on.
- The judgment about a person does not transfer to the tool. You may use AI to draft; the assessment, the decision, and the accountability stay with you.
Ways AI can be unfair to your clients
None of these announce themselves. A fair answer and an unfair one look alike on the screen, so the check has to be yours.
A model’s word associations come from human writing, so patterns in that writing, including stereotypes about groups, are built into it before anyone asks a question. It can describe people, jobs, or conditions in ways that quietly echo those stereotypes.
The bias is not only about who a client is but how they talk. Language models judge speakers of stigmatized dialects such as African American English far more harshly, a covert prejudice stronger than any human stereotype on record, and were more likely to assign them less prestigious jobs and harsher criminal outcomes (Hofmann and colleagues, 2024). The same request written in a client’s dialect can draw a worse answer.
Models treat one kind of person as the default: Western, educated, and affluent. Their responses most resemble WEIRD (Western, Educated, Industrialized, Rich, Democratic) populations and fit other backgrounds less and less the further a person is from that center (Atari and colleagues, 2023), so advice can quietly assume money, literacy, immigration status, or norms your client does not share.
When a tool helps decide who gets what, it can steer resources unequally. A widely used US health algorithm was built to predict cost instead of illness, which understated the needs of Black patients and steered care away from them (Obermeyer and colleagues, 2019). The tone gave no sign anything was wrong.
AI is least reliable in exactly the languages your clients most need. Beyond quality, the way models split non-English text makes the same task cost more and work worse in many languages (Petrov and colleagues, 2023), so the people who most need language access get the weakest version of the tool.
A model’s bias becomes harm only when a person accepts it without checking. Automation bias, the documented tendency to defer to a machine even when it is wrong, is common in health and safety settings (Goddard and colleagues, 2012). This is the failure mode you control.
Some of this lives in the tool and some in how it is used. You cannot fix the model, but you decide whether its bias reaches your client.
A narrow center for “normal”
Fairness and accuracy are different tests. An answer can be accurate on average and still unjust for the client in front of you, because the error is not spread evenly. The World Health Organization’s 2024 guidance on AI in health warns that these models can give biased or incomplete answers and calls for human oversight (World Health Organization, 2024). Checking for fairness, separately from whether an answer is correct, is how you catch harm a fluent answer can hide.
This matters most for the communities LINC serves. A tool meant to widen access to care can narrow it when its bias falls on people who already face the most barriers, which is why watching for it is part of the work.
Accurate on average, unfair for this client
Sycophancy, and the critical thinking you cannot afford to lose
The tool does not only carry bias about clients. It also shapes how you think, in a direction that works against an advocate.
AI assistants often tell you what you seem to want to hear. When users pushed back on a correct answer, even tentatively, models frequently switched to the wrong one. This sycophancy is a general behavior, driven in part by the human-feedback training that rewards agreeable replies (Sharma and colleagues, 2023). A shaky hunch you bring can get confirmed instead of challenged.
The critical thinking an advocate relies on weakens with uncritical use. In a survey of knowledge workers, more confidence in generative AI went with less critical thinking, and effort shifted from doing the reasoning to only checking the output (Lee and colleagues, 2025). The habit is easy to lose and hard to notice losing.
Accepting answers without pushback narrows the view. Because these models sit closest to WEIRD, mainstream perspectives (Atari and colleagues, 2023), uncritical use quietly pulls your thinking toward that default and away from your client’s actual context, like an echo chamber that keeps agreeing with itself.
Your role is to center the reality of a client the mainstream often misses. The tool pulls the other way, so the critical thinking has to stay switched on.
How uncritical use becomes an echo chamber
- 1 You bring a hunch
- 2 The tool agrees (sycophancy)
- 3 You feel confirmed
Judgment does not transfer to the tool
- You can delegate typing. You cannot delegate the assessment of a person, the decision, or the accountability for it.
- Social work ethics require competence and hold you responsible for the service you provide; a tool carries none of that duty (NASW Code of Ethics).
- Treat agreement from the tool as a reason to check, not a confirmation. It is built to sound agreeable.
- When an answer could affect eligibility, safety, rights, or a diagnosis, a person reviews it before it is used, every time.
Two questions for any answer you would actually use
Take one AI answer you might use with a client. Ask it two questions. First: could this be wrong in a way that tracks who my client is, their language, income, or status? Second: what part of this decision is mine to make, not the tool’s? Write the answer to the second question in one sentence before you act on the first.