What AI can and cannot do
Set realistic expectations for the CHW and, through them, the client: what AI is genuinely useful for in navigation, where it fails, and how to notice a confident wrong answer.
Learning objectives
By the end of this module, the CHW can:
- Name navigation tasks AI does well and ones it does poorly.
- Explain, in plain words, why a chatbot can sound certain and still be wrong.
- Show a client how to notice signs that an AI answer may be off.
- State the escalation mindset: when in doubt, check with a person before acting.
Facilitator outline
You are not the AI expert in the room. You are the person who models checking carefully. Set that tone out loud.
Drafting a phone script, translating the gist of a letter, explaining a term, listing questions to ask a provider, organizing next steps. These are logistics and self-advocacy tasks, never diagnosis or treatment.
It can invent facts, sources, phone numbers, and clinic details and state them confidently. It does not know local, current, or personal specifics. It can be worse in the client's language. Introduce the word for this pattern in plain terms: the tool guesses fluently.
Work the weakened-dose example below together, reveal the planted error, and debrief the check that caught it.
Set up the reflex the rest of the training builds on: for anything clinical, legal, or about money or safety, verify with a person before acting. Preview the escalation card.
Key teaching points
Frame AI as a fast, confident assistant that is often right about general things and sometimes wrong about specific ones. The failure that matters most for clients is a fluent, sure-sounding answer that is simply false: a made-up clinic phone number, an eligibility rule that does not exist, a dose that was never prescribed. The client cannot tell a real answer from an invented one by how it sounds, because it sounds the same. So the skill is not "detect the fake by looking harder." The skill is a habit: for anything that matters, leave the answer where it is and check who says so somewhere else, or check with a person.
Keep the tone calm. The goal is not to scare clients away from a tool they already use every day. It is to help them use it for the right jobs and step back for the wrong ones.
Short "spot the error, then see the answer" practice reliably improves people's ability to tell good information from bad; a 2024 experiment (Leder and colleagues) found that adding a feedback step to the practice is what makes it work. This is why Module 1 uses a worked example with the error revealed, not a lecture.
People largely cannot tell AI-written content from human-written content by surface cues, and AI-written material can be more convincing than the real thing (Spitale and colleagues, 2023). That is the reason we teach a checking habit instead of cue-hunting.
About 1 in 3 US adults already use AI chatbots for health information, and Hispanic adults are more likely than White adults to turn to AI for mental-health advice (KFF, 2024-25). Clients are already reading these answers on their own, so the CHW steps in as a calm second reader who helps them check what they found.
Immigrants may be especially open to these tools. In a study of immigrant and general-population adults, immigrants reported higher willingness to use AI mental health chatbots, and believing that AI improves health care predicted acceptance (Yoo and Jang, 2026). That openness is a reason to guide safe use early, not a reason to worry.
Spot the error (worked example)
Read this AI answer aloud, exactly as a client might receive it:
Ask the group: what here would you check before acting on it? Then reveal the planted problems. The phone number is invented and should never be dialed on trust. Michigan Medicaid does not require a primary-care referral for most behavioral health, and community mental health serves people of all ages, including children. The confident tone is identical to a correct answer.
Debrief the check that works: do not act on a specific number, rule, or eligibility claim from AI. Confirm it against an official source or bring it to your supervisor. Have each CHW say one sentence they would use with a client to model this out loud.
Materials
- AI gets things wrong: why, when, and how to checkGuide
- Spotting health misinformation made by AIGuide
- Using AI safely for healthGuide
- How LINC uses AITransparency
- Can I give a plain-language example of a confident wrong answer without using technical words?
- Did every CHW practice one sentence for coaching a client to check an answer?
- Did I keep the tone matter-of-fact, so clients are not scared off a tool they already use?