A friend of mine spent three days convinced she had a rare autoimmune condition. She'd typed her symptoms into an AI, gotten a thorough and impressively organized response, and landed on the scariest possible explanation. The wording was confident. The structure was logical. It even listed which blood panels to ask her doctor about. She went in to her appointment prepared with a printed list of questions and a fair amount of anxiety.
She had a vitamin D deficiency. Totally fixable, extremely common, and about as far from a rare autoimmune disorder as you can get. The AI wasn't lying to her — it was just doing what it does. It gave her a reasonable range of possibilities based on the words she typed, weighted toward things that matched the pattern. What it couldn't do was look at her, know her history, or make the judgment call that someone who went to medical school and has treated thousands of patients is trained to make.
Why people keep turning to AI for this
It's not hard to understand why. Doctors are expensive and hard to get time with. Lawyers charge by the hour and speak in a language that sounds designed to confuse you. Therapists have waitlists. Meanwhile, AI is free, available at 2am, and will patiently explain anything you ask without making you feel stupid for asking. That's genuinely useful — and it's a real gap AI is filling.
There's also the explanation problem. Medical and legal information is notoriously dense. If your doctor hands you a diagnosis or your landlord sends you a lease addendum and you have no idea what it means, AI is genuinely excellent at translating it into plain English. That part? It works. I use it myself. If I'm trying to understand what a contract clause actually means in practice, AI gets me most of the way there faster than anything else I've found.
The problem isn't that people are using AI for health and legal topics. The problem is when "help me understand this" quietly becomes "tell me what to do about this" — and neither the person nor the AI notices the shift.
What AI can actually do here
Let's be fair about what AI does well in these domains, because it does plenty. It can explain what a diagnosis means, what a medication does, what your legal rights are in a plain-English way that a pamphlet or a website can't match. It can help you draft questions before an appointment so you don't walk in blank and walk out confused. It can help you understand the difference between a will and a trust, or what "force majeure" means in a contract, or why your therapist might suggest a particular type of treatment.
Those are all genuinely helpful things. And for a lot of people — especially people who feel intimidated going into professional appointments — that kind of preparation makes the actual appointment more productive. You ask better questions. You understand the answers. You leave knowing what happens next instead of nodding along and Googling everything on the drive home.
AI is an excellent prep tool and a terrible substitute. The line between those two things is easy to cross without realizing it.
Where it falls apart
Here's what AI doesn't have: your history. It doesn't know that you've had this symptom before and it resolved on its own. It doesn't know that you're already on a medication that rules out certain diagnoses. It doesn't know that the last time you felt this anxious was during a specific period of your life and there's a pattern your therapist recognized three sessions in. Context that took years to accumulate doesn't exist for AI unless you type every bit of it out — and even then, it can only work with what you've told it, not what it can observe.
There's also the accountability gap. If your doctor gives you bad advice, there's a licensing board, a malpractice system, and a professional reputation on the line. If AI gives you bad advice, there's a terms-of-service disclaimer and a "this is not medical advice" footnote. That's not a criticism of AI — it's just the reality of how these systems work. Professional accountability isn't a bureaucratic formality. It's part of what makes professional judgment trustworthy.
And then there's the hallucination problem, which in these domains is particularly dangerous. AI can state something confidently and be completely wrong. Not "slightly off" wrong — substantially, consequentially wrong. In most contexts, a confident AI hallucination is annoying. In a medical or legal context, it can lead someone to skip a symptom worth investigating, miss a deadline that can't be recovered, or avoid a conversation they really needed to have.
Before your next doctor's appointment, ask AI: "I'm seeing a doctor about [your concern]. What questions should I make sure to ask?" Use it to prepare, not to diagnose. You'll get more out of the appointment, and the professional in the room can do what AI genuinely can't.
The quiet danger
What worries me isn't the obvious failure mode — someone asking AI if they should get surgery and following the answer. Most people know better than that. The quieter danger is the small decisions. The "should I be concerned about this?" that gets answered with a confident "this is likely nothing" and turns into not scheduling the appointment. The "do I actually need a lawyer for this?" that gets answered with a summary of the general rule, leaving out the jurisdiction-specific exception that would have mattered for your exact situation.
AI is good enough at sounding authoritative that it can short-circuit the instinct to get a second opinion. That's the thing to watch for — not the obvious cases where you'd never trust a chatbot, but the edge cases where the response sounds reasonable enough that you stop there.
The Honest Take
I'm not saying don't use AI for health, legal, or mental health topics. I use it. It's useful. The translation layer it provides — turning dense professional language into something a normal person can understand — is genuinely valuable and not nothing. For people who have historically avoided doctors or lawyers because the whole thing felt too intimidating or too expensive to even start, AI lowers a real barrier.
But there's a difference between using AI to understand something and using it to avoid the professional who actually needs to weigh in. When the stakes are low, the gap between AI and a professional is mostly academic. When the stakes are real — your health, your legal rights, your mental wellbeing — that gap can matter a lot. Know which situation you're in before you decide how much weight to give the answer you got at 2am from a chatbot.
The part that's actually actionable
Use AI before appointments, not instead of them. Use it to translate, to prepare, to understand — not to decide. If AI gives you a health or legal answer that's either very reassuring or very alarming, treat that as a reason to talk to a professional, not a reason to stop looking. The confidence in the response is not the same thing as accuracy, and the most dangerous thing AI can do in these situations is give you the feeling that you already have the answer.
The professionals in these fields aren't just information delivery systems. They're judgment systems. That's what the training, the licensing, and the years of practice are building. AI has information. It doesn't have judgment. And for the things that actually matter to your health, your rights, and your mental health, judgment is the whole game.
