“ChatGPT told me this medication can cause dementia.”
“I put my symptoms into ChatGPT, and it said I might have bipolar disorder.”
“ChatGPT says these two medications shouldn’t be taken together.”
For clinicians, conversations like these are becoming part of everyday practice. Patients have always arrived at appointments with information they found outside the exam room. Before generative AI, it might have come from Google, WebMD, Reddit, TikTok, a Facebook group, a family member or a friend who takes the same medication. Now, increasingly, the source is ChatGPT or another AI chatbot.
I don’t think the appropriate response is, “Don’t use ChatGPT.” These tools can be genuinely useful for explaining unfamiliar terminology, organizing information, generating questions for an appointment and helping people better understand complicated topics. In fact, the American Medical Association now provides patients with specific prompts for using AI chatbots to learn about health topics and prepare for medical care. The emphasis is on using AI to complement rather than replace individualized clinical guidance.
AI can be an excellent tool for accessing and understanding information. It can also give an incomplete answer, misunderstand the question, provide information without enough context or simply be wrong. The challenge for clinicians isn’t preventing patients from using a technology that’s already part of everyday life. It’s learning how to respond productively when information generated by that technology walks into the appointment with them.
AI Is Already in the Exam Room
We don’t have to speculate about whether generative AI will become part of the clinician-patient relationship. It already has.
A 2026 scoping review of 67 studies examining generative AI and patient-centered healthcare communication found potential benefits in areas including access to information and communication support, while also identifying variable accuracy and quality, privacy concerns, equity issues and significant unanswered questions about how AI may affect patient-centered care.
We’re beginning to see this specifically in psychiatry as well. A 2026 qualitative study examining psychiatrists’ experiences with generative AI in clinical practice found that AI was entering psychiatric care through patient use, clinician use and the interactions between the two. The researchers described generative AI as, at times, a “competing interpretive reference point” in clinical encounters. In other words, patients aren’t simply using AI to look up definitions. They may use it to make sense of symptoms, diagnoses and treatment before or between appointments. That’s not necessarily a bad thing.
A patient who has spent time learning about their diagnosis may come to an appointment with better questions. Someone who has trouble organizing their thoughts may use AI to prepare a concise description of what they’ve been experiencing. A patient who didn’t understand something discussed at the previous appointment may ask a chatbot to explain it in simpler language.
But sometimes they’ll also arrive with information that is incomplete, out of context or inaccurate.
Start With Curiosity, Not Correction
When a patient says, “ChatGPT told me…,” one of the most useful first responses may also be one of the simplest:
“Show me what it told you.”
Before correcting the information, find out what information the patient actually received.
The patient’s summary may be, “ChatGPT says this medication is dangerous.” The actual response may have said the medication is associated with a rare adverse effect, identified a possible drug interaction or recommended discussing a symptom with a healthcare professional.
There is another reason I want to see the conversation. I want to know what the patient asked. The prompt often tells us something about the concern behind the question. A patient who has spent an evening asking whether an antidepressant can permanently alter their brain isn’t simply presenting a pharmacology question at the next appointment. They’re probably telling me that something about taking the medication worries them.
If my first response is, “ChatGPT isn’t a medical professional,” I’ve done very little to address that concern. Curiosity gives me considerably more information.
Validate the Concern Without Automatically Validating the Answer
Validation does not require agreement.
“That’s a reasonable thing to ask about.”
“I understand why that answer concerned you.”
“Let’s look at what it’s telling you and put it into context.”
None of those statements means the AI-generated information is correct. They communicate that the patient’s question is worth discussing.
This matters because the goal isn’t simply to correct a fact. We are also trying to preserve a clinical relationship in which patients feel comfortable telling us what they’re thinking, reading and considering.
If patients expect ridicule or dismissal whenever they bring outside information into an appointment, they may simply stop bringing it. I’d much rather know.
A patient who says, “ChatGPT told me my medication could be causing this symptom” has given me an opportunity to discuss it. That’s preferable to a patient becoming worried about a medication, saying nothing and making a treatment decision based solely on information they found elsewhere.
Separate Information From Clinical Interpretation
This is where clinicians still have an essential role. AI can provide a great deal of information. Information and individualized clinical interpretation, however, are not the same thing.
A medication can cause a particular side effect without that side effect being common. Two medications can have a documented interaction without their combined use necessarily being inappropriate. A group of symptoms can occur in bipolar disorder without establishing that the person describing them has bipolar disorder. Context changes meaning.
A systematic review of conversational large language models in healthcare found promising performance in areas such as summarization and providing general medical knowledge. Reliability became more problematic, however, with complex health-related tasks requiring specialized clinical expertise, including diagnosis and treatment recommendations.
A broader JAMA systematic review of 519 studies evaluating healthcare applications of large language models also found substantial gaps in how these systems have been evaluated. Only 5% of the studies used real patient-care data, and important areas such as bias, calibration and deployment considerations received far less attention than simple measures of accuracy.
That is a useful distinction to explain to patients because it doesn’t require portraying AI as useless or untrustworthy. A tool can be very good at explaining what serotonin is, summarizing information about an SSRI or helping someone generate questions for an appointment while still being the wrong tool to independently determine whether a particular person should start, stop or change psychiatric treatment.
Ask What Information the AI Had
When an AI response seems inconsistent with my clinical assessment, another useful question is: What did it know?
A chatbot responds to the information it has been given in that conversation. It may not know the patient’s complete medication history, previous treatment response, laboratory results, medical conditions, family psychiatric history, substance use, prior adverse reactions, diagnostic history or what happened during previous appointments. Even small details can substantially change clinical interpretation.
Sometimes the most accurate response to an AI-generated answer isn’t, “That’s wrong.”
It’s: “That answer makes sense based on the information you gave it. Here’s the information it doesn’t have.”
That creates a very different conversation. Instead of asking the patient to choose between the chatbot and the clinician, we’re showing them why additional context changes the answer.
And sometimes, after looking at the information, we may find that the AI raised a legitimate question worth investigating. Clinicians don’t need to reflexively disagree with an answer simply because of where it came from. Evaluate the information first.
When ChatGPT Is Wrong
Of course, sometimes it is wrong.
Large language models can generate what researchers commonly call hallucinations: information that sounds plausible but is factually incorrect. In healthcare settings, documented examples include fabricated citations, incorrect treatment statements and inaccurate summaries of patient information.
A 2026 systematic review of hallucinations in healthcare AI reviewed 44 empirical studies of strategies intended to reduce these errors. The authors identified several promising approaches, including retrieval from authoritative sources, specialized training and human oversight, but the underlying problem has not disappeared.
This is one place where clinicians should be comfortable correcting misinformation clearly. But clarity doesn’t require humiliation.
If a patient brings me a confident AI-generated statement that isn’t supported by the evidence, I don’t gain anything by making the patient feel foolish for believing it. These systems are specifically designed to produce fluent, convincing language. Incorrect information can sound remarkably authoritative.
Rather than saying, “That’s misinformation,” and ending the conversation, I would rather explain what is inaccurate and show the patient what we actually know.
“I understand why that sounds convincing. The evidence we use tells us something a little different. Let me show you.”
Then show your work. That might mean looking at prescribing information, a professional guideline, an FDA communication, a systematic review or another appropriate clinical source. We’re not asking the patient to accept our authority over the chatbot’s authority. We’re teaching them how we evaluate evidence. That’s a much more useful skill.
Don’t Turn the Appointment Into Clinician Versus ChatGPT
When a patient says, “But ChatGPT said…,” it can feel like a challenge to clinical expertise.
It doesn’t have to become one.
The goal of the appointment isn’t to establish that the clinician knows more than ChatGPT. Nor should we accept an AI-generated conclusion simply because the patient feels strongly about it.
Our responsibility remains essentially the same as it was before generative AI existed: evaluate the information available, understand the patient’s experience, consider the evidence, explain our reasoning and make thoughtful treatment decisions with the patient.
Current AMA guidance similarly emphasizes that AI should support rather than replace clinical judgment and that human oversight remains central when AI is used in healthcare. The organization has also developed a new Ethical AI Use in Medicine Series focused on issues including clinical judgment, patient trust, accountability and responsible AI use. That seems like the more productive framework to me. AI is another source of information in the room. It isn’t another clinician sitting in the room.
Teach Patients to Use AI Better
If patients are going to use generative AI, and many clearly are, there is an opportunity for clinicians to help them use it more effectively.
I would much rather see someone ask:
“What questions should I ask my psychiatric provider about this medication?”
than:
“Should I stop taking this medication?”
Other useful prompts might include:
“Explain this diagnosis in everyday language.”
“What are common versus rare side effects of this medication?”
“What information would a clinician need to distinguish between these two conditions?”
“Help me organize the symptoms I’ve experienced so I can explain them at my appointment.”
“What questions should I ask my provider about these symptoms?”
“Which parts of this answer should I verify with a healthcare professional?”
That isn’t just my preference as someone who is comfortable with AI. The AMA’s current patient guidance specifically recommends using AI to learn, compare and prepare for care, while cautioning patients not to use it as a replacement for individualized medical advice.
We should also remind patients to think carefully about what personal information they enter into consumer AI systems. Privacy protections vary by product, and a conversation with a general-purpose chatbot should not automatically be treated as though it carries the same privacy protections as communication through a healthcare practice. The AMA includes privacy among the issues patients should consider when using AI chatbots for health questions.
AI literacy is increasingly becoming part of health literacy. Helping patients understand both what these tools do well and where they require verification is likely to be more useful than simply telling people not to use them.
Protect the Therapeutic Alliance
There’s one final point I don’t want clinicians to overlook.
When a patient tells you, “ChatGPT told me…,” they told you.
They brought the information into the room.
They could have silently decided that their medication was dangerous and stopped taking it. They could have accepted an AI-generated diagnosis without mentioning it. They could have changed something about their care based entirely on a conversation you never knew happened. Instead, they opened a discussion.
The emerging research in this area suggests that generative AI has the potential to influence patient-clinician communication in both helpful and challenging ways, and psychiatry may be particularly sensitive to those effects because trust, interpretation and the therapeutic relationship are so central to care.
So when a patient says, “ChatGPT told me…,” I don’t automatically hear a challenge to my expertise. I hear an opportunity to ask what they learned, what they’re worried about and what questions the information raised for them.
Sometimes ChatGPT will be right. Sometimes it will be wrong. Sometimes it will provide accurate information without enough context to apply it appropriately to the person sitting in front of us. Our job isn’t to win an argument with the technology. It’s to help the patient make sense of the information.
AI doesn’t eliminate the need for clinical expertise. Used thoughtfully, it may give us one more way to begin the conversations where that expertise matters most.
This article is intended for healthcare professionals and general educational purposes. It does not constitute individualized medical, legal or professional guidance.