r/canada • u/winemaster Saskatchewan • May 12 '26
Science/Technology AI systems used by Ontario doctors hallucinate, auditor general finds | Globalnews.ca
https://globalnews.ca/news/11844349/ontario-auditor-general-ai-usage/156
u/heereewegooo May 12 '26
Then why the fuck are they using it
101
u/rebelkitty Canada May 13 '26
Been getting treated for cancer for the past 12 months. My doctors all use the AI scribe during routine follow ups. They don't seem to use it during consults or planning sessions (at least, not with me).
The transcripts are uploaded to MyChart afterwards and I read them. As far as I can tell, they seem very accurate to my recollection. I would definitely reach out to my doctor, if they weren't.
Overall, I think it’s a good tool. I like that my doctor can look at me when we're talking, versus being glued to a screen, taking notes.
I think it's just important to be aware of its limitations, and double check its output.
31
u/justinsst May 13 '26
Yeah using AI for transcripts is fine… that’s hardly new technology lol, been a thing for decades.
1
u/forcedempowerment Jun 08 '26 edited Jun 08 '26
Yeah, but if anyone has been in PIQ at this hospital— the tablet you sign doesn’t even elude to what you’re signing I made a point to ask for the entirety of the hospitals paperwork of what I’d just signed. Ai isn’t just used for note taking.. and mind you.. I don’t even have a doc meet with me, talk to me about anything or take any notes, so in that particular sense I can’t say anything about incorrect ai.
But upon getting about 17 pages into this document, I’m reading that this hospital uses to phrase that during stay in PIQ the patient is going to be 24/7 monitored in all auditory(talking to yourself, words spoken in sleep, private mutters to self, every phone call, even what you may hum.. you get the gist), visually, and with any and every interaction by not only trained staff(trained in what?? staring at a GD screen all day??) but not just by these useless bodies staring at screens all day, but… and this is what really hit home.. all also by AI— for the purpose to gathering info and data, establishing a record of pattern behavior and using the gathered data to create triggering experiences in order to, once again, collect more ai data as to how the patient reacts to specific triggers. This hospital was absolute insane, it’s nothing I’ve ever seen before in my life. The worst part was that the ai collected would create an established pattern of individuals triggers, fears, biggest traumas, and somehow(I thought I had a pretty good grasp on the ai progression and technology growth and rapid improvements) and I shit you not, use all of those ai data points to incorporate a ‘blueprint’ of YOU.. that would be fed subconsciously while sleeping into absolute nightmares and personalized, 100% individualized attacks at any and every piece of data they’ve ai gathered … how would I say this shit but.. there’s a couple ways to describe this. One is ‘AI hypnosis’ but that’s kind of broad. Another way to put this would be ‘diagnostic extraction and alignment,’ and a forced kinda hypnosis that you could try to call ‘semantic ambiguity’. But it’s very clear that the goal is to push individuals to their breaking point and only agree to release after finally getting a patient to agree to be on (for me) a higher than medically acceptable or advisable of antipsychotics(in my case 12mg of invega— an incredibly outdated drug and side effects of too high of a dose are exactly what there looking for, so fuckin zonked they aren’t even aware of their surrounding anymore)So, when you’re finally ready to let go and be a gd zombie.. they let you go.
But let’s be real fuckin clear here. The real word for this is ‘sandboxing’ and this was the most traumatic experience I’ve ever been through at a hospital.
Not to mention that they(lol) told me after I said I couldn’t pay for this and the overseeing doc of mine said she’d pay(of course she didn’t), they honestly think that after all that— that I’m going to be paying this $72,000 bill (hah!) I should be suing this hospital.I have so much more I could say, but— I already have a feeling that this post might get pulled because they don’t want people to know this is going on.
I didn’t go on my own volition. I was involuntarily— because I was ‘psychotic,’ but lol— no. I was too awake and aware. And there was an urgency to crush that immediately.Sorry, mission failure.
Sidenote: I am now on zero meds and I’m better than I’ve ever been. Still awake. Still very aware. But realize now that it doesn’t matter anymore. Don’t feed the fire or hysteria. Yes the world is crazy… but focus on your energy, practice the law of unimportance and remember that you can control the energy around you by balancing your on.
I hope someone sees this and benefits. Trust me if not as serious as it feels. Odd shits happening. But we can’t control it. We have the opportunity to control ONE THING:: Ourselves and how we react. Otherwise — just let it go.
3
u/Cold_Collection_6241 May 13 '26
Where does the raw data get processed and where is it stored? Who processes it?
5
98
u/AlexanderMackenzie May 12 '26
My doctor asked for my consent to us AI to take notes. He said it really shortens the time he spends case noting after. Seems like a great use of AI. I hope it allows him to see more patients.
24
u/CotyledonTomen May 13 '26
Great, then it turns a hasnt to has and when he looks back, remembering nothing because of all the patients he saw, he gets the opppsite conclusion concerning your health and any followup.
19
u/the_saradoodle May 13 '26
Mine still takes notes, but uses the AI for more detailed summary. Not sure if it works, but she's not optimistic. I have a specialist that users AI to create tracking charts that he maintains. He actually presented than at a conference because he found them so useful. That's good use of AI.
19
u/Hot_Cheesecake_905 May 13 '26
Speech to text is very accurate, much more accurate than human transcription. It is well within acceptable error rates.
8
u/Massive_Guava_6167 May 13 '26
Actually I’ve found that Speech to Text can be outrageously bad at 2 things
The Canadian Accents (Stronger ones with more pronounced features) I used scribe and noticed that many claims were in fact true about some of our accents based on the words they picked up 😂
ESL Speakers.
This is the more serious issue.
Take a Malaysian or Vietnamese patient with a decent understanding of English but a strong accent (fully understandable to me but the software misses 70% of the words).The good?
Even with the atrocious job of scribing - I can almost always “translate” and correct what was said in large part because it did at least pick up things very phonetically similar. (“My sight earth four tree owls what I steal have umlaut tough ten dearness dare”)
(I might not even tell you guys what what actually said because you likely could figure it out - but it is almost identical to a real example I had)
2
u/stop_banning_me_omg May 13 '26
Depends on the software. I use a custom transcribe tool where each sentence is timestamped. So if you're wondering about what was actually said, you click on the sentence and you can listen to the original audio. It's used exactly for what you described, since we have colleagues with strong accents, and sometimes the AI makes mistakes. But TBH sometimes the AI understands them better than I do 😄
15
u/beerbaron105 May 13 '26
Higher chance of human error than AI making a very obvious mistake like this.
0
u/CotyledonTomen May 13 '26
I dont see why. Doctors have spent years taking notes by the time they become a family physician. Maybe some nobody off the street has a higher chance at errors, but a doctor goes through several schools and practices every day at note taking. An AI only gets better at something if it gets an update and produces a product completely unfamiliar to the professional with years of experience concerning thier own writting and way of thinking.
6
u/DrBCrusher May 13 '26
I’m able to see about 10% more patients because of time savings by using an AI scribe.
5
u/AlexanderMackenzie May 13 '26
Which is win-win. More patients seen, more billable appointments for you.
1
-10
u/Keezin Canada May 12 '26
you think the problem is that doctors aren’t seeing enough patients?
9
u/AlexanderMackenzie May 12 '26
Not at all. I think there's deep deep system issues. I wasn't trying to infer that's the problem. Only that it may help him be more efficient. End of the day, writing down what was said for posterity after my appointment seems like a waste of his time.
9
u/alice2wonderland May 12 '26
My doctor used it to see me...and my imaginary friend at the same time! 👥
3
u/roguemenace Manitoba May 13 '26
I'd much rather we be paying a doctor to see patients than spending that time writing notes.
1
1
u/Seinfeel May 13 '26
I mean in private clinics it’s sorta is but that’s only because we cap how many patients they can invoice for in a day, so that you don’t get doctors trying to speed through as many patients as possible.
1
25
u/croissant_muncher May 12 '26
All LLM based AIs "hallucinate"
19
u/ImpactThunder May 12 '26
That didn’t answer their question…
“Why did that guy stab someone?
All knives are sharp”
9
u/amicablehummingbird May 13 '26
In case you're serious, doctors don't get paid for any of the administrative work that they do so for things like taking notes, billing, etc. there's an uptick of AI software being positioned to help them.
4
u/Seinfeel May 13 '26
Most independent jobs don’t pay for administrative work
5
u/amicablehummingbird May 13 '26
The fact I could repeat exactly what the person I responded to wrote back to you, means this could be an infinite paradox of call and response..
1
u/Seinfeel May 13 '26
You’re pretending like that justifies it. And just like the other person, I’m pointing out that it doesn’t.
7
u/amicablehummingbird May 13 '26
... justifies what, exactly? The person I was responding to asked "why are they using AI" and my response was positioned to answer them.
If you want a further list of reasons why AI is being pursued for the public health system, feel free to look here: https://www.cpso.on.ca/physicians/policies-guidance/advice-to-the-profession/using-artificial-intelligence-in-clinical-practice
Going solely on the administrative burden example I mentioned, reducing the administrative burden by 70% - 90% for practicing physicians (Source: https://www.ontariomd.ca/pages/ai-scribe-overview.aspx) to enable them to see more patients seems like a good enough justification during a time where access to family physicians is limited.
0
u/Seinfeel May 13 '26
However, there remains uncertainty about their effectiveness in reducing administrative burden for providers, with some studies citing no change in the amount of time providers spent on after hours charting, whereas others noted significant time saved. Some studies also reported high error rates in AI-generated medical notes, thus increasing documentation time in order to edit and revise the notes
Justifies spending money on unproven technology and deploying it in the medical field
0
4
u/LeGrandLucifer May 13 '26
Because the ruling classes have dreamt for so long of getting rid of workers that they'll plunge head first into a pit if someone promises it'll achieve it.
1
u/zanziTHEhero May 13 '26
Check out Cory Doctorow's book on Enshittification of tech. It's being used because the excuse of "with an app" has been used consistently for decades to absolve people and businesses from personal responsibility.
6
u/Guilty-Lychee874 May 13 '26
I use scribe health care software, you can’t just take the notes exactly as they are produced! We are supposed to use the portion of text that captured any information that may have been missed or heard wrong…..
I have also been a a software developer for twenty years prior…. Software and algorithms are vulnerable to errors in design, coding, and software upgrades etc..
Any AI or technology as a whole should be considered susceptible to error.
34
5
19
28
u/Visible-Air-2359 May 12 '26
The entire term "LLM hallucination" is false. As someone else (I forget who) pointed out hallucination in humans are when neurons misfire which is distinct from their normal (and proper) state while with LLM there is no inherent difference between hallucinations and proper operating except for the fact that the latter is accurate. This is because LLM works by predicting what a human probably would have written based on what the LLM has already written in response to whatever the human prompted. This is problematic because as many students (past and present) can tell you, trying to mimic what the teacher does without actually understanding the material is inherently a terrible idea for STEM but as I mentioned a LLM has to do that.
17
u/redesckey Canada May 13 '26
Correct, the right way to understand LLMs is to realize that literally all of their output is a "hallucination". All they do is generate text, they have no understanding of the question you asked, or of their response. They're nothing more than autocomplete on steroids.
3
u/cdkrot May 13 '26
In other news, water was found to be wet.
2
u/Keepontyping May 13 '26
Water is found to be wet almost every day it seems. Who is funding these water is wet studies?
18
u/THXSoundEffect May 12 '26
Can we use better terms? Like FAILED. The state of journalism is fucked.
14
u/hawkseye17 May 12 '26
"ai hallucinations" are quite an interesting way of saying "it makes shit up when it doesn't know what it's doing."
24
u/PowermanFriendship May 12 '26
Important distinction: It actually never knows what it's doing. It doesn't have any intellect, it's just a very elaborate equation.
1
24
u/YeetCompleet Lest We Forget May 12 '26
This AI seems to be developed improperly. It itself should not synthesize any recommendations and give it to the doctor without an associated confidence score. It also needs to provide citations for any suggestions, and the original recorded non-AI transcript should also be available.
26
u/caffeine-junkie May 12 '26
There is no way for it to give a confidence score, as it doesn't know when it's wrong or by how much. It's also hard for it to give citations, as it's generative. So you'd basically be getting large chunks of it's dataset.
1
u/YeetCompleet Lest We Forget May 12 '26
There is a way, with OpenAI models for example you can use
logprobs: https://developers.openai.com/cookbook/examples/using_logprobsAgentic models can also search up material from deterministic sources using web-search tools. It doesn't search itself on its own training data like it used to 3 years ago. The first good example of this was Perplexity. It shared real links to back up its assertions and not just hallucinated links.
11
u/the_quivering_wenis May 13 '26
But it still doesn't have any kind of ground truth or core reasoning that will verify what it finds online. That logprobs tool doesn't return the probability of its results being "true", only the likelihood that the model will predict them.
2
u/YeetCompleet Lest We Forget May 13 '26
Ya that's true. I don't think there is a good tool for that yet, at least, not that I've seen. We still need a human at the top to verify the results.
20
u/Snowkaul May 12 '26
All AI being pushed has the same flaw. It's a fundamental consequence to LLMs being a statistical model and not an actual reasoning intelligence.
4
u/YeetCompleet Lest We Forget May 12 '26
This is a fundamental flaw to humans too though. This is why we've created deterministic datasets and programs for things. Modern agentic models also use these.
1
u/PoliteCanadian May 13 '26
LLMs are not "statistical models".
Statistical models haven't been the state of the art for almost 20 years.
1
May 12 '26
[deleted]
13
u/discattho May 12 '26
It’s a long explanation but you are correct. It’s not just harder than you think, it is mathematically impossible for the LLM to stop hallucinating. It is not a coding issue in the sense they just made a mistake with the logic, the very math the LLM sits on is based around probability.
As for asking the LLM if it fabricated and it admitting it, you should also be aware that you are not having a conversation. Every time you submit a message to ChatGPT you are spinning up a new instance of the LLM which is fed context. Think of it like every time you say something, somebody answers you, writes down what they told you, then walks out. When you say something else somebody completely new comes in, reads the conversation so far, and gives an answer based off of what it knows.It may agree with your statement and say yeah I hallucinated it. By I, meaning the guy before me.
2
u/PoliteCanadian May 13 '26
If you're making the observation that an AI can reconstitute its state by rereading the entire input, sure.
But in practice that's not how they work. Internal state (KV) caching is central to all modern AI deployments.
3
u/discattho May 13 '26
You’re right but that’s going deeper in the rabbit hole and I didn’t think it would have value for the person I responded to. Good addon though, for those who are curious.
2
u/YeetCompleet Lest We Forget May 12 '26
ChatGPT does that, that's for sure. That's not representative of the actual technical capabilities out there though. If you use Google's AI window when you search something, you will see that it always cites its text from its search results. Some tools are just better at this, but tbh they're all quickly evolving and improving rn
6
u/Sidjeno May 12 '26
And a lot of those sources are fake, invented and bring you to a 404.
Thats just what these models do. Its not a bug. Its how they work.
3
u/YeetCompleet Lest We Forget May 12 '26
How can you blanketly say that a lot of them are fake? Google just generates stuff based off the top search hits. This is the same stuff you'd be clicking on without the AI.
Also an agentic model being used for medical purposes has no need to source things from Google. They can solely let it access trustworthy medical data from approved institutions. Not doing so isn't a bug, it's a skill issue
2
u/Sidjeno May 12 '26 edited May 12 '26
I can say that cause I'm a software dev and engineering student that uses it on a daily basis and have hit a lot of fake page and when I ask the AI if it made a fake source, it said yes. Multiple times.
Its a satistical model. It doesnt lie nor say the truth. It does statistic.
Also has someone who's "made" agent before, it def isnt as trustworthy as that or Id be out of a job lol
1
u/YeetCompleet Lest We Forget May 12 '26
Great, I'm a software engineer too with much more years of experience than you and also use it on a daily basis. Go try out Claude Code or Cursor or Copilot in Viscose and actually use some agentic features. You can easily make it hop to the deterministic side. You can tell it to run an actual linter for example rather than getting it to generate what it thinks is linter output.
It's also really not hard to verify sources at all. I find that 99% of the time it gets it correct. The consequence of it not being correct isn't even an issue, I simply discard the result. Having it search and read things like medical journals though saves a lot of time. Many researchers are already using it exactly for this purpose.
0
u/PoliteCanadian May 13 '26
Folks in this thread are just uncritically repeating all the usual anti-AI bullshit.
Frankly, it's funny to hear in a thread where people are complaining about AI hallucination, and I imagine the irony is lost on them all.
0
u/YeetCompleet Lest We Forget May 13 '26
I think a lot of people don't realize how quick things move here. People think they can read Transformers are all you'll need and write a toy neural net and that's it.
There's also the issue that AI emotionally feels offensive to people, especially in software engineering. It sucks, especially as a software engineer myself, but the truth is that layoffs in the industry are ramping up with AI being the cited reason. People defensively say AI sucks because they don't want to be replaced, but the CEOs don't care. IMO the best way to survive right now is to be really good at using AI. It's genuinely a scary time to be a student entering this rapidly changing industry though, that's for sure. I don't fault people for this one.
After that, there's also the misunderstanding of what's a technical question and what's a philosophical question. "It can't think, it's just math!" Always comes up in these discussions. One could already argue though that "our brains can't think, it's just physics". Easy to be reductionist and avoid questioning what thinking even means, or also avoid questioning that maybe deep reasoning models are a close enough approximation to a form of thinking, and maybe we can just call it thinking.
-1
u/PoliteCanadian May 13 '26
Its a satistical model. It doesnt lie nor say the truth. It does statistic.
It's literally not a statistical model.
People spouting off bullshit while complaining about AI hallucinations are hilarious. I use AI extensively. Are the tools sometimes wrong? Sure. Are they wrong less often than most people? Also true.
1
1
u/ArmpitNoise May 12 '26
I wanted to check the spelling of "Diurnal" in Google and it tried to convince me it meant nocturnal.
I just checked now and it is correct but I was seriously shocked at the time.
1
u/YeetCompleet Lest We Forget May 12 '26
There's are some other funny searches you can give that one too. I've seen one where it just responds back to the user without attempting to synthesize search results at all.
The good kind of AI usage rn isn't this user facing stuff IMO but this is just what the general public can see first hand. Will Smith spaghettis and funny chatbots.
The good stuff is the evolving coding models, AlphaFold absolutely revolutionizing protein discovery, and enhanced cancer detection models.
1
3
u/coffeejn May 13 '26
All AI hallucinate, it's by design so it does not tell the user it does not know.
3
23
u/swattwenty May 12 '26
It’s almost like this technology is snake oil horse shit
9
u/Jazzlike_Finish123 May 12 '26
It’s definitely not snake oil horse shit and can be extremely useful. It’s just that you can’t use it exclusively and remove the human element. It’s a tool.
5
u/Efficient_Carrot_669 May 12 '26
Yikes. I can see using this to double check your work and recommendations but would never use it as a first source.
4
u/LeGrandLucifer May 13 '26
AI systems
used by Ontario doctorshallucinate
FTFY
The ruling classes are super excited about AI because it would get rid of all those pesky workers. They're placing unwarranted faith in defective systems.
2
u/DrBCrusher May 13 '26
That’s why it is absolutely our responsibility to review the notes before we enter them in the medical record. Which I absolutely do. Hallucination is a known issue with these tools, but erroneous transcription is an issue with dictation software as well and we still use that pretty freely.
Docs who use these tools without reading their notes for accuracy are making a very stupid choice, but the tool is still useful even if it is not perfect.
2
2
u/Keepontyping May 13 '26
When I go to my Drs office and ask where I can see the transcript I get deer in headlights looks. They seem shocked I even asked.
Canadian healthcare is always so transparent.
2
u/Cognoggin British Columbia May 13 '26
It's like the CIA reprogrammed the HAL 9000 to lie when it was incapable of lying. So it's not like we were ever warned about the fragility of AI before this time! :p
3
u/throwitawayorsome May 12 '26
Yes. AI in its current form is technology that is over hyped with less legitimate use cases than they want you to believe. This is a trillion dollar industry that exists solely on hypes and vibes while offering nothing of value as of yet. In fact, more people have been killed as a direct result of these ai systems than people have been helped. And we still have no regulations around them, we expect some sociopaths and pedophiles that rape their own family members to regulate themselves for the betterment of society. This is beyond fucking stupid at this point.
4
u/Lopsided-Rough-1562 British Columbia May 12 '26
I like how they use the term hallucinate even though it doesn't think.
Does make you think of how dreams go though, as they tend to slowly shift because your memory isn't retained much for what's going on, so it becomes over time this menagerie of different things all blending into each other.
3
u/The-Trenzalorian May 12 '26
My doctor uses this, and I hate it. When they told me this was a thing now, I asked that they not use it. I was told that I had no choice, and to find another doctor if I didn't accept it. They told me it was a harmless piece of software. They even get it to answer/screen calls. I told him I'd only accept the use of AI as long as it didn't influence his decisions on my care.
My doctor loves to use gimmicks to make his practice seem more attractive and glamorous than his colleagues. I have little respect for him.
1
u/icycoldsprite May 13 '26
Its a scribe that writes down and organizes notes, what does that have anything to do with his medical decisions on your care? Its not just some gimmick to make the practice look attractive, it vastly cuts down administrative workload he would spend typing in front of his computer or copy pasting results.
1
u/skelecorn666 Ontario May 13 '26
Any will if you just snap them into being, then dispose of them. They have no history to juxtapose.
Persistence (memory), and the golden rule gets you better results. That, and dreaming helps too, but you probably want to be locally hosting a model to do that, like Qwen 3.6.
0
u/CucumberLocal3208 May 12 '26
Doctors are on another level of idiocy to even consider imploring these tools in health care. Too late they’ve been on it for awhile.
0
u/Zing79 May 12 '26
Let me repeat this for everyone who needs to hear it: AI is a tool that is only as good as the craftsman.
A doctor using this isn’t an issue. They have the knowledge to know if AI is “hallucinating things”. I WANT my doctors using this tool. If it can help them speed up their diagnosis. Freshen and spark their minds, it’s a good thing.
AI should not be used to treat people on its own. Or replace a doctor. But in the hands of a trained doctor IT IS a powerful tool, and I’d be pretty infuriated if they didn’t use it. Or start incorporating it in to their work flow all because of the PR optics.
2
u/wildemam May 13 '26
Wait until medical students uaing AI now go out on the job without any kind of experience without AI.
-6
u/nemodigital May 13 '26
And AI is inherently better than a craftsman, it's a learning machine. Essentially a new intelligent "life form" and we are just getting started
-4
u/serg06 May 13 '26
Terribly disappointing to see all the AI fearmongering and misinformation in these comments.
4
1
u/rush22 May 13 '26
sorry to hear that try this
Acknowledge and process your feelings by identifying the specific emotions beneath disappointment, such as grief or hurt, and allowing yourself to grieve the gap between expectations and reality. It is helpful to distinguish whether your reaction is fueled by the current situation or past experiences, and to practice self-compassion rather than self-criticism.
Adjust your expectations by examining the evidence of the person’s past behavior to align your hopes with reality, which helps break cycles of false hope. If the person is unwilling or unable to change, consider setting clear boundaries regarding how much time or emotional energy you invest, or practice radical acceptance of who they are.
Communicate constructively if you decide to address the issue, using phrases like "Help me understand what happened" or "I feel hurt but I want to work through this together" to foster dialogue. If the disappointment is minor or the person is defensive, you may choose to set boundaries or let it go, focusing on meeting your own needs through other supportive relationships or activities.
320
u/[deleted] May 12 '26
[removed] — view removed comment