Patients should be told when doctors use AI to listen in and take notes, expert says
Nearly half of all General Practitioners (GPs) are using Artificial Intelligence (AI) ‘scribes’ to listen to consultations and make notes, but patients are not always told, or asked for their permission, says a University of Melbourne Law School researcher.
Dr Megan Prictor warns there are important issues around patient privacy and also accuracy, with studies finding that AI-produced medical notes suffer from mistakes, omissions and ‘hallucinations’ (when a computer generates information that seems accurate, but is actually false or misleading).
“A study published recently in the British Medical Journal looked at seven different scribe products and found between one and six important omissions in each consultation (family history, smoking habits, medications). Half the recordings had factual errors, and hallucinations were common; for example, saying someone smoked when they didn’t, or inventing diagnoses, or listing medications that were not used – one recorded a man on the contraceptive pill.
“Time-pressed doctors might not always be reviewing those notes as they should. I am worried about the risk of harm, actual physical harm, that people might suffer due to misdiagnoses and medication errors.”
Dr Prictor, who specialises in the study of emerging health technologies, discusses the issues in the latest episode of the podcast JUSTICE with Jon Faine, out today. She describes AI scribes as software products that listen in to healthcare consultations and create a summary for a medical record.
She says that use of AI scribes by GPs has doubled from 20 per cent to 40 per cent last year but is largely unregulated in Australia: “It is the Wild West of medical technology.Doctors might not even think to tell patients they are being recorded in this way.
“We need regulations and protocols to be set around its use, to help protect patients not just from errors, but from invasions of privacy. These tools are listening in to some of our most intimate conversations and making summaries of them.
“We need the Therapeutic Goods Administration to set rules around its use, and greater transparency about where our health information is going. Some scribe products store patient data for shorter or longer periods, making it vulnerable to hacking.
“With large hospitals now setting up AI scribes in their record systems, it is time to look at these questions.”
Dr Prictor says it is understandable that GPs find AI scribes useful, as they allow doctors to focus on speaking to patients without dividing their attention between the patient and note-taking. They can also save time. “There is even some evidence that they can help prevent burnout in doctors.
“But it is crucial that patients are informed and offered a choice. The doctor should explain that patients are consenting to the ‘listening’ in by the AI scribe, which is required by Australia’s surveillance laws in many states. The doctor should also explain that patients are consenting to the collection of their personal medical information, which qualifies as sensitive information under our privacy laws.
“It might also include consenting to the algorithm processing your information, which is covered by privacy laws governing the use of information by the machine.”