The increasing use of artificial intelligence scribes by general practitioners (GPs) to generate medical notes could put patients at risk by missing vital information conveyed through gestures and facial expressions, according to a new study.
Approximately 40% of GPs now utilize these AI tools, often referred to as 'ambient voice technology,' which convert spoken conversations during appointments into text and create summaries. While intended to free up doctors' time by reducing administrative tasks and allowing them to focus more on patients, researchers from the University of Edinburgh have raised concerns.
Their review of 27 articles published in scientific and medical literature indicates that these AI programs capture spoken words but fail to register non-verbal cues such as body language, facial expressions, and the tone of voice used by both patients and clinicians. This 'written language bias' means that potentially crucial information, which a human clinician might glean, can be overlooked.
The study, published in BMJ Digital Health and AI, suggests that patients might alter their behavior in front of AI technology, potentially withholding sensitive information related to substance abuse, domestic violence, or mental health struggles due to a lack of trust. This could lead to less disclosure and further disadvantage individuals already facing marginalization in healthcare settings.
Furthermore, the AI-generated summaries tend to prioritize clinical details, sometimes at the expense of patients' personal narratives and their lived experiences of illness. Researchers noted that the technology might lead to a 'deskilling' of doctors, reducing their opportunities to hone their clinical reasoning abilities.
NHS England has encouraged the adoption of ambient voice technology, emphasizing that doctors must review AI-generated notes and obtain patient consent before using them during consultations. However, the review highlights that the technology's implementation needs careful adaptation to clinical contexts, as it has the potential to reshape how healthcare work is organized and performed.
Some clinicians have reported not recognizing AI-generated notes and experiencing difficulty recalling patient details from previous visits. Medical Protection, an organization providing medical indemnity, acknowledged the administrative burden on doctors and welcomed AI tools that help alleviate this. However, they stressed the importance of a clear framework for safe AI utilization.
Dr. Sarah Townley, deputy medical director at Medical Protection, advised doctors that they remain responsible for ensuring AI-produced notes are clear, accurate, and complete. She emphasized the need for informed patient consent, especially when personal data is shared with third parties, and recommended that doctors receive adequate training on AI scribes to understand their limitations and data protection implications.