My VIEW - The Rise of Ambient Scribes in Healthcare

VIEW - Saswata Raay, PhD

If you have been in a clinical setting recently — a GP clinic, hospital, or even a telehealth consult — you might have noticed something different. The doctor is not typing as much. They are present. The screen is no longer the third wheel in the room.

What changed?

Enter ambient AI scribing — a new wave of technology that listens, learns, and quietly writes your notes for you. New Zealand is part of this global shift, but it faces some unique challenges: diverse accents, robust consent protocols, and a deep commitment to patient trust.

What is ambient scribing?

Imagine a digital assistant in the background. With patient consent, an AI-powered tool records the clinical conversation and, in real-time or shortly after, generates structured notes ready for clinician review and integration into the electronic health record (EHR).

What this means for care

• Less screen time
• More patient focus
• Reduced clinician burnout

How does it work?

Ambient scribing technology typically combines three core AI components:

1. Speech-to-Text (ASR):
   Automatic Speech Recognition converts spoken interactions into text, forming the foundation of the scribing process. The accuracy of this transcription is crucial — any misstep here affects the entire downstream workflow.
2. Natural Language Processing (NLP):
   NLP interprets the transcribed text, extracting clinical concepts like symptoms, medications, and conditions, while distinguishing subtleties — such as whether a patient is reporting chest pain or denying it.
3. Large Language Models (LLMs):
   Advanced language models structure this information into clinician-friendly documentation, often using familiar formats like SOAP. "SOAP" is a widely used format in clinical documentation that stands for Subjective, Objective, Assessment, and Plan. It helps clinicians organise patient notes clearly by separating what the patient reports (subjective), what the clinician observes or measures (objective), the clinical impression or diagnosis (assessment), and the next steps for treatment (plan). Some tools work in real time, allowing immediate review; others opt for asynchronous processing to prioritise accuracy.
   

The outcome is note that is not just a transcript, but a coherent, clinically meaningful summary — streamlining workflow and improving the quality of documentation.

Can it handle accents and Te Reo Māori?

Accent adaptability is a critical success factor, especially in a culturally rich country like New Zealand. It is not enough to transcribe English well. Systems must accurately capture the nuances of regional dialects and Te Reo Māori.

Locally trained solutions have begun incorporating diverse voices into their training data, with in-clinic testing to fine-tune recognition across Aotearoa’s varied linguistic landscape. Community-led feedback loops are also emerging as key tools in refining accent and cultural responsiveness.

Accent handling is not just a technical challenge. It is a matter of equity, trust, and clarity in care.

Consent: the grey zone

Consent remains one of the most critical — and least standardised — aspects of ambient AI deployment.

Currently, most systems rely on verbal or manual consent, placing the onus on the clinician to inform patients at the start of each encounter. This raises important questions:

• Is the consent consistently recorded?
• Can it be audited?
• Should it be integrated into the EHR workflow?
  

Some innovators are exploring embedded consent prompts, digital flags, or even audio-logged opt-ins.

However, until a national standard emerges, variability will persist.

Consent cannot be an afterthought.

It must be designed into the workflow — visibly, consistently, and ethically.

Data privacy: whose story is it anyway?

At the core of ambient scribing lies something profoundly personal: your voice, your words, your health narrative.

Under the Privacy Act 2020 and Health Information Privacy Code, strong legal protections are in place. Many vendors claim to offer:

• End-to-end encryption
• Local (or ANZ-region) data hosting
• Pseudonymisation for model improvement
  

But deeper questions remain:

• Who owns the recordings?
• How long are they stored?
• Can patients opt out without losing access to services?
  

Transparent data governance and patient-first privacy practices will determine whether ambient AI earns — and keeps — public trust.

When did this all begin?

Ambient scribing is still in its early stages in Aotearoa, but momentum is growing.

• 2022–2023: Quiet adoption in private clinics and early pilots.
• 2023: Wider rollout in hybrid care environments.
• 2024: Public hospital pilots begin, tailored for local clinical and cultural contexts.
• 2025: New entrants focus on lightweight, clinician-friendly solutions with feedback-driven iterations.
  

What began as a tech trial is fast becoming a staple of daily practice.

What do clinicians think?

While early feedback is promising, clinician buy-in is not guaranteed. Usability, reliability, and workflow integration remain top concerns. Some find ambient scribes a game-changer; others feel it adds a layer of complexity. Continuous feedback and iterative improvement will be key to long-term success.

Final thoughts

Ambient AI holds the promise of giving healthcare professionals something invaluable: time. Time to listen. Time to care. Time to focus on the patient, not the keyboard.

But technology alone is not the answer. Its success depends on how we implement it - with care, accountability, and local context in mind.

That means:

• Empowering clinicians, not overwhelming them
• Making consent and privacy integral, not optional
• Designing for real-world Kiwi voices and workflows
  

Because ultimately, this is not just about automation. It is about bringing the human connection back to the heart of healthcare.

Read more VIEWS

Return to eHealthNews.nz home page