ChatGPT blocked while Te Whatu Ora considers its use

eHealthNews.nz: National Systems & Strategy

ChatGPT blocked while Te Whatu Ora considers its use

Friday, 5 May 2023

Share |

NEWS - eHealthNews.nz editor Rebecca McBeth

Access to websites such as OpenAI’s ChatGPT have been blocked by Te Whatu Ora while the organisation works to understand how to best use these platforms safely.

Jon Herries, Te Whatu Ora – Health New Zealand, emerging health technology and innovation, says a recent update to one of the organisation’s security firewalls blocked access to websites like OpenAI’s ChatGPT.

“Before making any changes to this firewall we are using this opportunity to understand how we best use these platforms safely,” he says.

“Therefore we are working with our AI and Algorithm Expert Advisory Group to understand how we best manage access to AI like ChatGPT in ways that are aligned with our strategic intent and values, as well as evolving legal and community standards.”

Herries says Te Whatu Ora recognises the speed with which artificial intelligence is moving and the potential this presents for many sectors, both good and bad.

You’ve read this article for free, but good journalism takes time and resource to produce. Please consider supporting eHealthNews by becoming a member of HiNZ, for just $17 a month.

“This is leading to many questions across government and industry, with concerns raised in terms of using personal or confidential data to train AI, how right or wrong (accurate) it is and how it can be deceived or perverted,” he says.

During a HiNZ webinar on AI in Health – What does ChatGPT mean for me? Chris Paton, director, postgraduate digital health programme, Otago University said there are two obvious use cases for ChatGPT in health. One is for the AI to listen to consultations and provide a summary, to save the health professional’s time in writing up notes.

The other is providing initial triage of patients and provision of some basic health information.
“I think both of those use cases will become quite sophisticated in the very near future and be real possibilities,” said Paton.

However, the danger is that these tools are constantly evolving, so if evaluated at one point to be safe and effective, it would change “almost immediately”.

“There are all sorts of paradoxes and ethical dilemmas related to doing things like triage and recording consultation so we are going to have to be very careful in thinking about how we evaluate and monitor these systems to ensure that they stay useful and helpful and do not start getting dangerous.”

He said that OpenAI was initially using data input into ChatGPT to train its newer versions, so any data entered could be used and made available to other users. However, recent changes mean you can opt out of training the algorithm with your data.

Albert Bifet, director, Te Ipu o te Mahara AI Institute, Waikato University, said some large language models (LLMs) are open source, but OpenAI is not open about what data it is using to train ChatGPT, or how the model works.

He said data privacy and sovereignty is very important, particularly in relation to Māori data, so it is something the health systems needs to be concerned and careful about.

Nick Kemp, chief executive of Wild Bamboo, said he would not recommend inputting any patient information into a tool such as ChatGPT but it could be used for generic queries about conditions and how to manage them, such as suggesting a meal plan for someone with coeliac disease.

“There needs to be a balance between harnessing and leveraging these tools in the right way, whilst protecting what is important as well,” he said.

Tania Moerenhout, lecturer, Bioethics Centre, Otago University, said responsibility currently sits with clinicians for providing care, but this question becomes much harder when considering the use of AI systems.

“We will have to think about regulation for cases when the outputs of these tools are incorrect,” she says.

“At this point, we cannot use ChatGPT in this way as it is highly unreliable in the output it provides, but it is already past the time to ask these questions, and we have still not really dealt with them in a thorough way.”

Watch the AI In Health webinar on-demand.