Use of artificial intelligence (AI) in healthcare is on the rise. Bodies including UK Governments, the National Institute for Health and Care Research and the NHS AI Lab are all investing in developing and deploying the technology.
As the landscape evolves, health information producers are investigating the risks and benefits of using AI in their everyday work. Developed in collaboration with PIF’s AI working group, this position statement aims to help members understand the AI landscape and how to manage it.
A full framework for policy creation is in development and will be published in the autumn of 2024.
AI is a catch-all term covering diverse technology, from self-driving cars to analysis of large scientific data sets. The UK Government defines AI as “the use of digital technology to create systems capable of performing tasks commonly thought to require human intelligence”*. There are different subsets of AI*. The most relevant to health information is machine learning (ML). Generative AI (GAI) is based on ML techniques.
Machine learning (ML)
A type of AI that uses computer algorithms that “learn”. It detects patterns in sample data, then applies these findings to new data to create “outputs”. Typical ML outputs include predictions or recommendations.
Deep learning (DL)
A form of Machine Learning using computational structures called neural networks to spot patterns in data and provide a prediction or evidence for a decision.
Large language models
A type of AI model that uses Machine Learning to learn patterns in large volumes of text to carry out language-related tasks. ChatGPT is an example of an ML-driven LLM.
Generative AI (GenAI or GAI)
A form of AI that uses Machine Learning techniques to generate new text, images, audio, or video from existing content.
ML works by using statistical techniques to spot patterns in large data sets. It then uses these patterns to create “outputs”. Typical ML outputs include predictions or recommendations. For example, website “live chat” and chat bot functions learn to spot patterns in the questions people commonly ask a health charity. They can then direct people with similar queries to the relevant information. Deep learning (DL) is a more sophisticated form of ML. It “layers” algorithms to make more accurate predictions. ML and DL models tend to be either supervised or unsupervised.
In supervised learning, programmers tell the model what to look for in the training data. Unsupervised models are left to spot patterns for themselves.
Generative AI uses ML and DL to create new text, images, video, audio, or other content. When related to text, you might also see these referred to as Large Language Models (LLMs). Examples include user interfaces such as ChatGPT and BLOOM.
For example, if ChatGPT is asked to summarise bowel cancer symptoms it will search its training datasets for content. The model will then use its probability distributions and internal control mechanisms to generate a text summary.
Used as part of a robust information production process, AI may streamline the development and delivery of health information.
Applications in use and under consideration include:
This could help healthcare information providers to “do more with less”. Delegating tasks such as data analysis to computers, for example, can free staff to work on other activities.
AI-enabled solutions could also help extend an organisation’s reach. For example, AI translation can help teams serve seldom-heard communities and automated chat bots can provide support outside of office hours. Generative AI (GAI) is based on ML techniques.
There are risks associated with the use of AI which health information providers should be aware of.
These challenges relate to AI in all its uses. But, arguably, they are of critical importance in the health information space where we strive to produce accurate, unbiased, inclusive materials.
AI runs on data. Whether developing models in-house or outsourcing to developers, organisations need to ensure personal information is secure and used in line with relevant regulations*.
ML and DL algorithms learn from the data they are given. If the training data contains bias, this is likely to be reflected or compounded in the AI model’s outputs. AI has been shown to mirror existing misinformation and prejudices.
Hallucinations are when AI generates convincing but completely made up content producing incorrect or misleading results*. This can include fake references.
Some AI includes relevant content, regardless of source or accuracy, in its analysis. For example, GAI models will trawl their training data or the internet looking for answers to the questions they have been given*. If the information found is biased or incorrect, the AI model may spot patterns that do not exist.
Models also tend not to have access to paywall or otherwise protected content. This means primary sources are often excluded from analysis.
GAI models are trained on pre-existing data, meaning they do not have access to the latest information or research.
For example, the first free-to-use version of ChatGPT was trained on data published before 2021*. This meant any searches relating to COVID-19 returned drastically out-of-date results.
GAI tends to over simplify health topics because it lacks the ability to apply context or nuance to its results, or to understand the meaning behind the data*.
While it can be useful for researching general information on a specific health condition, it is unable to interpret and relay how different health conditions or interventions may impact on each other.
AI cannot show its workings. We may know or be able to control what data goes in, and the results that come out, but not how the model comes to its conclusions.
This makes it challenging to know where the AI-generated information came from, and whether it is accurate or biased*.
A recent survey of more than 17,000 people from 17 countries found 61% were wary about trusting AI systems*. It is vital the use of AI does not undermine public trust in vital health information resources.
Currently AI has a high risk of inaccuracy. This means using AI to produce health information comes with complex and unanswered questions around the liability of AI-generated output used by an organisation*.
AI tools make use of content from health charities, the NHS and other trusted sources. This could lead to a reduction in direct traffic to websites which place information in its full context. This could have wider impacts on an organisation’s sustainability.
The use of AI tools creates a risk of copyright breach5. In January 2024, UK Parliament government confirmed AI training data will infringe copyright unless permitted under licence or an exemption*.
GAI tools which search the internet for answers can pull large sections of copyrighted text from sources including charity and commercial websites. This can have both ethical and legal implications.
The challenges and opportunities of AI are issues attracting attention at the highest levels.
Taking no action is not an option. AI is here to stay, and forbidding its use is unfeasible. We need to manage the risks, not ignore them.
PIF recommends health information providers create AI usage policies. Clear policies will provide teams with the guardrails they need to use AI responsibly and with confidence.
Transparency should be central to all applications of AI technology. The public needs to be able to trust our members if they are to trust the information they provide.
Being open and upfront about how and why the organisation is using AI is key to building and maintaining that trust.
Trust will also be central in helping us to learn from each other as we continue on our mission to balance the risks and benefits of AI. This technology is rapidly evolving and here to stay.
Our role at PIF is to work with members to harness the opportunity, manage the risk and provide education and support to the members and the public.
This position statement has been developed in collaboration with the PIF AI Working Group. The group is made up of members and non-members representing the charity sector, public health bodies, NHS Trusts, regulatory bodies and commercial companies.
The position statement is the result of a webinar and a member round table. We would like to thank members who took part in both events. A recording of the webinar is available here.
Contributors:
Trishna Bharadia, The Spark Global
Sheena Campbell, PIF
Caroline De Brún, UK Health Security Agency
Sue Farrington, PIF
Lynsey Hawker, The Kings Fund
Steph Jury, Guy’s and St Thomas’ NHS Foundation Trust
Raisa McNab, Association of Translation Companies
Dr Knut Schroeder, Expert Self Care
Julie Smith, EIDO Systems
Sophie Randall, PIF
Author: Amanda Barrell, medical and life sciences writer.
Algorithm: The sequence of rules a computer uses to convert an input, or the dataset into an output, or a pattern in the data.
Artificial intelligence (AI): Machines that can perform tasks previously carried out by human intelligence.
Chat bot: Software designed to mimic human conversation that “talks” to users via speech or text.
Deep learning (DL): A form of Machine Learning (ML) using computational structures called neural networks to spot patterns in data and provide a prediction or evidence for a decision.
Generative AI (GenAI or GAI): A form of AI that uses Machine Learning (ML) techniques to generate new text, images, audio, or video from existing content.
Human in the loop: AI systems that combine the power of human and artificial intelligence. The human can intervene by training, fine-tuning or testing the system’s algorithm, for example, to help it produce more useful results.
Large Language Model (LLM): A type of AI model that uses Machine Learning (ML) to learn patterns in large volumes of text to carry out language-related tasks. ChatGPT is an example of an ML-driven LLM. LLMs are notable for their ability to achieve general-purpose language generation and understanding. Modern LLMs can produce entire sentences, paragraphs, or even entire documents.
Machine learning (ML): A type of AI that uses computer algorithms that “learn”. It detects patterns in sample data, then applies these findings to new data to create “outputs”. Typical ML outputs include predictions or recommendations.