Investigation 1 draws a blank

How easy is it for people to judge the quality and reliability of YouTube videos on Covid and Covid vaccines? And what can be done to make such judgements easier for people?

As part of my studies towards an MSc in public health, I searched the biomedical literature to find out.[1] Disappointingly, I found no studies with the main aim of answering these particular questions.

As a PIF member, I think it’s worth pointing out straight away that the PIF TICK, a UK-wide quality mark for trusted health information, is one thing that seems well placed to help here. No matter what the overall quality of health-related videos on YouTube, the PIF TICK would help people to identify videos they can trust.

 

Investigation 2 offers learnings for PIF members from 25 studies

As the questions asked in my first literature search are quite specific, I broadened the line of enquiry and found 25 studies that evaluated nearly 3,000 YouTube videos on Covid and/or vaccines with over 1.7 billion views.

The studies recommend ways to help ensure information on YouTube about Covid and vaccination is reliable and high quality. They also detail how they go about judging quality.

I think they offer valuable learnings for PIF members who are creating, or signposting to, videos on any health-related matter, so I’ve shared findings here.

 

But first, why is this important?

YouTube is the second-most-visited website globally, after Google.[2]

Over 1 billion hours’ worth of video is streamed on YouTube each day.2 Over 2 billion unique users visit the site monthly.[2]

Given YouTube’s vast reach, it seems important that people can judge the reliability of videos they find, to understand what might help people to make such judgements and to know what videos on YouTube are like and how to make them better.

 

What recommendations did the studies make?

Between them, the studies made 28 different recommendations. The top five, in terms of number of mentions, are listed below:

  • Increase the presence and visibility of videos from reliable and authoritative organisations
    • E.g. videos from the World Health Organisation, national health agencies, and medical or academic institutions
    • It would be interesting to know what PIF members think of sharing their content on YouTube
  • Encourage more collaboration in video production between different groups/sectors
    • E.g. between public health agencies or hospitals and the entertainment or news industries or social media influencers
    • Again, I wonder how many PIF members join, or would like to join, such collaborative efforts
  • Monitor the content and quality of information
  • Use YouTube to disseminate public health information, e.g. WHO recommendations
  • Block misleading videos and/or increase editorial control.

 

What is YouTube doing?

I can’t claim any expertise here, and YouTube is best placed to answer this, but many of the studies mentioned YouTube’s Covid-19 Medical Misinformation Policy, announced in May 2020. This policy states that YouTube does not allow medical misinformation about Covid-19 that poses a serious risk of egregious harm.[3]

YouTube is also planning to showcase videos produced by credible outlets in a ‘health content shelf,’ near the top of search results for a few hundred health-related queries.[4]

Such efforts seem important. And PIF members may want to be on that list of credible outlets.

It seems equally important to do what we can to help people to judge videos for themselves. It is worth plugging the PIF TICK again, because this quality mark is well placed to help.

 

How did the studies go about evaluating the quality of YouTube videos on Covid?

Combinations of approaches were used:

  • Words: When using words to describe what high-quality videos are like – what ‘good’ looks like – the studies used a range of terms, including quality, reliability, understandability, actionability, completeness, relevance, depth and accuracy.
  • Tools: The researchers used questionnaires and other tools designed for evaluating consumer health information, such as the DISCERN and modified DISCERN questionnaires, the Global Quality Score (GQS), the Journal of the American Medical Association (JAMA) benchmark criteria and HonCode (the Health on the Net Foundation Code of Conduct).
  • Topic/content checks: Content coverage of the videos was checked against lists of topics deemed most relevant to Covid-19 and/or vaccines, with several researchers using a checklist called the Medical Information and Content Index (MICI).
  • Fact checks: Claims made in videos were compared with information from authoritative sources (such as the World Health Organisation).
  • Other characteristics, e.g.: 
    • Tone (e.g. whether pro or anti-vaccines)
    • Categorisations (e.g. useful/misleading)
    • YouTube metrics (such as number of views, likes and comments)
    • Type of person or organisation that uploaded the video (e.g. public health organisations, news agencies, healthcare professionals and consumers)
    • Other things such as the duration and language of videos
    • It’s important to note that these characteristics aren’t necessarily all indicators of quality. The studies investigated relationships between quality and characteristics such as number of likes, with varying results.

 

A big thank you to my tutors at Brighton and Sussex Medical School

This blog summarises some of the findings of a literature review that I conducted as part of my studies towards an MSc in public health at Brighton and Sussex Medical School. I hope it helps.

I’d like to thank my tutors for their support, particularly Drs Louise Sigfrid, Priya Paudyal and Marija Pantelic, and for running the course in a way that enables students to pursue lines of investigation that fit with their own work and interests.

 

 

References

  1. Databases searched: PubMed, Embase, PsycInfo, Cochrane and MedRxiv.
  2. Chan C, Daniels E, Normahani P, Markar S, Sounderajah V, Acharya A, et al. The Reliability and Quality of YouTube Videos as a Source of Public Health Information Regarding COVID-19 Vaccination: Cross-sectional Study. JMIR public health and surveillance. 2021;7(7):e29942. Available from: https://publichealth.jmir.org/2021/7/e29942/
  3. YouTube. COVID-19 medical misinformation policy 2021. Available from: https://support.google.com/youtube/answer/9891785?hl=en-GB.
  4. Grant Currin. YouTube’s plan to showcase credible health information is flawed, experts warn. Scientific American, August 27, 2021. Available from: https://www.scientificamerican.com/article/youtubes-plan-to-showcase-credible-health-information-is-flawed-experts-warn/

See more

A year of progress on health and digital literacy

Published:
27th October 2021
Read More

Promoting evidence-based health information

Published:
20th October 2021
Read More

PIF TICK – one year on and looking to the future

Published:
9th June 2021
Read More

The Perfect Patient Information Journey (PPIJ) and integrated public private pathways

Published:
22nd April 2021
Read More

Digital exclusion in the time of COVID-19

Published:
31st March 2021
Read More

Health Information Challenges and Solutions 2021

Published:
25th March 2021
Read More

PIF: A year in review

Published:
6th January 2021
Read More

Engaging Long COVID patients

Published:
30th November 2020

Patient Safety Learning looks at some of the concerns facing people living with Long COVID.

Read More

Health Information Challenges and Solutions

Published:
25th November 2020

This week PIF held its Health Information Challenges and Solutions Workshop online for the first time.The open space event is a chance for members to…

Read More

Transforming healthcare through number confidence

Published:
18th November 2020
Read More

'Together we can continue to ensure we move to a health literate world'

Published:
6th October 2020
Read More

Using behavioural insights to encourage a return to care

Published:
14th September 2020

Lack of information and fear of Covid-19 is preventing some patients from getting the healthcare they need. Applying behavioural insights to information…

Read More

Finding credible healthcare information

Published:
15th July 2020
Read More

Why is health information important to me? It isn’t just my job

Published:
7th July 2020
Read More

Key practice points on user engagement

Published:
25th June 2020
Read More

Supercharged Superhero by Gemma Everson

Published:
8th April 2020
Read More

Event report – Using digital tools to maximise health information

Published:
6th March 2020
Read More

Children's book aims to raise awareness of invisible illnesses

Published:
14th January 2020
Read More

Making digital services accessible

Published:
18th December 2019
Read More

Event report – Health Information for Children and Young People

Published:
14th November 2019
Read More

Is digital and social inclusion key to transformational change?

Published:
15th October 2019
Read More

Patient information a ‘linking thread’ at NHS Expo 2019

Published:
12th September 2019
Read More

Health literacy challenge and the NHS Long Term Plan

Published:
2nd September 2019
Read More

Developing health information to engage everyone

Published:
2nd July 2019
Read More

EU clinical trial sponsors to publish lay-friendly summaries of key findings

Published:
11th April 2019
Read More

Patient-centred care: all in this together

Published:
11th April 2019
Read More

Build trust and confidence to engage with seldom heard groups

Published:
4th April 2019
Read More

NHS plan promises empowered and informed patients via digital tools

Published:
11th January 2019
Read More

Health Information. How has it changed over the last 21 years?

Published:
4th January 2019
Read More

Changes in health information over the past 21 years – and the outlook for the future

Published:
4th January 2019
Read More

Vlog: Celebrating 21 years of health information

Published:
14th December 2018

Listen to Ellie Rose, Patient Information Editor at the Royal Marsden NHS Foundation Trust, vlog about what she finds useful about her…

Read More

Being heard above the noise

Published:
11th December 2018
Read More

Developing information for older people with long-term health conditions

Published:
30th November 2018
Read More

Reviewing public information issued by health and social care bodies

Published:
27th November 2018
Read More

Measuring and evaluating the impact of health information

Published:
22nd November 2018
Read More

Industry needs to work in partnership with patients and groups

Published:
22nd November 2018
Read More

Wales moves towards a digital health future for all

Published:
15th November 2018
Read More

Helping the #HiddenHalf – health information for maternal mental health

Published:
13th November 2018
Read More

Patients encouraged to ask professionals about medicines

Published:
4th October 2018
Read More

Users views needed on digital health records

Published:
26th September 2018
Read More

Getting health information right impacts health outcomes

Published:
20th September 2018

Blog by Amy Corkery – Editor, Patient Information, British Heart Foundation

Read More

Shifting the goalposts: the quest for a digital NHS

Published:
16th June 2017
Read More