New Recommendations to Increase Transparency and Tackle Potential Bias in Medical AI Technologies

Recommendations aims to improve the way datasets are used to build Artificial intelligence (AI) health technologies and reduce the risk of potential AI bias

Health technologies being shown by a doctor

Patients will be better able to benefit from innovations in medical artificial intelligence (AI) if a new set of internationally-agreed recommendations are followed.

A new set of recommendations published in The Lancet Digital Health and NEJM AI aims to help improve the way datasets are used to build Artificial intelligence (AI) health technologies and reduce the risk of potential AI bias.

Innovative medical AI technologies may improve diagnosis and treatment for patients, however some studies have shown that medical AI can be biased, meaning that it works well for some people and not for others. This means some individuals and communities may be ‘left behind’, or may even be harmed when these technologies are used.

An international initiative called ‘STANDING Together (STANdards for data Diversity, INclusivity and Generalisability)’ has published recommendations as part of a research study involving more than 350 experts from 58 countries. These recommendations aim to ensure that medical AI can be safe and effective for everyone. They cover many factors which can contribute to AI bias, including:

  • Encouraging medical AI to be developed using appropriate healthcare datasets that properly represent everyone in society, including minoritised and underserved groups;
  • Helping anyone who publishes healthcare datasets to identify any biases or limitations in the data;
  • Enabling those developing medical AI technologies to assess whether a dataset is suitable for their purposes;.
  • Defining how AI technologies should be tested to identify if they are biased, and so work less well in certain people.

Dr Xiao Liu, Associate Professor of AI and Digital Health Technologies at the University of Birmingham and Chief Investigator of the study said:

“Data is like a mirror, providing a reflection of reality. And when distorted, data can magnify societal biases. But trying to fix the data to fix the problem is like wiping the mirror to remove a stain on your shirt.

“To create lasting change in health equity, we must focus on fixing the source, not just the reflection.”

Under-representation from minority groups

The STANDING Together recommendations aim to ensure that the datasets used to train and test medical AI systems represent the full diversity of the people that the technology will be used for. This is because AI systems often work less well for people who aren’t properly represented in datasets. People who are in minority groups are particularly likely to be under-represented in datasets, so may be disproportionately affected by AI bias. Guidance is also given on how to identify those who may be harmed when medical AI systems are used, allowing this risk to be reduced.

STANDING Together is led by researchers at University Hospitals Birmingham NHS Foundation Trust, and the University of Birmingham, UK. The research has been conducted with collaborators from over 30 institutions worldwide, including universities, regulators (UK, US, Canada and Australia), patient groups and charities, and small and large health technology companies. The work has been funded by The Health Foundation and the NHS AI Lab, and supported by the National Institute for Health and Care Research (NIHR), the research partner of the NHS, public health and social care.

In addition to the recommendations themselves, a commentary published in Nature Medicine written by the STANDING Together patient representatives highlights the importance of public participation in shaping medical AI research.

Sir Jeremy Farrar, Chief Scientist of the World Health Organisation said:

“Ensuring we have diverse, accessible and representative datasets to support the responsible development and testing of AI is a global priority. The STANDING Together recommendations are a major step forward in ensuring equity for AI in health.”

The recommendations have been published today (18th December 2024), and are available open access via The Lancet Digital Health.

These recommendations may be particularly helpful for regulatory agencies, health and care policy organisations, funding bodies, ethical review committees, universities, and government departments.

Notes for editors

For media enquiries please contact Tim Mayo, Press Office, University of Birmingham, tel: +44 (0)7815 607 157.

The University of Birmingham

The University of Birmingham is ranked amongst the world’s top 100 institutions. Its work brings people from across the world to Birmingham, including researchers, educators and more than 40,000 students from over 150 countries.

  • England’s first civic university, the University of Birmingham is proud to be rooted in of one of the most dynamic and diverse cities in the country. A member of the Russell Group and a founding member of the Universitas 21 global network of research universities, the University of Birmingham has been changing the way the world works for more than a century.
  • The University of Birmingham is a founding member of Birmingham Health Partners (BHP), a strategic alliance which transcends organisational boundaries to rapidly translate healthcare research findings into new diagnostics, drugs and devices for patients. Birmingham Health Partners is a strategic alliance between seven organisations who collaborate to bring healthcare innovations through to clinical application:
    • University of Birmingham
    • University Hospitals Birmingham NHS Foundation Trust
    • Birmingham Women's and Children's Hospitals NHS Foundation Trust
    • Aston University
    • The Royal Orthopaedic Hospital NHS Foundation Trust
    • Sandwell and West Birmingham Hospitals NHS Trust
    • West Midlands Academic Health Science Network
    • Birmingham and Solihull Mental Health NHS Foundation Trust

STANDING Together

STANDING Together (building STANdards for data Diversity, INclusivity, & Generalisability) was established in 2021 as part of the NHS AI Lab’s AI Ethics initiative.

  • It is a partnership between over 30 academic, regulatory, policy, industry, and charitable organisations worldwide.
  • STANDING Together was funded jointly by the NHS AI Lab at the NHS Transformation Directorate, and by The Health Foundation, and is managed by the UK’s National Institute for Health and Care Research (NIHR).
  • It is led by medical researchers based at University Hospitals Birmingham NHS Foundation Trust, and The University of Birmingham (both based in Birmingham, United Kingdom).
  • The STANDING Together recommendations have been included in guidance published by the Medical Research Council (MRC), the Medicines and Healthcare Products Regulatory Agency (MHRA), and the Therapeutic Goods Administration (TGA).
  • The recommendations were also cited in both the Equity in Medical Devices Independent Review chaired by Professor Dame Margaret Whitehead, and in the Government’s official response.

Alongside their recommendations, the STANDING Together initiative has published a series of related scientific papers (in reverse chronological order):

Exploring patient and public participation in the STANDING Together initiative for healthcare artificial intelligence (AI) in healthcare. https://www.nature.com/articles/s41591-024-03200-6

Revealing transparency gaps in publicly available COVID-19 datasets used for medical artificial intelligence development—a systematic review. https://www.thelancet.com/journals/landig/article/PIIS2589-7500(24)00146-8/fulltext

A Global Health Data Divide. https://ai.nejm.org/doi/full/10.1056/AIe2400388

The value of standards for health datasets in artificial intelligence-based applications. https://www.nature.com/articles/s41591-023-02608-w

Tackling bias in AI health datasets through the STANDING Together initiative. https://www.nature.com/articles/s41591-022-01987-w

 

The Health Foundation

The Health Foundation is an independent charitable organisation working to build a healthier UK. www.health.org.uk

 

About the National Institute for Health and Care Research

The mission of the National Institute for Health and Care Research (NIHR) is to improve the health and wealth of the nation through research. We do this by:

  • Funding high quality, timely research that benefits the NHS, public health and social care;
  • Investing in world-class expertise, facilities and a skilled delivery workforce to translate discoveries into improved treatments and services;
  • Partnering with patients, service users, carers and communities, improving the relevance, quality and impact of our research;
  • Attracting, training and supporting the best researchers to tackle complex health and social care challenges;
  • Collaborating with other public funders, charities and industry to help shape a cohesive and globally competitive research system;
  • Funding applied global health research and training to meet the needs of the poorest people in low and middle income countries.

NIHR is funded by the Department of Health and Social Care. Its work in low and middle income countries is principally funded through UK Aid from the UK government.

 

University Hospitals Birmingham NHS Foundation Trust

University Hospitals Birmingham NHS Foundation Trust (UHB) runs Heartlands, Good Hope, Solihull and the Queen Elizabeth hospitals; UHB also hosts the Institute of Translational Medicine (ITM).

  • The Trust has over 2,700 beds, over 80 theatres and a 100-bedded critical care unit - the largest in Europe, and cares for over 2.2 million patients each year, employing over 24,000 staff.
  • QEHB is a Major Trauma Centre treating the most severely injured casualties from across the region.
  • Our hospitals are regional centres of excellence for trauma, burns, plastics, neurosciences and cancer and in 2014 became a lead genomics centre as part of the NHS 100,000 genomes project.
  • UHB is proud to host the Royal Centre for Defence Medicine (RCDM). The RCDM provides dedicated training for defence personnel and is a focus for medical research.