While some bias is related to the data itself, such as transcription errors or poor standardisation, perfect data collection will not eliminate every problem since accurate data might reflect health inequalities across society, says Dr Joseph Alderman, digital health clinical research fellow at the University of Birmingham. “We need to talk about how society has structural inequality, about people who cannot access healthcare. There are undertones and continuations of structural oppression, discrimination and racism that have reverberations throughout society, and those power dynamics lead to differences in how healthcare is delivered,” he said. “These inequities are so deeply embedded, unless someone delves into them, we never realise the problem.”
Alderman gave the example of a race ‘correction’ performed for over two decades that saw black patients incorrectly assigned higher kidney function than other ethnic groups, leaving some ineligible for important interventions like dialysis and kidney transplants. “Whenever we apply categories to things that are innate human social constructs with blurred boundaries and continua of variation, we inevitably lose resolution and coerce people into categories that probably do not represent them very well,” he said.
Those working in AI and health technologies do not have the ability to fix the root cause of the problem. “It is useful for them to flag the issue, but they are the wrong people to make a decision. It requires a multi-stakeholder, multidisciplinary — almost a transdisciplinary approach by policymakers and those who control budgets”, he said.
Solutions may include adjusting diagnostic thresholds for some groups despite algorithm suggestions, and addressing the health inequalities beyond the algorithm. “We must assume that bias exists unless proven otherwise”, said Liu. Without addressing biases at source, which could be beyond the data itself, any health technologies built on these algorithms risk amplifying existing biases.
STANDING Together
To tackle the underlying causes of data bias, Alderman and Liu, worked with partners on the STANDING Together (STANdards for data Diversity, Inclusivity and Generalisability) initiative, consisting of 58 countries and over 350 experts across several disciplines.
This was a transdisciplinary approach to solving the problem, where people from different sectors who would not traditionally agree on things come together to create a workable solution. “There are so many high-profile examples of where algorithms have gone wrong, both inside and outside of medicine, we did not have to convince anyone about the necessity of the exercise”, said Alderman.
The initiative employed a Delphi consensus study, which consists of a series of questionnaires designed to develop a consensus on topics that tend to have controversial opinions or have little empirical evidence. The project sought expert consensus on the key principles and practices to improve the quality and completeness of medical data.
Their published recommendations are freely available, and encourage transparency about the limitations of health datasets and their effects to encourage more informed choices about data usage and limits. That has reflected diverse perspectives across cultures. For instance, more homogeneous populations, such as those in Southeast Asia, are less concerned about ethnicity and race, and more concerned about income and employment status.
Similarly, some countries in Europe have historical reasons against collecting ethnicity data. “It became apparent how complex and difficult this would be. How do we create recommendations that are specific, yet general enough to provide coverage across the world? We decided that these issues are a much bigger conversation than we need for now”, said Dr Xiaoxuan Liu.
