While promising, the integration of artificial intelligence (AI) into healthcare systems may inadvertently lead to uneven access, according to collaborative research by the University of Copenhagen, Rigshospitalet, and DTU.
The study scrutinized AI's ability to identify depression risk across different demographic segments, highlighting the need for vigilance in algorithm implementation to curb potential biases. The researchers advocate for rigorous evaluation and refinement of algorithms before being released.
AI in the Healthcare Sector
The study noted that AI is progressively finding applications in the healthcare sector, from enhancing MRI scans to enabling swift emergency room diagnoses and improved cancer treatment plans.
Danish hospitals are among those testing AI's potential in these areas. Danish Minister of the Interior and Health, Sophie Løhde, envisioned AI as a future cornerstone in alleviating strain on the healthcare system.
AI's proficiency in risk analysis and resource allocation is invaluable in healthcare settings. It aids in directing limited resources where they can be most effective, ensuring, for instance, that therapies reach patients who stand to benefit the most.
Some countries have already employed AI to determine suitable candidates for depression treatment, a practice that might extend to Denmark's mental health system.
However, the University of Copenhagen researchers stressed policymakers' need for thoughtful consideration to prevent AI from inadvertently exacerbating inequality or becoming a tool for purely economic calculations. They cautioned that recklessness in implementation could inadvertently hinder rather than help.
Melanie Ganz from the University of Copenhagen's Department of Computer Science and Rigshospitalet emphasized the potential of AI but underlined the necessity for cautious deployment to avoid unintended distortions in the healthcare system. The study underscored how biases can subtly infiltrate algorithms designed to assess depression risk.
Evaluating Algorithms
The study, co-authored by Ganz and her colleagues from DTU, established a foundation for evaluating algorithms in healthcare and broader societal contexts. It aims to identify and rectify issues promptly, ensuring fair algorithmic practices prior to their implementation.
While algorithms, when appropriately trained, can be valuable tools for optimizing resource allocation in resource-constrained municipalities, the research revealed potential disparities in the algorithm's effectiveness across different demographic groups.
Several factors, such as education, gender, and ethnicity, influenced the algorithm's ability to identify depression risk, exhibiting variations of up to 15% between groups.
This signifies that even with well-intentioned implementation, an algorithm designed to enhance healthcare allocation can inadvertently skew efforts. The researchers warned that it's imperative to scrutinize the algorithms for hidden biases that may lead to the exclusion or deprioritization of specific groups.
Furthermore, the researchers raised ethical concerns surrounding AI implementation, particularly regarding the responsibility for resource allocation and treatment decisions resulting from algorithmic outputs. This calls for transparency in decision-making processes, especially when patients may seek explanations for algorithm-driven decisions.
"Both politicians and citizens must be aware not only of the benefits, but also the pitfalls associated with the use of AI. So, one can be critical instead of just 'swallowing the pill' without further ado," said co-author Sune Holm from the Department of Food and Resource Economics.
The study's findings were presented at the 2023 ACM Conference on Fairness, Accountability, and Transparency.
Related Article : Listening to Your Favorite Music May Help You Deal Better With Pain, New Study Shows
