Accuracy, explainability, and avoidance of bias in AI-based medical resource allocations: A procedural fairness perspective

The increasing application of Artificial Intelligence (AI) to healthcare raises both hope and ethical concern. Some advanced machine learning methods provide accurate clinical predictions at the expense of a significant loss of explainability. Alex John London (2019) has defended that accuracy is a more important value than explainability in AI medicine. In this talk, we invoke procedural fairness as a reason to object to his position and argue that explainability is a necessary aspiration of medical AI with particular importance in the allocation of scarce healthcare resources. We start by clarifying the concepts of ‘accuracy’ and ‘explainability’ in medical AI and by presenting the main argument of London. Then, we situate the trade-off between accurate performance and explainable algorithms in the context of distributive justice. We acknowledge that accuracy is cardinal from outcome-oriented justice because it can maximize benefits for patients and can efficiently optimize limited resources. On the other hand, we claim that the opaqueness of the algorithmic black box and its absence of explainability is problematic from a procedural fairness perspective because it hinders the avoidance of biases. To illustrate this concern, we discuss various cases in which AI-based distribution of critical medical resources can be contentious. Finally, we conclude that procedural fairness requires that the underlying algorithmic processes in which healthcare rationing results are based need to be explainable. Explainability aligns with algorithmic justice as long as it enables the detection of biases in order to make accurate clinical decisions fairer in AI medicine.