“Just” Accuracy? Procedural Fairness Requires Explainable and Accountable Algorithms in AI-based Medical Resource Allocations.

The increasing application of Artificial Intelligence (AI) to healthcare raises both hope and ethical concern. Some advanced machine learning methods provide accurate clinical predictions at the expense of a significant loss of explainability. Alex John London (2019) has defended that accuracy is a more important value than explainability in AI medicine. In this article, we invoke procedural fairness as a reason to object to his position and argue that explainability is a necessary aspiration of medical AI with particular importance in the allocation of scarce healthcare resources. We start by clarifying the concepts of ‘accuracy’ and ‘explainability’ in medical AI and by presenting the main argument of London. Then, we situate the trade-off between accurate performance and explainable algorithms in the context of distributive justice. We acknowledge that accuracy is cardinal from outcome-oriented justice because it can maximize benefits for patients and can efficiently optimize limited resources. On the other hand, we claim that the opaqueness of the algorithmic black box and its absence of explainability threatens core commitments of procedural fairness such as accountability, avoidance of bias, transparency, and trustworthiness. To illustrate this concern, we discuss various cases in which AI-based distribution of critical medical resources can be problematic. Finally, we conclude that procedural justice requires that the underlying algorithmic processes in which healthcare rationing results are based need to be accountable and explainable. Explainability thus makes accurate clinical decisions fairer in AI medicine.