Just” accuracy? Procedural fairness demands explainability in AI-based medical resource allocations.

The increasing application of Artificial Intelligence (AI) to healthcare raises both hope and ethical concern. Some advanced machine learning methods provide accurate clinical predictions at the expense of a significant loss of explainability. Alex John London (2019) has defended that accuracy is a more important value than explainability in AI medicine. In this article, we invoke procedural fairness as a reason to object to his position and argue that explainability is a necessary aspiration of medical AI with particular importance in the allocation of scarce healthcare resources. We start by clarifying the concepts of ‘accuracy’ and ‘explainability’ in medical AI and by presenting the main argument of London. Then, we situate the trade-off between accurate performance and explainable algorithms in the context of distributive justice. We acknowledge that accuracy is cardinal from outcome-oriented justice because it helps maximizing patients’ benefit and optimizes limited resources. However, we claim that the opaqueness of the algorithmic black box and its absence of explainability threatens core commitments of procedural fairness such as accountability, avoidance of bias, transparency, and trustworthiness. To illustrate this, we discuss liver transplantation as a case of critical medical resources in which lack of explainability in AI-based allocation algorithms is problematic. Finally, we conclude that procedural justice requires that the algorithmic processes underlying healthcare rationing results are accountable and explainable. Explainability makes accurate clinical decisions fairer in AI medicine.