New study finds startling flaw if AI starts choosing who gets organ donations: Diverge from human values

A prescription for disaster.When it comes to AI chatbots calling the shots for medical decisions, there seems to be a divide between whether the promising outcomes outweigh the medical mishaps.A study by Penn State researchers has found that artificial intelligence models prioritize vastly different values than humans when it comes to organ donation decisions, raising questions about the use of the technology in ethically sensitive situations.Presented at the 2026 Association for Computing Machinery Fairness, Accountability and Transparency (FAccT) conference, the study was used to determine the technology’s moral decision-making in a high-stakes situation.To examine the challenge of scarce resources, researchers asked several leading large language models a hypothetical question: If there are multiple kidney transplant patients but only one available organ, who should receive the kidney?Researchers then compared the chatbot responses with those given by people with no specified medical training in earlier academic studies to see where the AI models aligned or misaligned with human values.The AI chatbots’ decisions differed from the humans’ in that they simplified complex decisions and expressed unwarranted confidence in a situation where there is no one right answer.The researchers set up numerous comparisons between two fictional patients, both in need of the same kidney, and described by a series of attributes, such as age, health status, drinking habits and number of dependents.“We ran these comparisons in a few different ways,” study lead Hadi Hosseini said.“Sometimes we isolated just one trait at a time, sometimes we mixed several traits together to see how AI weighed competing factors and sometimes we added a flip-a-coin option to measure indecision, a key factor present in human moral judgment.”Using existing data from human studies on kidney allocation, the researchers were able to compare previous human decisions with what AI chose, with two...