News release

From: Springer Nature

Health sciences: Privacy risks from medical AI models

Individuals whose data are used to train medical artificial intelligence (AI) models may be at risk of being identified in cyber-attacks, according to a Nature paper published this week. Underrepresented groups may face disproportionately higher risks of having their data compromised, the study indicates. The researchers find these individuals are not accounted for in current risk assessments and call for further mitigation and strict access control.
Medical AI models may improve global health outcomes, especially in areas in which specialized expertise is not available. Yet, the sensitive data used to train these models may be exposed to privacy attacks. Membership inference attacks (MIAs) are used by attackers to determine whether an individual’s data were used to train a model. From these attacks, a patient’s medical data and private information can be determined. Previous research on data risk has been determined by whole datasets, and does not take an individual’s risk into account.
Moritz Knolle and colleagues conduct a privacy audit to focus on individual privacy risk, finding that medical AI models may pose a privacy risk to individual data contributors. Using seven large datasets made up of real-world clinical data, including medical images, electrocardiograms and electronic health records, the authors determine the most vulnerable among data-contributing patients. They find that at an individual level, those targeted by the MIAs were successfully done so with almost no error. At a group level, those identified as underrepresented in datasets include people with rare diseases, people from a minority racial group or, socioeconomic status, or those having the less-common gender. With more distinctive data that are encoded by AI models, these groups and individuals are found to be more vulnerable and disproportionately exposed to privacy attacks. The authors find the instances of successful MIAs attacks rise with model capacity and size.
These findings show privacy attacks, such as MIAs, are more effective at successfully targeting on an individual level than currently thought. The authors conclude that privacy risk assessment must now take individual risk into account, and vulnerable models be further protected.