Sara Amare
Associate editor
Loyola University Chicago School of Law. JD 2028
AI use in health care is increasing. As of 2026, more than 80% of physicians reported using AI, double the rate reported in 2023. Physicians reported using health AI for assisting in diagnoses and drafting responses to patient portal messages. They believe that AI can be a useful tool for aiding patient convenience, work efficiency, and providing value based care. But do patients know that their providers are using AI? To ensure patients are making truly informed decicisons about their health, providers should be required to inform patients when AI is used in their treatment or care.
Who or what is making medical decisions?
A Pew Research Center survey revealed that U.S. adults are uncertain if AI is used while being treated by healthcare providers. Despite that, approximately 72% of U.S. adults believe it is important a provider disclose whether or not they are using AI for matters directly affecting their medical judgment. Others believe that AI disclosure should occur even if AI is being used for administrative tasks, such as obtaining prescription refills or scheduling appointments.
Informed consent is an essential aspect of ethical healthcare service, which cannot be given unless patients are properly educated about their care. Part of that education includes who, or what, is making medical decisions. Currently, the legal doctrine of informed consent does not explicitly require the disclosure of AI used to assist in providing medical care. However, there may be specific situations or circumstances that require disclosure. For example, if AI is used in a way that conflicts with the patient’s best interest, like the recommendation against a more costly but health effective treatment option, then disclosure is required. While patients are informed of AI use in some situations, there is currently no uniform mandate for AI disclosure.
Since there is no uniform federal law that requires healthcare providers to disclose AI use in medical decisions, various state laws have emerged. Some states have enacted laws that mandate disclosure of AI use in clinical treatments while others have adopted mandate transparency laws regarding AI use in matters such as patient communication or healthcare administrative decisions. For example, California enacted a statute requiring AI disclosure when AI is used to create patient communications as well as instructions on how patients can reach human healthcare professionals.
Federal regulation of AI use in healthcare does exist
Under HTI-1, a federal regulation made by the Office of the National Coordinator for Health Information Technology (ONC), a Department within Health and Human Services (HHS), the ONC has the power to regulate AI transparency, but only between certified health focused IT developers and healthcare providers. One way ONC regulates AI use in healthcare is by requiring developers of certified health IT to perform risk management for predictive decision support inteverntions (DSIs) within their software. Predictive DSIs are a healthcare AI tool that use AI to learn from new data and support the decision making process by implementing said data into an algorithm. Developers are required to disclose how data was acquired and managed, as well as evaluate risks related to accuracy and safety. ONC also requires that developers of certified health IT disclose risk management practice summaries to a public ONC site. Thus, some of this information is available to patients who actively look for it.
Additionally, ONC requires certified health IT modules that have predictive DSIs to provide comprehensive information regarding the design, development, and training of predictive DSIs. Developers are required to disclose whether or not their AI systems have undergone testing and validations. One of the purposes of this disclosure requirement is to help providers assess whether a predictive DSI meets the ONC ‘FAVES’ criterion, that is, whether the predictive DSI is fair, appropriate, valid, effective, and safe.
Congress should enact federal law mandating provider disclosure of AI use
These examples show that HHS has the ability to enforce disclosure of AI in healthcare. However, these transparency, risk management, and disclosure requirements do not apply to predictive DSIs that have been created internally by healthcare providers. They also do not mandate that providers disclose AI use in patient treatment, either for their records or directly to their patients. So, although a patient may have access to information on the risk management practices that developers of certified health IT employ, there is still no single federal mandate that requires disclosure of AI use by a provider directly to the patient.
Given the nature of what is at stake in making medical decisions, such as worsening conditions to a patient’s health, providers should be required to disclose AI use to their patients. Informed consent requires providers to ensure that their patients are fully informed about their treatments prior to agreeing with them. Mandating disclosure of AI use at a federal level ensures a patient’s informed consent is secure, regardless of the state the patient lives or seeks treatment in. A patient cannot be said to have freely made a decision to agree to a particular treatment if they do not have all of the necessary information, such as who, or what, is making these clinical decisions. Thus, Congress should enact federal law mandating that providers disclose to their patients when AI is used in their treatment or care.