The use of artificial intelligence in healthcare is rapidly expanding, and governments around the world are taking steps to regulate this technology. In the United States, the Therapeutic Goods Administration (TGA) is "stepping up its efforts" to regulate digital scribes, including those using AI technology, following calls for greater oversight.
The TGA warns that AI scribes with advanced features may need to be regulated as medical devices. These regulations would require AI scribes to comply with safety and medical-device rules before being sold or advertised. The regulator may take targeted action against non-compliant companies. Some experts have called for advanced AI scribes to be regulated as medical devices due to the sensitive data they handle and their ability to make medical recommendations.¹
In the US, states are passing laws to regulate AI use in healthcare. For instance, California, Colorado, and Utah have enacted laws requiring healthcare providers to disclose the use of generative AI to patients. Colorado healthcare providers who deploy AI to make consequential decisions about patient care or the cost of care must demonstrate that their AI systems have been tested for unfair discrimination. Other states, such as Kentucky and Rhode Island, regulate the use of AI devices to perform eye exams.²
At the federal level, the Biden Administration has released an executive order entitled "Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence," which prioritizes the responsible development and use of AI in healthcare. The order requires federal agencies to designate a Chief AI Officer and follow minimum risk-mitigation practices when using "rights-impacting and safety-impacting" AI. The Department of Health and Human Services (HHS) has also released guidance on AI in healthcare, emphasizing the need for transparency, fairness, and accountability.³ ⁴
As AI continues to reshape healthcare delivery, hospitals and health systems must take proactive steps to navigate the legal landscape. This includes monitoring class action litigation and state-initiated litigation and legislative efforts to protect consumers against AI abuses in healthcare delivery. Boards play a key role in assuring this vigilance and prioritizing integrated implementation of AI-focused compliance and risk management.
