<\/p>\n
Katherine O\u2019Malley<\/em><\/span><\/p>\n Associate Editor <\/em><\/span><\/p>\n Loyola University Chicago School of Law, JD 2025<\/em><\/span><\/p>\n \u00a0<\/em><\/span><\/p>\n The capabilities of generative artificial intelligence (AI) could completely transform our healthcare system as we know it. For better or for worse, the technology advancements in healthcare are rapidly growing<\/a>. Given the accelerated rollout, experts have yet to predict all the risks associated with such high-functioning computations in the healthcare system. Even though the Food and Drug Administration (FDA) regulates software being used as medical devices (SaMD<\/a>), there is an overall lack of urgency, agency oversight, and sufficient regulations to tame AI technology in the healthcare system.\u00a0<\/span><\/p>\n What is Generative AI technology?<\/strong><\/span><\/p>\n Generative AI technology<\/a> generally refers to supercomputers that can manage complicated chatbot features replicating human-like interactions in the form of a search engine. AI is not limited to chatbots like \u201cChat GPT<\/a>.\u201d Models such as \u201cStable Diffusion, Synthesia, and MusicLM\u201d can produce images, video, or audio. This technology employs advanced machine-learning techniques that analyze patterns within their training data to create new content based on human-submitted prompts. Although significant advancements have been made recently in generative AI, this technology started in the 1950<\/a>\u2019s.<\/span><\/p>\n With every benefit comes a risk<\/strong><\/span><\/p>\n Generative AI can make everyday administrative tasks more efficient, such as emails, task management, and innovations. AI can analyze data to improve decision-making in all industries, especially in the healthcare sector<\/a>.<\/span><\/p>\n The effects could help people gain access to diagnostic treatment<\/a> by automating mundane work done by licensed professionals who could then spend more valuable time face-to-face with patients. AI algorithms<\/a> have outperformed radiologists when spotting malignant tumors. AI is also used to analyze population health<\/a> and predict populations at risk of certain diseases or readmission to hospitals. AI-assisted robotic surgeries<\/a> have even proved to decrease complications in procedures as compared to surgeons operating alone.<\/span><\/p>\n Even though AI algorithms outperform radiologists in one area, there is a risk of bias<\/a> due to a lack of data robustness. AI could also have detrimental effects concerning data breaches and violate HIPAA<\/a>. For example, if a doctor\u2019s office uses an AI chatbot feature for intakes and pre-diagnoses, that could violate HIPAA since the data has left the protected healthcare system<\/a>.<\/span><\/p>\n FDA\u2019s ongoing attempt to regulate AI in the healthcare system<\/strong><\/span><\/p>\n In January 2021, the FDA announced its AI\/Machine- Learning SaMD Action Plan<\/a> to protect against the risks posed to the healthcare sector. The FDA plans to coordinate this action plan with the agency\u2019s medical device cybersecurity program<\/a>. Furthermore, the FDA emphasizes a need for transparency with patients to tackle issues such as usability, equity, trust, and accountability. The plan encourages manufacturers of AI-based devices to be transparent. This action plan lacks measurable goals, step-by-step actions, or designated responsibilities. Overall, it\u2019s filled with vague and conclusory statements. For example, it states the following without explanation of how or who: \u201c[t]his will be accomplished in coordination with other ongoing FDA programs focused on the use of real-world data.\u201d<\/span><\/p>\n The FDA has released many draft recommendations since the January 2021 action plan. The first assesses the credibility of computational modeling and simulation in medical device submissions<\/a>. Another draft guidance<\/a> proposed a risk-based approach to computer software assurance directed towards manufacturers of AI\/ML-enabled products. All of which are drafts and used for comment purposes only.<\/span><\/p>\n Next, in September of 2022, the FDA released the final Clinical Decision Support (CDS<\/a>) Software guidance. This provides a framework for determining when CDS software functions are included and excluded from the definition of device under section 520(o)(1)(E) of the FD&C Act. In other words, the framework explains when a software function may or may not be exempt from FDA regulations.<\/span><\/p>\n In April of this year, the FDA published another draft guidance<\/a> on using AI in medical devices. This draft states that the FDA wants to provide the least burdensome approach to support innovation while simultaneously promoting reasonable assurance of device safety. This more lenient draft seems to be in response to the uproar<\/a> AI developers gave to the September 2022 final guidance.<\/span><\/p>\n Although the FDA must balance patient protection without impeding scientific innovation, human protection should take precedence. There are even several tech executives and top AI researchers calling for a six-month\u00a0pause in the development of AI tools.\u00a0The letter<\/a> from several tech executives and top AI researchers advocates for new and capable regulatory authorities dedicated solely to AI. In addition, they call for oversight from an unbiased third party and tracking of highly capable AI systems. Even the CEO of one of the most infamous AI systems, stated<\/a> in March that, \u201c[a]t some point, it may be important to get an independent review before starting to train future systems…\u201d<\/span><\/p>\n The point has been reached where government agencies, like the FDA, need to limit this technology as it has already overridden essential safety measures such as Captcha<\/a>. ChatGPT\u2019s CEO is now calling on Congress to create a new government agency<\/a> focusing solely on regulating AI technology. The development of AI generative systems should halt so the government can create a new agency to regulate AI in healthcare.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":" The capabilities of generative artificial intelligence (AI) could completely transform our healthcare system as we know it. For better or for worse, the technology advancements in healthcare are rapidly growing. Given the accelerated rollout, experts have yet to predict all the risks associated with such high-functioning computations in the healthcare system. Even though the Food and Drug Administration (FDA) regulates software being used as medical devices (SaMD), there is an overall lack of urgency, agency oversight, and sufficient regulations to tame AI technology in the healthcare system.\u00a0<\/p>\n","protected":false},"author":155,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34,56],"tags":[810,942,1014],"class_list":["post-5531","post","type-post","status-publish","format-standard","hentry","category-healthcare-compliance","category-hipaa-health-information","tag-fda","tag-generative-ai","tag-healthcare"],"_links":{"self":[{"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=\/wp\/v2\/posts\/5531","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=\/wp\/v2\/users\/155"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=5531"}],"version-history":[{"count":0,"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=\/wp\/v2\/posts\/5531\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=5531"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=5531"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.luc.edu\/compliance\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=5531"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}