Key Takeaways
- Artificial intelligence(AI) in healthcare faces challenges and regulatory hurdles to adoption and longevity that do not occur in many other industries.
- Data privacy and security is of utmost importance in healthcare and safeguards must be used to protect patient information.
- Appropriate AI education, training, careful integration and continuous oversight are still required to protect the integrity of how we provide healthcare to patients.
AI is transforming industries worldwide, from finance to manufacturing to entertainment. However, AI implementation in healthcare presents a uniquely complex landscape that sets it apart from every other sector.
While an AI algorithm might suggest the wrong movie or an autonomous vehicle might take a suboptimal route, errors in healthcare AI can mean the difference between life and death. Given the inherent risks with medicine and patient care, the path to adoption and longevity of the technology within the industry differs compared to other sectors.
The Stakes in Healthcare are High
While AI is being implemented into healthcare even as we speak, adoption and widespread usage are slow as compared to other industries.
Every AI-generated output within healthcare has the potential to affect human life. AI-induced errors can lead to worsening patient illness or even death. Thus, any new technology, protocol, treatment, or process must be thoroughly studied and vetted to ensure that we are using the most accurate information to diagnose, treat, and address patient concerns.
Regulatory Hurdles
The healthcare industry operates under stringent regulatory frameworks.
Approval: The Food and Drug Administration (FDA) has issued a comprehensive draft guidance for AI-enabled medical devices and established transparency principles for machine learning-enabled medical devices, creating multiple regulatory layers that don’t exist in most other sectors. Meeting guidance and standards and ultimately obtaining FDA approval can take months or years.
Surveillance: Once deployed, healthcare AI systems require ongoing monitoring and reporting to ensure that outputs are free from bias and continue to support positive patient outcomes. E.g., the European Medicines Agency published tools and guidelines that include monitoring of AI systems to ensure patient safety and data integrity.
Multi-Jurisdictional Complexity: Healthcare AI must also navigate hurdles in place by state medical boards, hospital accreditation bodies, and insurance regulations.
Together these layers create a regulatory maze that’s far more complex and time consuming to deal with versus the relatively straightforward requirements in industries like retail or entertainment.
Skepticism, Hallucinations, & Workflows
Skepticism from clinicians, who are trained to make patient care decisions based on scientific evidence and experience, toward AI is common and a difficult challenge to overcome.
Medicine operates on evidence-based practice principles that require extensive validation, approval and adoption by governing medical societies before accepting new tools. This cultural commitment to proven methodologies creates natural resistance to AI adoption that doesn’t exist in industries more comfortable with rapid iteration and experimentation.
In addition, clinical bias exists within healthcare; machine learning and AI models that are fed data containing these biases may further exacerbate inaccuracies, which further feeds the skepticism for the technology.
While students and currently practicing clinicians are slowly being exposed to AI use to improve their skills, part of their training is to learn the ethical implications of the technology and to understand associated patient safety concerns.
Hallucinations
AI hallucinations are instances where AI systems generate false or fabricated information, often due to incorrect or biased training data. These fabricated responses can appear authoritative and medically plausible, making them particularly dangerous as they can lead to misdiagnoses, improper or delayed treatments, and ultimately, patient harm.
It’s important that all clinicians trained to use AI are aware of its limitations, and double check the outputs.
This extra step has further created barriers to adoption. A bad decision made due to AI may not only erode clinician trust, but also patient safety and trust in the system.
Workflow Integration Challenges
Healthcare workflows can be incredibly complex, involving specialists and sometimes time-sensitive decision making. To ensure seamless integration of new technology into physician practices, hospital operations, or nursing workflows, without disrupting care or reducing efficiency, requires appropriate training with the technology.
Business alignment is also necessary. The HealthTech Investment Act includes plans to expand Medicare billing codes for FDA-authorized AI enabled devices or algorithms. Education and training on appropriate billing and coding is imperative, as is understanding the AI checks and balances that insurance companies are using to deny claims.
Legal and Business Challenges Unique to Healthcare
Professional Liability
Healthcare providers face personal malpractice liability for their decisions in ways that workers in other industries typically do not.
For physicians, it is not yet clear who bears the brunt of liability if AI makes a mistake. The liable party could be the physician using AI, or the company who created the technology. Clarity on this topic is needed especially in regards to AI that is utilized directly for patient care and treatment.
Insurance and Reimbursement
Companies in other industries that use AI may be able to directly measure their AI-generated revenues. However, healthcare operates on complex reimbursement models, and as of yet, they don’t easily accommodate AI-assisted care.
As mentioned above, the HealthTech Investment Act is bringing changes to reimbursement. AI-supported technology that falls within the realm of these reimbursements will soon face a tailwind for their innovations; however, as the technology becomes ubiquitous, these models will need to evolve to meet usage and demand.
Data Privacy and Security
While many industries handle sensitive data, healthcare data operates under particularly strict privacy regulations of the Health Insurance Portability and Accountability Act of 1996 (HIPAA).
Under HIPAA, clinicians and hospitals are constantly working to ensure that patient data is secure and private. Any breach of privacy is cause for consequence.
The risks of cybersecurity with AI supported digital technology is an ongoing concern. We’ve already seen the cybersecurity breach of Change Healthcare and how that affected clinicians and hospitals downstream. Cybersecurity challenges with AI are of a higher magnitude, necessitating appropriate vigilance and controls that can protect physicians, health systems and patients.
Mitigation Strategies and Best Practices
Despite these challenges, healthcare organizations are still successfully (albeit slowly) implementing AI by adopting industry-specific best practices:
Human-in-the-Loop: Maintaining human expertise is crucial, with most successful healthcare AI implementations requiring clinician oversight and approval for AI-generated recommendations. Further, many hospitals and hospital systems are employing AI governance committees for dedicated oversight to ensure integration and monitoring of any AI-based applications.
AI Transparency: Developing transparent AI models aids in identifying and rectifying hallucinations, which allows tracing back of erroneous predictions to specific data points. Healthcare organizations are prioritizing AI systems that can explain their reasoning.
Continuous Validation: To gain patient (and clinician) trust, continuous monitoring and evaluation of AI generated outputs, correction of errors, and updates to models to ensure they are still working, is required.
Enhanced Training and Education: Using AI in healthcare requires verification of each statement, reference, and recommendation generated, prompting the need for comprehensive training programs that exceed AI education needs in other industries.
Why “Slow” Breeds Longevity: Once Baked In, AI Will Likely Stay
When a health system is able to overcome the challenges listed above, and accepts an AI tool, it’s not going to swap it out overnight. The very challenges of getting something to work in healthcare, also produce challenges in getting out.
For instance:
- Integration into electronic health records (EHR) means integration into physician workflows, and possible integration into patient communication and documentation. Once integrated, and a behavioral change occurs, undoing it becomes a bigger hurdle.
- Regulatory frameworks in healthcare operate on multi-year timelines, so updating or changing something can’t get approval until the next cycle.
Looking Forward: The Future of Healthcare AI
AI technology will continue to evolve and become integrated into the industry, but it will always face hurdles and operate under constraints that don’t apply to other industries. FDA officials have cited the need for safe, effective, and trustworthy AI tools, involving both agency oversight and stakeholder collaboration, to adapt to AI’s unique challenges.
The future success of healthcare AI depends on acknowledging and addressing these fundamental differences rather than trying to apply approaches that work in other industries. Organizations that recognize the unique challenges will be better positioned to implement these powerful tools safely and effectively.
Healthcare AI isn’t just another technology implementation; it’s a fundamental shift in how medical care is delivered, requiring approaches as careful and evidence-based as medicine itself.
Sanjana Vig MD, MBA
Dr. Vig is a co-founder and Chief Marketing Officer of Langar Holdings. She is a board-certified anesthesiologist specializing in Perioperative Management. She is also the founder The Female Professional, a website geared toward empowering professional women in life and their careers.
