News & Comment

AI holds the potential to transform healthcare, promising improvements in patient care. Yet, realizing this potential is hampered by over-reliance on limited datasets and a lack of transparency in validation processes. To overcome these obstacles, we advocate the creation of a detailed registry for AI algorithms. This registry would document the development, training, and validation of AI models, ensuring scientific integrity and transparency. Additionally, it would serve as a platform for peer review and ethical oversight. By bridging the gap between scientific validation and regulatory approval, such as by the FDA, we aim to enhance the integrity and trustworthiness of AI applications in healthcare.

Digital health technologies (DHTs) can transform neurological assessments, improving quality and continuity of care. In the United States, the Food & Drug Administration (FDA) oversees the safety and efficacy of these technologies, employing a detailed regulatory process that classifies devices based on risk and requires rigorous review and post-market surveillance. Following FDA approval, DHTs enter the Current Procedural Terminology, Relative Value Scale Update Committee, and Centers for Medicare & Medicaid Services coding and valuation processes leading to coverage and payment decisions. DHT adoption is challenged by rapid technologic advancements, an inconsistent evidence base, marketing discrepancies, ambiguous coding guidance, and variable health insurance coverage. Regulators, policymakers, and payers will need to develop better methods to evaluate these promising technologies and guide their deployment. This includes striking a balance between patient safety and clinical effectiveness versus promotion of innovation, especially as DHTs increasingly incorporate artificial intelligence. Data validity, cybersecurity, risk management, societal, and ethical responsibilities should be addressed. Regulatory advances can support adoption of these promising tools by ensuring DHTs are safe, effective, accessible, and equitable.

Recent developments in large language models (LLMs) have unlocked opportunities for healthcare, from information synthesis to clinical decision support. These LLMs are not just capable of modeling language, but can also act as intelligent “agents” that interact with stakeholders in open-ended conversations and even influence clinical decision-making. Rather than relying on benchmarks that measure a model’s ability to process clinical data or answer standardized test questions, LLM agents can be modeled in high-fidelity simulations of clinical settings and should be assessed for their impact on clinical workflows. These evaluation frameworks, which we refer to as “Artificial Intelligence Structured Clinical Examinations” (“AI-SCE”), can draw from comparable technologies where machines operate with varying degrees of self-governance, such as self-driving cars, in dynamic environments with multiple stakeholders. Developing these robust, real-world clinical evaluations will be crucial towards deploying LLM agents in medical settings.

In the medical literature, promising results regarding accuracy of medical AI are presented as claims for its potential to increase efficiency. This elision of concepts is misleading and incorrect. First, the promise that AI will reduce human workload rests on a too narrow assessment of what constitutes workload in the first place. Human operators need new skills and deal with new responsibilities, these systems need an elaborate infrastructure and support system that all contribute to an increased amount of human work and short-term efficiency wins may become sources of long-term inefficiency. Second, for the realization of increased efficiency, the human-side of technology implementation is determinate. Human knowledge, competencies and trust can foster or undermine efficiency. We conclude that is important to remain conscious and critical about how we talk about expected benefits of AI, especially when referring to systemic changes based on single studies.

Historically, the Centers for Medicare and Medicaid Services (CMS) has formed partnerships with select private sector entities, including large traditional hospital and health system networks, nursing homes, and payer groups. However, innovations from technology-enabled services companies and digital technology companies are uniquely poised to aid CMS in addressing key barriers toward advancing its mission of improving healthcare access and equity. There are four pivotal opportunity areas where partnerships with technology businesses and tools would enhance the work of CMS: (1) improving consumer awareness about CMS programs, (2) mitigating access gaps through virtual care programs, (3) streamlining the complexity of different payer plan models, and (4) using technology-enabled services to address social risk factors without imposing additional burdens on providers. We offer examples of digital and technology-enabled solutions that improve patient access to care and close equity gaps, as well as propose specific recommendations for CMS to advance and expand the reach and impact of these solutions. Namely, these recommendations include partnerships with private sector companies that can educate and support consumers about their benefits, the extension of telehealth reimbursement parity for virtual care solutions, allowing for cross-state licensure across plans and reimbursement for care coordination services that alleviate provider burden to screen and address patients’ social determinants of health needs. We argue that CMS has an imperative role in leveraging the innovations of technology-enabled services and digital health technologies to lower healthcare access barriers, mitigate provider burden, stimulate innovation, and close equity gaps at the patient, provider, and innovator levels.

The integration of Quality Management System (QMS) principles into the life cycle of development, deployment, and utilization of machine learning (ML) and artificial intelligence (AI) technologies within healthcare settings holds the potential to close the AI translation gap by establishing a robust framework that accelerates the safe, ethical, and effective delivery of AI/ML in day-to-day patient care. Healthcare organizations (HCOs) can implement these principles effectively by embracing an enterprise QMS analogous to those in regulated industries. By establishing a QMS explicitly tailored to health AI technologies, HCOs can comply with evolving regulations and minimize redundancy and rework while aligning their internal governance practices with their steadfast commitment to scientific rigor and medical excellence.

As result of the pandemic-related increase in telehealth and the 21st Century Cures Act, technology is playing an increasing role in healthcare. This has led to organizational investments in the “digital front door” of healthcare. The promise that these technologies can revolutionize care by better connecting us to our patients, overcoming analog barriers to care, and addressing health disparities is grounded in “techno-optimism.” We advocate for organizational leaders to inform their digital health equity strategies with a balanced measure of “techno-skepticism”, grounded in implementation science, that can ensure alignment between health technology and health equity.

At a time of diminishing clinical skills and diagnostic endeavours for clinicians worldwide, a method of rapid automated microscopy coupled with pH measurement is introduced with early clinical success results in women with common vulvovaginal symptoms facilitating rapid diagnosis and enhanced therapeutic measures.

Sleep recordings are visually classified in stages by experts in the field, based on consensus international criteria. This procedure is expensive and time-consuming. Automatic sleep scoring systems have, progressively over the years, demonstrated good levels of accuracy. Although the performance of these algorithms is believed to be high, however, there remains widespread skepticism in their daily use in clinical and scientific practice. In this comment to a recent publication of NPJ Digital Medicine, we express the reasons why we think the sleep expert should remain the central pivot in the pendulum between visual and automatic methodology, trying to find a new balance in the scientific debate.

Digital health technologies (DHTs) enable remote data collection, support a patient-centric approach to drug development, and provide real-time data in real-world settings. With increasing use of DHTs in clinical care and development, we expect a growing body of evidence supporting use of DHTs to capture endpoint data in clinical trials. As the body of evidence grows, it will be critical to ensure that available prior work can be leveraged. We propose a framework to reuse analytical and clinical validation, as well as verification data, generated for existing DHTs. We apply real life case studies to illustrate our proposal aimed at leveraging prior work, while applying the V3 framework (verification, analytical validation, clinical validation) and avoiding duplication. Utilizing our framework will enable stakeholders to share best practices and consistent approaches to employing these tools in clinical studies, build on each other’s work, and ultimately accelerate evidence generation demonstrating the reproducibility and value add of these new tools.

Artificial Intelligence-supported digital applications (AI applications) are expected to transform radiology. However, providers need the motivation and incentives to adopt these technologies. For some radiology AI applications, the benefits of the application itself may sufficiently serve as the incentive. For others, payers may have to consider reimbursing the AI application separate from the cost of the underlying imaging studies. In such circumstances, it is important for payers to develop a clear set of criteria to decide which AI applications should be paid for separately. In this article, we propose a framework to help serve as a guide for payers aiming to establish such criteria and for technology vendors developing radiology AI applications. As a rule of thumb, we propose that radiology AI applications with a clinical utility must be reimbursed separately provided they have supporting evidence that the improved diagnostic performance leads to improved outcomes from a societal standpoint, or if such improved outcomes can reasonably be anticipated based on the clinical utility offered.

The generalizability of predictive algorithms is of key relevance to application in clinical practice. We provide an overview of three types of generalizability, based on existing literature: temporal, geographical, and domain generalizability. These generalizability types are linked to their associated goals, methodology, and stakeholders.

Digital tools are transforming mental health care. The promise of this transformation to improve outcomes has not yet been realized fully. While some have become skeptical, this article argues that we are just at the end of Act 1, with several opportunities and challenges ahead.

Postpartum mental health conditions are a public health concern, affecting a large number of reproductive-age women and their families. Postpartum depression alone affects at least 14% of new mothers and their families. However, very little has been written about how advances in digital mental health can benefit women in the postpartum period, or how those advances may poorly serve this vulnerable population. This manuscript takes a high-level view of the advances in different areas of digital mental health, including telehealth, apps, and digital phenotyping. In this comment, we explore ways in which digital interventions for postpartum mental health may help with connection to treatment, accessibility, agency, and ease of access. We also note particular concerns for how digital postpartum mental health may encounter issues of low-quality resources, ethical considerations, and equity considerations. We provide suggestions for how to leverage the promise and avoid the pitfalls of digital mental health for postpartum women.

This paper reviews the current state of patient safety and the application of artificial intelligence (AI) techniques to patient safety. This paper defines patient safety broadly, not just inpatient care but across the continuum of care, including diagnostic errors, misdiagnosis, adverse events, injuries, and measurement issues. It outlines the major current uses of AI in patient safety and the relative adoption of these techniques in hospitals and health systems. It also outlines some of the limitations of these AI systems and the challenges with evaluation of these systems. Finally, it outlines the importance of developing a proactive agenda for AI in healthcare that includes marked increased funding of research and evaluation in this area.

Digital technology is increasingly important in people’s lives, particularly for new parents as it allows them to access information, stay connected to peers and offers them seductive solutions for improving infant sleep and parental well-being. Digital technology has been developed to support parents in the following four ways: (1) providing digital information on infant sleep, (2) offering targeted support for night-time care, (3) managing infant sleep and (4) monitoring infant sleep and safety. Evidence on the effectiveness of these strategies is varied and there are concerns regarding the reliability of information, use of personal data, commercial exploitation of parents, and the effects of replacing caregiver presence with digital technology.

Artificial Intelligence (AI) in medicine has grown rapidly, yet few algorithms have been deployed. It is not the problem with the AI itself but with the way functions and results are communicated. Regulatory science provides the appropriate language and solutions to this problem for three reasons: First, there is value in the intentionally interdisciplinary regulatory language. Second, regulatory concepts are important for AI researchers because these concepts enable tackling of risk and safety concerns as well as understanding of recently proposed regulations in the US and Europe. Third, regulatory science is a scientific discipline that evaluates and challenges current regulation—aiming for evidence-based improvements. Knowledge of the regulatory language, concepts, and science should be regarded a core competency for communicating medical innovation. Regulatory grade communication will be the key to bringing medical AI from hype to standard of care. Foregoing the possible benefits of regulatory science as a unifying force for the realization of medical AI is a missed opportunity.