Earning Trust for AI in Health 2025

Healthcare expenditure has been rising faster than GDP over the past 20 years, with at least 20% deemed to be wasteful.1 At the same time, healthcare is facing a serious workforce crisis. The World Health Organization (WHO) estimates a deficit of 10 million health workers by 2030, particularly in low- and middle-income countries (LMICs).2 Healthcare workers are exhausted: approximately 50% of healthcare professionals suffer from burnout.3 In this context, artificial intelligence (AI) technologies bring significant opportunities to address health system crises . AI technologies are poised to fundamentally change how society organizes medical care, shifting critical tasks and augmenting health workers’ performance leading to improved patient outcomes4 and operational efficiency.5 Industry must be a responsible leader and visionary in the process of carving out spaces for AI technologies in the health sector. Industry leaders need to balance AI risks related to impact on patient safety and privacy (direct and often indirect, such as delayed diagnosis and treatment) with the need to advance innovation. In doing so, the private sector can contribute to the development of a positive public perception of AI technologies in order to earn the trust of the health sector. However, this process is likely to take place in a very challenging environment in which practices and policies struggle to keep up with the rapid pace and disruptive nature of AI innovations in healthcare: –AI is an emerging industry, with most players less than 10 years old, whereas healthcare is a mature industry dominated by established organizations. This mismatch risks slowing innovation – for example, as entrenched processes and structures may limit the adoption of new technologies. –The number of AI products is growing rapidly, with the global AI market estimated at almost $200 billion in 2023, a threefold increase from $62 billion in 2020.6 In contrast, the pharmaceutical market is characterized by a small number of products that require long and costly development. For instance, the United States Federal Drug Administration (FDA) approves an average of 47 drugs per year (2021–2023),7 with the average development cost and timeline per drug being $2.8 billion and 15 years, respectively.8 In 2020, the number of new AI technologies entering the health sector eclipsed that of new pharmaceuticals.9 –Deterministic and rule-based AI and machine learning (ML) models can be used to perform an array of tasks (e.g. image segmentation, classification and risk prediction) and are generally considered reproducible (even if not fully explainable), whereas probabilistic AI technologies, such as generative AI (GenAI), are:10 –Intended to create new data in a non- deterministic and dynamic way rather than identify patterns –Developed on (unstructured) datasets so large that developers cannot know everything about the data –Not created for an individual product, as foundational models are adapted for various applications Current evaluation processes predominantly focus on the safety, effectiveness and economic dimensions of healthcare innovations, covering products such as pharmaceuticals, medical technologies and deterministic software. The probabilistic nature of AI technologies results in some incompatibilities with existing processes. There is a need to adapt and modify the current frameworks to accommodate the unique characteristics of probabilistic AI technologies.11 The World Economic Forum’s Digital Healthcare Transformation (DHT) Initiative, in partnership with BCG, aims to bring a fresh perspective on how to build high-quality AI technologies that help build trust within the health sector. The initiative engaged with more than 50 experts in this field,12 who highlighted three areas that present urgent challenges: –Current health AI ecosystems are fragmented, with insufficient understanding of AI technologies in health from health leaders. –Stakeholders in the healthcare ecosystem must ensure that evaluation processes offer sufficient adaptability and flexibility to keep pace with the swift advance of AI technologies, while retaining high standards of evidence. –There is no global consensus on when public– private interactions are most vital to facilitate the development and deployment of high- quality AI technologies that earn the trust of the health sector. Throughout this journey, it is imperative to remain focused on the goal, which is to improve health outcomes for all. The path forward requires policy, systems and technological innovation stemming from public–private collaboration as well as a steadfast commitment to using technology for the improvement of healthcare and health systems. Earning Trust for AI in Health: A Collaborative Path Forward 6