Earning Trust for AI in Health 2025

Regulatory frameworks for AI are crystallizing around the world, with countries proposing the first generation of AI-specific legal frameworks, especially in the Global North: –The United States now prioritizes national competitiveness and economic strength, favouring policies to foster innovation. This is based on the hypothesis that limiting federal oversight will promote innovation and the development of a skilled workforce in the private sector. However, technical assessors will likely remain important for assessing AI as a medical device. –In contrast, the European Union has enacted the Artificial Intelligence Act (EU AI Act), which was adopted by the EU Parliament in March 2024. This comprehensive legislation categorizes AI systems according to risk levels and applies proportional control on high-risk applications. Within the healthcare sector, AI technologies are also subject to other regulations, such as the Medical Device Regulation, In Vitro Diagnostic Regulation, General Data Protection Regulation and European Health Data Space Regulation.13 The resulting framework provides comprehensive legislative coverage for AI technologies in the health sector, though the fragmented nature of this regulatory framework may result in legal inconsistencies. –Other jurisdictions in Organisation of Economic Co-operation and Development (OECD) countries – such as Canada, Japan, South Korea, the United Kingdom and Australia – are advancing their own AI regulations, many of which align closely with the EU’s norms and values on safety, privacy and accountability. –In contrast, regulation in the Global South is fragmented and often under-resourced, resulting in significant governance gaps. However, some nations are proactively developing AI regulations that are adapted to their unique socioeconomic, cultural and technological circumstances. These divergences are creating friction in the deployment of AI-based health technologies across countries and regions, especially for multinational companies that must navigate multiple legislative environments. Greater international harmonization of regulatory approaches could help reduce such barriers.141.1 Global divergences challenge the scaling of AI in health The private sector should play a pivotal role, building high-quality AI systems capable of operating effectively under diverse global regulations and addressing the unique risks associated with AI in healthcare in order to maintain trust in the health sector. Healthcare industry players and institutions developing and using AI systems face greater scrutiny and hesitancy to change compared to other industries deciding to accept AI systems15 (both deterministic and non-deterministic), requiring information on consistency, reproducibility, biases in data (such as demographic disparities), unintended AI responses (i.e. “hallucinations”), data privacy,16 opacity and potential for technology misuse. These requirements all help to ensure that AI technologies can be safely, securely and equitably deployed within the health sector. To help the development of high-quality AI technologies conducive to building trust, the OECD categorized three types of tool for trustworthy AI: procedural, technical and educational.17 Based on this classification, the authors of this paper studied the tools companies can develop depending on their own context and jurisdictions: –Procedural tools: This includes the development of rigorous evaluation and evidence generation processes or robust risk detection mechanisms that are built on outcomes that matter to patients, healthcare professionals and health systems. –Technical tools: Technical tools deal with issues related to use of AI such as transparency, detecting bias and how explainable AI systems are. It notably includes life-cycle or data-management tools, with meticulous management of data sources, systematic classification and tracking of data lineage and ensuring metadata completeness.1.2 The private sector is key to driving progress and standardization Earning Trust for AI in Health: A Collaborative Path Forward 8