Global Partnership to Govern Generative AI in Medicine

Generative artificial intelligence (GenAI) models, such as generative adversarial networks (GANs) and transformer-based large language models (LLMs), are advancing rapidly and are poised to integrate into clinical workflows and healthcare systems worldwide. This surge in GenAI presents unprecedented opportunities, but it also brings systemic risks and critical vulnerabilities in safety, governance, and regulatory oversight.

GenAI and LLMs are non-deterministic, possess broad generalist functionalities, and display evolving capabilities. These characteristics challenge conventional regulatory frameworks designed for deterministic, task-specific artificial intelligence (AI) models, such as those for Software as a Medical Device (SaMD).

Some of the fundamental risks associated with GenAI and LLMs in healthcare include a lack of transparency in training data, susceptibility to bias, hallucination of incorrect medical content, and potential misuse in high-stakes clinical settings. Known unknowns include the possible use of synthetic data for training, which can introduce biases and inaccuracies. Unknown unknowns are risks and challenges that have yet to surface.

To address these issues, international organizations are forming partnerships to develop and implement robust governance frameworks. These frameworks aim to ensure that GenAI is used safely and ethically, protecting patient data and ensuring the accuracy and reliability of AI-generated content.

One of the key strategies is to enhance transparency in the training data used for GenAI models. This involves ensuring that the data is diverse, representative, and free from biases. Additionally, regulatory bodies are working on standards for testing and validating GenAI models to prevent the generation of incorrect or harmful medical advice.

Another critical aspect is the development of ethical guidelines and oversight mechanisms. These guidelines will help prevent the misuse of GenAI in clinical settings and ensure that the technology is used to benefit patient care and not for malicious purposes.

Collaborative efforts are also focusing on the continuous monitoring and updating of GenAI models to adapt to new challenges and emerging risks. This includes regular audits and assessments to ensure that the models remain safe and effective over time.

The integration of GenAI into healthcare is a complex and multifaceted issue that requires a coordinated global response. By working together, international partners can harness the potential of GenAI to revolutionize healthcare while mitigating the associated risks.

Clear guidelines and robust governance frameworks will be crucial in ensuring that GenAI is a force for good in the medical field. As GenAI continues to evolve, ongoing collaboration and innovation will be essential to maintain the balance between technological advancement and patient safety.