Mitigating AI Product Liability Risk in Life Sciences: A Principle-Based Approach By Jaclyn Jaeger Wednesday, November 1, 2023
Medical devices enabled by artificial intelligence/machine-learning (AI/ML) software are transforming the life sciences and healthcare industries in many positive ways, but they also potentially create new product liability risks that regulators have only just begun to address. In the race between AI and digital health regulations, it’s critical that life sciences companies keep pace to mitigate the risks. “AI products are advancing at a much faster rate than the regulations that govern them,” said Erin Bosman, a partner at Morrison Foerster and founder and co-head of the firm’s AI Group. To date, the FDA has authorized more than 600 AI/MLenabled medical devices, with 171 new ones having just been approved in October, according to the FDA’s Center for Devices and Radiological Health (CDRH) database. Because AI/ML-driven devices continue to learn and develop new capabilities, “product designers cannot test each possible foreseeable use or misuse,” Bosman added. Moreover, there exists a large void of knowledge and research into how AI systems can fail—all factors that make AI/ML-driven devices difficult to uniformly regulate, she said. 28th Annual Conference on
DRUG&MEDICAL DEVICE L I T I G AT I O N
December 5–6, 2023 | NY Marriott Marquis, New York, NY
Network with over 400 attendees including the who’s who of the life sciences products liability defense bar.
LEARN MORE
FDA Guidance In the United States, regulators have begun with a principle-based approach. In April 2023, the Food and Drug Administration (FDA) published draft guidance proposing recommendations for what information to include in a pre-determined change control plan (PCCP) as part of a marketing submission for an AI/ML-enabled device. The draft guidance incorporates stakeholder feedback from a 2019 FDA discussion paper and represents next steps in the development of a regulatory framework for AI/ML-enabled devices, as described in the FDA’s January 2021 action plan. By way of background, the FDA first introduced the PCCP framework in the 2019 discussion paper as an optional mechanism for AI/ML-enabled device makers when submitting a plan for modifications during the initial premarket review stage. The Food, Drug, and Cosmetic Act was later amended with the addition of section 515C, providing the FDA express authority to approve or clear PCCPs for devices requiring premarket approval or premarket notification. Specifically, Section 515C provides that a new marketing submission for specified, planned modifications to a device are not required and, thus, do not require rereview by the FDA, so long as the modifications are within the scope of the originally authorized PCCP. The draft guidance brings further clarity. “To address the data-driven nature of ML-enabled devices, which lends itself to rapid technological improvement over time, the PCCP mechanism described in this guidance provides a way for developers to pre-specify specific, planned device modifications and how those modifications will be implemented,” Matthew Diamond, chief medical officer of the FDA’s Digital Health Center of Excellence, said in an FDA webinar. “Our aim is for this type of technological agility to be facilitated by the regulatory agility of the PCCP.” A new marketing submission will be needed when an authorized device has been significantly modified. In practical terms, that means life sciences companies should closely review any change that may be outside the scope of an authorized PCCP to determine whether a new marketing submission is needed. Because PCCPs are optional to use, device makers “may continue to implement device changes via additional premarket submissions to FDA, as applicable,” Diamond said.
1/3
PCCP guiding principles The general recommendations discussed in the draft guidance for what information to include within a PCCP as part of a marketing submission for an AI/ML-enabled device fall under three main buckets, summarized below. Description of the modifications: When crafting proposed modifications, “remember to describe each one with a level of detail that helps the reader understand the specificity of the change and the impact of the change on the device’s safety and performance,” said Catherine Bahr, assistant director for Emerging Technology Assessment and Strategy in the FDA’s Digital Health Center of Excellence. Modification protocols: The modification protocol section describe the methods to be used for developing, validating, and implementing the modifications defined in the description of modifications. “The modification protocol section should include information tailored explicitly to every proposed change included in the PCCP,” Bahr said. This means, Bahr added, for each modification outlined in the description of modifications the modification protocol should address the four components outlined in the draft guidance: data management practices; retraining practices; performance evaluation protocols, including verification and validation plans; and procedures explaining how the devices will be updated to implement the modifications. Appendix A in the draft guidance provides specific recommendations for each of these four components, which is intended to further illustrate for AI/ML-enabled device makers the recommended level of specificity to include in this section of the PCCP. Thus, device makers may want to use it as a framework for their premarket submissions. The draft guidance further recommends clearly communicating to all users any modifications made. “The guidance proposes to place a significant and increased emphasis on the importance of transparency – in other words, the importance of clearly communicating valuable information about the device to stakeholders, including users,” Diamond said. Impact assessment: The impact assessment section should discuss how the planned modifications could impact the safety and effectiveness of the device and plans for mitigating those risks. “Basically, the impact assessment should help make clear how all these activities help to reasonably assure the device with any PCCP-defined modifications implemented will remain as safe and effective as the version of the device that will be reviewed at the time of the initial marketing submission,” said Mira Jacobs, biomedical engineer for Policy Leadership and Development at the FDA’s Digital Health Center of Excellence. “
International efforts The draft guidance is just one of several regulatory initiatives addressing AI/ML-enabled medical devices. On Oct. 24, the FDA, Health Canada, and the U.K. Medicines and Healthcare products Regulatory Agency (MHRA) published a joint guidance identifying five guiding principles that highlight where these agencies align in their expectations of what content to include in a PCCP. The five guiding principles closely align with the same core components described in the FDA’s draft guidance. In summary, the agencies recommend that a PCCP be: Focused and bounded: Specific, planned changes are described, limited to modifications within the intended use or intended purpose of the original AI/ML-enabled medical device. Risk-based: The intent, design, and implementation of a PCCP are driven by a risk-based approach that adheres to the principles of risk management throughout the total product lifecycle (TPLC) – from inception, through implementation and to use. Evidence-based: Evidence generated throughout the TPLC of the device ensures the device’s ongoing safety and effectiveness; demonstrates that the benefits outweigh the associated risks; and establishes that the risks are adequately managed and controlled. Transparent: Clear and appropriate information and detailed plans are provided for ongoing transparency to users and other stakeholders.
2/3
The fifth principle speaks to elevating the “quality and integrity of a PCCP by continually considering the perspectives of all stakeholders, as well as risk management practices throughout the TPLC; and use and support existing regulatory, quality, and risk management measures throughout the TPLC to ensure device safety by monitoring, reporting and responding to safety concerns.” This guidance is intended to complement each country’s own version of recommended PCCP principles. In addition to the FDA, Health Canada also recently published draft pre-market guidance for AI/ML-enabled devised. The MHRA will be publishing its guidance in 2024. The five guiding principles draw upon another joint guidance issued by the FDA, Health Canada, and the MHRA in October 2021, which identified 10 guiding principles for Good Machine Learning Practice (GMLP). The key principles discussed in that guidance include: • Practicing good data collection and data analysis protocols; • Leveraging multi-disciplinary expertise throughout the TPLC;
• Focusing on the performance of the human-AI team; • Providing users with clear, essential information; and • Monitoring deployed models for performance and managing re-training risks.
• Implementing good software engineering and security practices; In addition to regulatory developments happening in North America and the United Kingdom, the European Union’s Artificial Intelligence Act will be the first law in the world to regulate AI technologies. As regulators around the world begin to develop AI-related regulations and guidance, it will be prudent for life sciences companies to stay on top of these developments to help mitigate product liability risks unique to the rapidly evolving digital health landscape. ACI will be holding its 28th annual conference on “Drug & Medical Device Litigation” on Dec. 5-6 in New York. For more information, and to register, please visit: https://www.americanconference.com/drug-medical-device-litigation/ 28th Annual Conference on
DRUG&MEDICAL DEVICE L I T I G AT I O N
December 5–6, 2023 | NY Marriott Marquis, New York, NY
Network with over 400 attendees including the who’s who of the life sciences products liability defense bar.
For questions, concerns or more information about ACI Insights, please contact: Chris Corbin Associate Director of Marketing American Conference Institute | The Canadian Institute | C5 E: c.corbin@americanconference.com
3/3
LEARN MORE