Introduction
In more detail
Description of modification
Modification protocol
Impact assessment
Authorised PCCPs
Comment


Introduction

On 3 April 2023, the Food and Drug Administration (FDA) issued a draft guidance for industry,(1) for manufacturers whose medical devices use machine learning (ML) technologies to improve patient care. Through the draft guidance, the FDA intends to provide a least burdensome approach to support iterative improvements to machine learning-enabled device software functions (ML-DSFs) through modifications, while continuing to provide a reasonable assurance that the device is safe and effective. In essence, manufacturers can proactively seek the FDA's concurrence with the intended modifications to the applicable devices without additional marketing submissions for each modification in the future.

The draft guidance follows the FDA's April 2019 publication of a Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device, which delineates the FDA's general framework for a potential approach to premarket review for AI/ML driven software modifications. Specifically, this draft guidance proposes recommendations on the information to be included in the predetermined change control plan (PCCP) in a marketing submission for a device that is or includes an ML-DSF, including:

  • the planned modifications;
  • the methodology to develop, implement and validate the modifications; and
  • an assessment of the impact of those modifications.

By including a PCCP in the marketing submission, manufacturers can potentially implement certain modifications to an ML-DSF that generally would otherwise require additional marketing submissions prior to implementation.

Stakeholders may submit comments regarding the proposed framework to the FDA until 3 July 2023.

In more detail

ML can allow software to learn through data, and to support the continuous improvement of the software through ML. In the draft guidance, the FDA described a PCCP that can be included in a marketing submission. A PCCP is essentially a plan that includes device modifications which would otherwise require:

  • a premarket approval supplement;
  • de novo submission; or
  • a new premarket notification.

Section 515C of the Federal Food, Drug, and Cosmetic (FD&C) Act provides the FDA with express authority to approve or clear PCCPs for devices requiring premarket approval or notification. As such, by including a PCCP in a marketing submission for a medical device, manufacturers can be proactive in pre-specifying and seeking premarket authorisation for intended modifications to an ML-DSF without the need for additional marketing submissions.

The guidance describes the FDA's recommendations for the content of the PCCP, including that the PCCP should generally include:

  • a description of the modification – a detailed description of the specific, planned device modifications;
  • the modification protocol – the associated methodology to develop, validate and implement those modifications in a manner that ensures the continued safety and effectiveness of the device across relevant patient populations; and
  • an impact assessment – to describe the assessment of the benefits and risks of the planned modifications and risk mitigations.

The PCCP should be included as a standalone section within the marketing submission and prominently included and discussed in the cover letter. The PCCP should also appear in the table of contents as "Predetermined Change Control Plan" and be discussed as part of:

  • the device description;
  • labelling; and
  • relevant sections used for determining substantial equivalence or reasonable assurance of safety and effectiveness.

If information pertaining to the PCCP section is included outside of the PCCP section, it should be referenced within the PCCP section.

Description of modification

A description of each planned modification to an ML-DSF should be included in the description of modifications section of a PCCP. The detailed description should describe changes to the device characteristics and performance resulting from implementation of the modifications. The FDA recommends that a PCCP include only a limited number of modifications that:

  • are specific;
  • can be verified and validated; and
  • can be presented at a level of detail that permits understanding of the specific modifications that will be made to the ML-DSF.

Each modification should be linked to a specific performance evaluation activity within the modification protocol. The description of modifications should make clear whether the modifications will be implemented automatically or manually and whether they will be implemented in a uniform manner across all devices on the market or implemented differently depending on the different devices on the market.

Because modifications should be able to be verified and validated within the existing quality system of the device, the draft guidance makes clear that not all modifications are appropriate for inclusion in the PCCP. The types of modifications that may be acceptable include:

  • modifications relating to quantitative measures of ML-DSF performance specifications;
  • modifications relating to device inputs to the ML-DSF; and
  • limited modifications relating to the device's use and performance.

All modifications included in a PCCP must maintain the device within the device's intended use and indications for use.

Modification protocol

The modification protocol includes the documentation describing the methods that will be followed to develop, validate and implement the modifications delineated in the description of modifications section. It includes the verification and validation activities for the modifications and is intended to provide a step-by-step explanation as to how the modifications will be implemented to ensure the device remains safe and effective.

The documentation of modifications verified and validated per the modification protocol must be compliant with the quality system regulation. The draft guidance identifies four primary components of a modification protocol:

  • data management practices – to outline how the new data that was not used to develop the initial ML-DSF will be collected, annotated, curated, stored, retained, controlled and used by the manufacturer for each modification;
  • re-training practices – to identify the processing steps (the steps from the point the ML-DSF receives the input data to the point it provides an output) that are subject to change for each modification and the methods that will be employed to implement modifications to the ML-DSF;
  • performance evaluation protocols – to describe the processes that will be followed to validate the modified ML-DSF to ensure it will meet the specifications identified as part of a specific modification, while maintaining the specifications that are not part of, but may be impacted by, the modification; and
  • update procedures – to describe how the device will be updated to implement the modifications, provide appropriate transparency to users, and, if appropriate, provide updated user training about the modifications and perform real-world monitoring.

The FDA noted in the guidance that these four components generally provide the agency with what it needs to evaluate the PCCP. The PCCP should also include a description of any labelling changes that will result from the implementation of the modifications.

Impact assessment

An impact assessment is the documentation of the assessment of the benefits and risks of implementing a PCCP for an ML-DSF. It also includes information relating to the mitigation of the identified risks.

The manufacturer's existing quality system should be used as the framework to conduct an impact assessment for the modifications. According to the draft guidance, the impact assessment contained in a marketing submission containing a PCCP should:

  • compare the version of the device with each modification implemented to the version of the device without any modifications;
  • discuss the benefits and risks, including risks of social harm, of each modification;
  • discuss how the activities proposed within the modification protocol continue to reasonably ensure the safety and effectiveness of the device;
  • describe how the implementation of one modification impacts the implementation of another; and
  • discuss the collective impact of implementing all modifications.

Authorised PCCPs

A PCCP that has been reviewed and established through a device marketing authorisation is referred to as an "authorised PCCP". Because modifications made to an ML-DSF in accordance with an authorised PCCP were reviewed and authorised through the marketing submission by the FDA, the modifications can be implemented by the manufacturers without triggering the need for a new marketing submission. Deviations from the authorised PCCP, however, could significantly affect the safety or effectiveness of the device. In such a circumstance, continued distribution of the ML-DSF without a new marketing submission would constitute adulteration and misbranding under sections 501(f)(1)(B) and 502(o) of the FD&C Act, respectively.

The labelling for ML-DSFs with an authorised PCCP should explain that the device incorporates ML and has a PCCP so that users are aware that the device may require the user to perform software updates and that the updates may modify the device's performance, inputs or use. Once reviewed and established in the marketing authorisation process, a PCCP:

  • is part of the marketing authorisation;
  • should be evaluated within the existing risk management framework of the device; and
  • should be implemented in accordance with the manufacturer's quality system.

Comment

The increased use of ML-DSFs in recent years has shed light on the need for the FDA to develop a regulatory framework suitable to the adaptive and evolving capabilities of ML and AI technologies. The draft guidance represents the agency's most recent step in advancing its framework and its approach to impose a least burdensome method to support iterative improvements to these technologies in the medical device space.

Appendix A of the draft guidance includes informative examples of the elements of the modification protocol components for ML-DSFs. These examples may serve as important tools in drafting PCCPs and ensuring they contain the requisite information the agency needs to evaluate the PCCP.

In addition to the agency's focus on its approach to support iterative improvements to these technologies in the medical device space, it is important for entities pursuing use of these technologies in the medical device space to consider their use of data more broadly. There is an increasing complexity of data privacy and security laws and regulations applicable to such data, with numerous proposals for additional laws and regulations being issued regularly.

For further information on this topic please contact Adam Aft, Xin Tao or Genevieve Razick at Baker McKenzie by telephone (+1 202 452 7000​) or email ([email protected], [email protected] or [email protected]). The Baker McKenzie website can be accessed at www.bakermckenzie.com.

Endnotes

(1) Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions.