As part of a series of efforts1 by the U.S. Food and Drug Administration (FDA) to "think outside the box" when it comes to the evidence required to support regulatory decision-making, on July 27, 2016, the agency issued a draft guidance document entitled, "Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices."2 The guidance is intended to clarify when real-world evidence (RWE) can be used in FDA decision-making and reaffirms that RWE retains a place in the medical device regulatory landscape, and will undoubtedly be well-received by industry. While the draft guidance does not appear to implement significant, concrete policy changes, it does provide tangible recognition by FDA of the multitude of non-traditional data sources that may contribute to a meaningful understanding of medical device safety and effectiveness. The guidance sets forth a number of acceptance criteria for RWE; therefore, the ultimate impact of this development will depend on FDA's openness to accepting proposed RWE despite its limitations.
The Agency's guidance comes amid an increased focus by FDA on evaluating creative regulatory options. This past May, FDA Commissioner Robert Califf stated that leveraging real-world evidence to inform FDA decision-making is a top agency priority, and in June, the nonprofit Bipartisan Policy Center issued a report calling for FDA to develop formal guidance outlining the circumstances under which RWE could be used to support clinical trials, post-market commitments, and regulatory decision-making, as well as the types of real-world data (RWD), analytical methods, and levels of evidence that the agency would deem acceptable. Separately, a new proposal under the Medical Device User Fee Act (MDUFA) requests funding to add 15 full-time employees to build a framework for using RWE for premarket decision-making as part of a new medical device coordinating center. In light of the recent attention placed on this issue, Agency policy in this area is likely to continue to develop in the coming months and years.
FDA's willingness to consider real-world data appears to align with a general shift in the way these types of data are viewed by the larger scientific community. At the July 21, 2016 Advisory Panel meeting convened to review the Dexcom G5 Mobile continuous glucose monitoring system, there was general consensus that the sponsor's simulation model data (validated by limited clinical data) in support of the proposed dosing indication were inferior to results from actual patients in randomized clinical trials. Nevertheless, the panel ultimately voted overwhelmingly in support of approval, relying heavily on real-world data demonstrating that many patients are already using the device off-label for the proposed expanded indication with encouraging results. While panel decisions are not binding on FDA, they do tend to influence final regulatory actions. The weight given by this panel to RWD sources is an encouraging sign that these data are gaining traction in the broader scientific community.
II. Summary and Analysis of New Guidance on Real-World Evidence
The draft guidance is intended to clarify when real-world evidence may be useful in FDA decision-making within the bounds of existing regulatory standards, and to explain how FDA evaluates real-world data and when an investigational device exemption (IDE) may be needed to prospectively collect and use such data to determine a device's safety and effectiveness.
Importantly, data used to support device clearance or approval must constitute "valid scientific evidence (VSE)." In defining how RWE can meet that threshold, the guidance serves as an acknowledgment by FDA that VSE can and should be interpreted more broadly to include information gathered outside of prospective clinical trials.
The draft guidance recognizes that conducting large clinical trials can be challenging and indicates that proper analyses of RWD potentially could provide comparable information. FDA also recognizes that, in some cases, RWD can provide information from a wider patient population. The agency issued the guidance in the hope that appropriate use of RWD can help bring new technologies to market efficiently without sacrificing the controls needed to ensure continued product safety and effectiveness.
A. Introduction and Scope
FDA defines "real-world data" (RWD) as data collected from sources outside of traditional clinical trials, including registry studies, retrospective database studies, case reports, and routine public health surveillance. Such data could be derived, for example, from electronic systems used in healthcare delivery, contained within a medical device, and/or developed in tracking patient experiences during care. This definition is fairly broad but does not specifically address one source of data often collected by companies in support of device applications: commercial experience and studies conducted outside of the United States (OUS). In addition, the guidance frequently mentions electronic health records (EHRs) as the main source of RWD, but it should be noted it can come in many forms, including paper medical records and retrospectively generated CRFs. "Real-world evidence" (RWE) is defined in the guidance as the evidence derived from aggregation and analysis of RWD.
As noted above, the guidance acknowledges that, under certain circumstances, RWD/RWE "could constitute valid scientific evidence" or supplement clinical trial data to enhance FDA's understanding of a device's benefit-risk profile and performance. The limitations noted by FDA relate principally to the relevance and reliability of real-world data/evidence, such as considerations of sources of bias for data collected outside the clinical trial setting and variability in the nature and completeness of information available.
B. Source and Quality of RWD
The guidance expresses a preference for RWD that could be characterized as "clinical trial-like," that is, RWD possessing many of the characteristics of clinical trials in terms of study design, collection, and reliability, including verifiability of source data. Consistent with that, much of the discussion is focused on well-designed registry data; however, FDA does recognize that RWD may sometimes differ from clinical trials in measurable ways. For example, collection of RWD may not be designed prospectively, but such information may be useful if, at a minimum, there is a prospective analysis plan for the data. The guidance notes factors that FDA may consider in deciding whether to accept RWD, including the recognition of the RWD source, the level of detail associated with the data, and whether a data source has been used by others (e.g., publications or practice guidelines) for clinically meaningful uses such as determining outcomes-based quality assessments or validated predictive risk modeling.
FDA states in the guidance that the increased use of electronic data systems in U.S. healthcare is likely to be one of the most significant potential sources of RWD in the coming years. To address concerns about inconsistent quality, FDA recommends that collection and analysis of RWD be performed in ways that would mitigate bias and support identification of any association between device use and the outcome of interest, as it would be in a traditional clinical trial. The agency cautions that retrospective analysis of RWD alone is unlikely sufficient to support a regulatory decision, but RWD collection instruments and analysis methods could be used prospectively if properly designed. FDA also recognizes that where RWD is not of sufficient quality to independently support a regulatory decision, it may still provide a valuable contribution to the totality of evidence that FDA will consider in making its decision.
The guidance provides a detailed discussion of the specific criteria by which FDA will appraise RWD and RWE intended to be used to evaluate a regulatory issue. Primary among these are the relevance and reliability of the data. In brief, relevance refers to whether data adequately addresses the applicable regulatory question or requirement; it involves consideration of factors such as the representativeness of the data and its generalizability to the population being evaluated; breadth of use/recognition of the data source; and how well the data reflects patient experience. Data reliability refers principally to the fitness of a given data source and depends on how data were collected, whether the data are complete and adequate to answer the question at issue, and whether the collection/analysis processes sufficiently assure quality, integrity, and bias minimization. Thus, companies should determine how the RWD they would like to use can answer a specific regulatory question (e.g., regarding long-term treatment effects, serious adverse events, etc.). Appropriate standardization procedures should also be in place, and data should be presented in a recognized format including discussions of the methodology used to assess statistical significance and clinically relevant differences between groups.
C. Potential Uses of RWD/RWE in Regulatory Decision-Making
The guidance presents several examples of actual uses of real-world evidence for regulatory decision-making.
With respect to the pre-market setting, the guidance explains how RWD potentially can be used to support expansion of the indications for use of a cleared/approved device, particularly in cases where the clinically accepted use has expanded over time beyond what is specified in the labeling. Much of the Agency's focus appears to be on use of RWE as an input to clinical trial design (e.g., hypothesis generation) or as comparative data (e.g., concurrent control group or development of an objective performance criterion (OPC) or performance goal (PG)). The guidance also explains that RWE may be used to augment the information needed to support clearance or approval of a next generation of a marketed device. The agency recommends that companies planning to use RWE to meet medical device data requirements discuss their proposed approach with FDA through the pre-submission process. However, going one step further, it can be argued that RWE that is deemed to constitute VSE should be able to be used to support the initial clearance/approval of a medical device.
Before beginning any collection of RWD in support of a marketing application, companies will also have to determine whether an approved Investigational Device Exemption (IDE) is needed to authorize the proposed data collection. An IDE permits shipment of an "investigational" (i.e., not yet cleared or approved for the specific intended use) device for use in a clinical investigation. FDA explains that whether an approved IDE is required for collection of real-world data depends on whether that collection constitutes an "investigation" as defined in FDA regulations.3 Gathering RWD for the purpose of determining device safety and/or effectiveness is likely to trigger the IDE requirements,4 while data gathered for other purposes as part of the practice of medicine—particularly for already commercialized devices—may fall outside the scope of these regulations. Importantly, the agency notes that certain data collected on off-label uses of cleared or approved medical devices may not require an IDE if it is collected in the course of clinical practice and not for the purposes of supporting a marketing application to FDA. Separate from the IDE question, informed consent for the underlying data collection and other human subject protections may be required to make RWE acceptable to FDA. The guidance does not specifically address the question of when RWD can be used when collected without informed consent.
On the post-market side, data collected as part of standard medical care or to follow patients with a particular health condition can be used for ongoing device safety surveillance. Importantly, FDA discusses use of registries in place of prospective clinical trials for post-approval studies (PAS). This reflects our recent experience, where FDA has been considering a wider range of study designs and data collection efforts to meet post-approval data needs. The guidance states that in cases where FDA would typically have mandated long-term follow-up and a formal post-approval study (such as for class III permanent implants), the agency may instead approve the device on the condition that the manufacturer answer the regulatory questions using registry data. Appropriately designed patient registries may also be accepted for the purposes of fulfilling a Section 522 post-market surveillance study order. Of note, the guidance focuses on sponsor-independent registries, i.e., those conducted through medical societies. However, in our experience, FDA is willing to accept registries initiated by sponsors themselves, as long as issues of relevance, reliability, and bias are adequately addressed.
Finally, the guidance indicates that FDA intends to use RWE to monitor for safety signals for cleared or approved devices on the market. It also notes that in some circumstances, RWE can be used in lieu of submitting individual Medical Device Reports (MDRs), but these circumstances are not clear.
The draft guidance represents tangible recognition by FDA of the multitude of data sources that exist in developing supportive data for medical device development, validation and ongoing improvement, and expresses a willingness to collaborate with industry to use this data for medical advancement and patient benefit (e.g., to expedite access to devices for unmet needs) without forgoing established regulatory protections. Even while a substantial proportion of RWE is unlikely to meet the acceptance criteria set forth by FDA, the guidance thus reaffirms that RWE retains a place in the medical device regulatory landscape. If companies have particular RWE which they believe can answer a specific regulatory question, FDA will consider its suitability. The ultimate impact of this development will depend whether FDA is open to accepting proposed RWE despite its limitations or whether the agency interprets the identified limiting criteria too stringently to allow significant use of such data.
Interested parties may submit comments on the draft guidance until October 25, 2016, to docket number FDA-2016-D-2153.