On April 12, 2018, the U.S. Food and Drug Administration (FDA or the Agency) finalized two guidances on the oversight of next generation sequencing (NGS)-based in vitro diagnostic tests. Unlike most IVDs that typically detect a limited number of predefined analytes for diagnosing pre specified conditions, NGS-based tests can be used to detect millions of DNA changes in a single patient sample in one test session. This information can, in turn, be used in a variety of ways, including as a companion diagnostic for a drug therapy. FDA has recognized the importance of developing appropriate regulatory frameworks for NGS-based tests to stimulate the advancement of the field of precision medicine. This effort by FDA also comes close on the heels of the recent CMS and FDA parallel review and subsequent National Coverage Determination for NGS tumor profiling tests.
The draft versions of these guidances were previously issued in 2016. Following extensive feedback from the public and stakeholders, FDA has made several significant changes in the final guidances, as discussed below. The guidances provide recommendations on the design, development, and validation for NGS-based tests, and demonstrate FDA's continued efforts in creating a more efficient path to market for developers of this innovative technology.
Use of public databases
The first guidance, entitled "Use of Public Human Genetic Variant Databases to Support Clinical Validity for Genetic and Genomic-Based In Vitro Diagnostics," describes an approach where test developers may rely on clinical evidence from FDA-recognized public genetic variant databases to demonstrate the relationship of these variants to a disease or medical condition and provide assurance of accurate clinical evaluation of test results.
Compared to the July 8, 2016, draft version of this guidance, an important change is that the final guidance broadens the scope of the guidance from NGS-based tests to genetic and genomic tests regardless of the underlying technologies (e.g., NGS, Sanger sequencing, or PCR). FDA indicates that publicly accessible genetic databases may be useful for establishing the clinical validity of NGS tests as well as single gene or panel tests using other technologies. Therefore, while the guidance was initially designed for NGS-based tests, the approach can be used for other genetic and genomic testing methods.
In addition, while FDA continues to believe that databases should follow an open-access model as the best practice, in the final guidance the Agency indicates that databases using licensing models and charging fees for commercial use may also fall within the scope of the guidance. FDA further notes that proprietary databases, although out of the scope of the guidance, can also be sources of valid scientific evidence to support the clinical validity of genetic or genomic-based tests. The recommendations in the guidance could be useful for test developers who rely upon these databases to prepare premarket submissions.
As in the draft guidance, FDA limits its scope to genetic variant databases that make assertions about human genetic variants. Specifically, "assertion" is defined in the guidance as "the informed assessment of a genotype-phenotype correlation (or lack thereof) given the current state of knowledge for a particular variant" and is the association of genotype-phenotype relationship to a disease or condition. According to FDA, the guidance does not apply to other types of databases, such as databases that direct therapies and databases used for microbial genome identification and detection of antimicrobial resistance and virulence markers. Further, the guidance is limited to curated databases using human evaluation and does not apply to databases using software interpretation.
The final guidance explains that if a genetic variant database conforms to FDA's recommendations, the evidence and assertions contained in the database would generally be deemed as valid scientific evidence that can be used by test developers to support the clinical validity of a new test. This approach is proposed to encourage public deposition of variant information, reduce regulatory burden on test developers, and stimulate advancements in precision medicine. To support FDA recognition of genetic variant database, the final guidance lays out the specific information to be included concerning database procedures and operations, data quality variant evaluations and assertions, and general guidelines on professional training and conflicts of interest concerning the genetic professionals curating the databases. For example, in terms of data quality, the metadata for different types of variants (e.g., germline variants, somatic variants) included in the database should have information concerning the analytical performance of the test used to detect variants and the characteristics of the independent sources reporting the genotype-phenotype relationship. For clinical relationship assertions, the types of evidence used for evaluating variants, and their corresponding strengths, should be provided in protocols. The guidance states that this information should come from multiple lines of scientific evidence supporting the "level of certainty and the nature of the genotype-phenotype relationship." The protocol should also be validated and publicly available. Further, databases should have mechanisms to receive feedback about individual variant assertions (e.g., new or contradictory evidence available regarding a variant assertion) and processes to document, evaluate, and resolve the discordances.
The guidance also lays out the recognition process for genetic variant databases. The database administrator can make a voluntary submission to FDA for recognition of the entirety or a subset of the genetic variant database. The submission should include SOPs, policies or other documents related to the recommendations in the guidance, validation data for evaluation SOPs, documentation for individual qualifications, data preservation plan, conflicts of interest, and a commitment to make all recommended documents publicly accessible via weblinks.
A proprietary database can remain confidential if it is submitted for FDA recognition. The database administrator should make the information publicly available at the time of recognition. After recognition, FDA may review the recognized databases annually, or more or less frequently as appropriate, to verify the compliance to SOPs and the continued transparency.
In addition, FDA indicates it may also consider using third parties to assist with database recognition. In the announcement, FDA highlights its experience in authorizing Memorial Sloan Kettering Cancer Center's MSK-IMPACT tumor profiling test, the clinical performance of which was evaluated by the New York State Department of Health (NYSDOH) based on a clinical evidence curation database (OncoKB).
NGS-based test to aid in the giagnosis of germline diseases
The second guidance, entitled "Considerations for Design, Development, and Analytical Validation of Next Generation Sequencing (NGS)Based In Vitro Diagnostics (IVDs) Intended to Aid in the Diagnosis of Suspected Germline Diseases," offers recommendations to developers of NGS-based tests on the design, development, and validation of tests used to diagnose individuals with suspected genetic diseases.
The guidance is intended to assist test developers and to inform the development of consensus standards by experts in the community. FDA indicated that once appropriate consensus standards for NGS-based tests intended to aid in the diagnosis of suspected germline diseases are developed by the community and recognized by FDA, test developers could certify conformity to such standards in a premarket submission, similar to how the standards recognition program has been used to meet premarket submission requirements for devices.
While there are no legally marketed NGS-based tests for general intended uses to aid in the diagnosis of suspected germline diseases, the guidance indicates that such tests may be appropriate for classification into class II through the de novo process. FDA believes that there is a reasonable possibility that the risks associated with such tests may be sufficiently mitigated by general and special controls. The subsequent 510(k) applications may be reviewed by accredited Third Party organizations. Ultimately, FDA hopes that conformance to robust FDA-recognized standards provide sufficient assurance of analytical validity such that, as long as there is also sufficient assurance of clinical validity, FDA can consider exempting these tests from premarket review altogether.
Similar to the July 8, 2016, draft version of the guidance, the scope of the final guidance is carefully crafted. According to FDA, the recommendations in the guidance are limited to NGSbased tests intended to aid clinicians in the diagnosis of symptomatic individuals with suspected germline diseases. It does not apply to NGS-based tests intended for aid in the diagnosis of microbial infection, cellfree DNA testing, direct-to-consumer testing, fetal testing, microbial genome identification and detection of antimicrobial resistance and virulence markers, preimplantation embryo testing, risk assessment, risk prediction, RNA sequencing, stand-alone diagnostic purposes, tumor genome sequencing, or use as a companion or complementary diagnostic.
The final guidance outlines FDA's recommendations for the development of standard(s), including test design considerations, performance characteristics, test run quality metrics, performance evaluations, supplemental procedures, variant annotation and filtering, labeling requirements, and test reports.
In the final guidance, FDA describes the Agency's expectations for the methods to assess accuracy, precision, limit of detection, analytical specificity, test run quality metrics, read depth, test run metric and performance thresholds for all critical NGS-based test steps, specimen quality, DNA quality, sequence generation/base calling, and mapping quality for these types of assays. Furthermore, in the final guidance FDA provides recommendations concerning performance evaluation studies, information to be included in test reports, and process for evaluating changes to an NGS-based test.
Notably, FDA removed minimum performance thresholds for specific metrics such as accuracy, precision, and coverage, which were provided in the draft guidance. Instead, FDA recommends that test developers predefine and report the minimum acceptable overall and target threshold metrics. FDA indicates that thresholds should be justified using objective evidence and valid statistical techniques, which should also be reported.
Another significant change compared to the draft version of the guidance is the additional information on calculating accuracy. In the final guidance, FDA provides a detailed method, as well as an illustrative example, to explain the Agency's expectations for accuracy determination. Specifically, accuracy should be calculated for each variant type and for clinically relevant variants using well-characterized reference materials or agreed-upon samples with high confidence calls. Positive percent agreement (PPA, the number of true positives divided by the number of known variants), negative percent agreement (NPA, the number of true negatives divided by the number of wild-type results for the tested variants), and technical positive predicative value (TPPA, the number of true positives divided by the total number of positive results obtained by the test) should be determined, as defined in the guidance document.
Taken together, the two final guidances demonstrate FDA's intent to adapt the regulatory review to provide flexible approaches for genetic tests using the emerging NGS technology. FDA outlines recommendations for the recognition of analytical performance standards or genetic variant databases, which, once recognized, can be relied upon for analytical or clinical validities, respectively.
Finally, in both guidances, FDA encourages applicants to engage with the Agency using the presubmission process in the development of the test.