On March 28, 2012, FDA issued a final guidance entitled, “Factors to Consider When Making Benefit-Risk Determinations in Medical Device Premarket Approval and De Novo Classifications.”i The guidance outlines the systematic approach FDA device reviewers take when making benefitrisk determinations during the premarket review process, explains the various principal factors considered by the agency during the review of PMA applications and de novo petitions, and describes an approach that takes into account patients’ tolerance for risks and perspectives on benefits, as well as the novelty of the device.

  1. Background and Scope of the Guidance

The final guidance applies to premarket approval applications (PMAs), the scientific and regulatory pathway to evaluate the safety and effectiveness of Class III medical devices, and de novo petitions, the regulatory pathway available for novel, low- to moderate-risk devices that have been found not substantially equivalent (NSE) to existing devices. When evaluating PMA applications or de novo petitions, the FDA relies upon valid scientific evidence to assess safety and effectiveness. Among other factors, FDA weighs the probable benefit to health from the use of the device against any probable risk of injury or illness from such use. The guidance states that “in general, a ‘probable risk’ and ‘probable benefit’ do not include theoretical risks and benefits, and instead are ones whose existence and characteristics are supported by valid scientific evidence.” Both clinical and non-clinical data play a role in FDA’s benefit-risk determinations. Because safety and effectiveness data alone may not provide a complete picture of the benefits and risks, FDA medical device reviewers objectively look at other factors such as the severity of the disease the product diagnoses or treats and whether or not alternative tests or treatments are available. FDA also may consider whether the device is new or a first-of-a-kind technology as part of the benefit-risk determination, particularly if the device treats a disease that has no other treatment.

FDA states that the concepts and risk-benefit factors discussed in the guidance are applicable to all medical device development phases from design to market, and thus should be considered during design, non-clinical testing, pre-IDE, and IDE phases as well as in the preparation of PMA applications and de novo petitions. The guidance applies to diagnostic as well as therapeutic devices. According to FDA, the recommendations made in the guidance are intended to improve the predictability, consistency, and transparency of the premarket review process for applicable devices, and should help manufacturers navigate the approval process more easily.

Unlike the draft guidance issued in August 2011 the final guidance’s benefit-risk determinations are not applicable to 510(k) notifications in any cases; the final guidance applies only to PMAs and de novo petitions. The agency is silent regarding whether it intends to provide separate guidance on factors to consider when making benefit-risk determinations in deciding substantial equivalence for 510(k)s. Another important difference between the draft and final guidance is that the final guidance significantly expands the discussion of patient tolerance for risk as a factor in benefit-risk determinations. The discussion of patient-centric concepts and metrics appears to indicate that FDA is willing to approve a device even if only a minority of the intended patient population may accept the risk. Indeed, FDA notes that the concept of “unreasonable risk—which must be absent for approval—should be construed to mean a risk that no set of reasonable patients would be willing to endure to achieve a probable benefit.

  1. Assessment of Device Benefits

The guidance states that FDA assesses information provided in a PMA application or de novo petition concerning the extent of the probable benefit(s) by taking into account the following factors individually and in the aggregate:

  • Type of benefit(s) – Examples include the device’s impact on clinical management, patient health, and patient satisfaction in the target population. Endpoints denoting clinical benefit are usually measured directly, but in some cases may be demonstrated by use of validated surrogate endpoints. For diagnostics, a benefit may be assessed according to the public health impact of a particular device, due to its ability to identify a specific disease and therefore prevent its spread, predict future disease onset, provide earlier diagnosis of diseases, or identify patients more likely to respond to a given therapy.
  • Magnitude of the benefit(s) – Benefit may be assessed along a scale or according to specific endpoints or criteria (types of benefits), or by evaluating whether a pre-identified health threshold was achieved. The change in subjects’ condition or clinical management as measured on that scale, or as determined by an improvement or worsening of the endpoint, allows FDA to determine the magnitude of the benefit in subjects. Variation in the magnitude of the benefit across a population may also be considered.
  • Probability of the patient experiencing one or more benefit(s) – Based on the data provided, it is sometimes possible to predict whether one or more subgroups of patients may experience benefit(s), whereas other times this cannot be well predicted. If the subgroups can be identified, the device may be indicated for those subgroups. In addition, FDA considers magnitude and probability together when weighing benefits against risks. For example, a large benefit, even if experienced by a small population, may be significant enough to outweigh risks, whereas a small benefit may not, unless experienced by a large population of subjects.
  • Duration of effect(s) – Some treatments are curative, whereas some may need to be repeated frequently over the patient’s lifetime. To the extent that it is known, the duration of a treatment’s effect may directly influence how its benefit is defined. Treatments that must be repeated over time may introduce greater risk, or the benefit experienced may diminish each time the treatment is repeated.
  1. Assessment of Device Risks

Under the guidance, FDA assesses the extent of the probable risk(s)/harm(s) by taking into account the following factors individually and in the aggregate: 

  • Severity, types, number and rates of harmful events associated with the use of the device – FDA considers the totality of the harmful events associated with the device, including device-related serious adverse events, device-related non-serious adverse events, and procedure-related complications (i.e., harms to the patient that would not be included under serious or non-serious adverse events, and that do not directly result from use of the device). 
  • Probability of a harmful event (i.e., the proportion of the intended population that would be expected to experience a harmful event) − FDA considers whether an event occurs once or repeatedly in the measurement of probability.
  • Duration of harmful events – Some devices can cause temporary, minor harm, some can cause repeated but reversible harm, and others can cause permanent, debilitating injury. FDA considers the severity of the harm along with its duration.
  • Risk from false-positive or false-negative results for diagnostics – If a diagnostic device gives a falsepositive result, the patient might receive an unnecessary treatment and incur all the risks that accompany that treatment, or might be incorrectly diagnosed with a serious disease. If a diagnostic device gives a false-negative result, the patient might not receive an effective treatment or might not be diagnosed with the correct disease or condition.

FDA also considers the number of different types of harmful events that can potentially result from using the device and the severity of their aggregate effect. When multiple harmful events occur at once, they have a greater aggregate effect.

  1. Additional Factors in the Assessment of Device Probable Benefits and Risks

Under the guidance, FDA assesses the following other factors in the assessment of the probable benefits and risks of devices:

  • Uncertainty – FDA considers the degree of certainty of the benefits and risks of a device when making benefitrisk determinations. Factors such as poor design or poor conduct of clinical trials, or inadequate analysis of data, can render the outcomes of the study unreliable. As another example, it is sometimes difficult to distinguish between a real effect and a placebo effect in the absence of a trial design that is capable of blinding investigators and subjects.
  • Characterization of the disease – The treated or diagnosed condition, its clinical manifestation, how it affects the patients who have it, how and whether a diagnosed condition is treated, and the condition’s natural history and progression (i.e., does it get progressively better or worse for the patient and at what expected rate) are all important factors that FDA considers when characterizing a disease and determining benefits and risks.
  • Patient tolerance for risk and perspective on benefit – When making a benefit-risk determination at the time of approval or de novo classification, FDA recognizes that there may be reasonable patients who are willing to tolerate a very high level of risk to achieve a probable benefit, especially if that benefit results in an improvement in quality of life. Therefore, FDA considers evidence relating to patients’ perspective of what constitutes a meaningful benefit when determining if the device is effective, as some set of patients may value a benefit more than others. However, if the probable risks outweigh the probable benefits for all reasonable patients, FDA would consider use of such a device to be inherently unreasonable. FDA encourages any sponsor that is considering developing data concerning patient risk tolerance and other patient-centered metrics to have early interaction with the appropriate FDA review division.
  • Availability of alternative treatments or diagnostics – When making benefit-risk determinations, FDA considers whether other treatments or diagnostics, including non-device therapies, have been approved or cleared for the intended condition and patient population. When considering other therapies, FDA takes into account how effective they are; what known risks they pose; how they are used in current medical practice; their benefit-risk profiles; and how well available alternatives address the needs of patients and providers. For a device with a known benefit and a probability of high risk that treats a condition for which no alternative treatments are available, FDA would consider the risk to the patient of having no treatment if a device were not approved.
  • Risk mitigation – The use of mitigations, when appropriate, can minimize the probability of a harmful event. The guidance states that the most common form of risk mitigation is appropriate information within labeling (e.g., warnings, precautions, etc.), or to restrict the indication to a more limited use. Some harms can be mitigated through other forms of risk communication, including training and patient labeling. For in vitro diagnostics, risks may be mitigated by the use of complementary diagnostic tests.
  • Postmarket data – The use of devices in a real world setting can provide a greater understanding of risks and benefits. FDA may consider the collection of postmarket data as a way to clarify the magnitude and effect of mitigations or as a way to develop additional information regarding benefits or risks for certain device types or in specific patient populations when making a benefit-risk determination.
  • Novel technology addressing unmet medical need – In assessing benefit and risk, FDA considers whether a device represents or incorporates breakthrough technologies and addresses an unmet medical need. A device may address unmet medical need by providing a clinically meaningful advantage over existing technologies, providing a greater clinically meaningful benefit than existing therapy, posing less risk than existing therapy, or providing a treatment or means of diagnosis where no alternative is available. FDA may find a novel device to be reasonably safe and effective even though the applicant demonstrates a relatively small probable benefit. In addition, the development of innovative technology may provide additional future benefits to patients. If the benefits increase or the risks are reduced with subsequent iterations of the device, the expected level of safety and effectiveness may change, and later versions may offer significant advantages over the initial device. In these circumstances, in order to facilitate patient access to new devices important for public health and to encourage innovation, FDA may tolerate greater uncertainty in an assessment of benefit or risk than for most established technologies, particularly when providers and patients have limited alternatives available.
  1. Worksheets for Benefit-Risk Determinations

The guidance also includes a Worksheet for Benefit-Risk Determinations for medical device reviewers to use to document how they make benefit-risk determinations. The Worksheet lists all of the principal factors that influence benefit-risk determinations, i.e., type of benefit, magnitude of benefit, probability of the patient experiencing one or more benefits, duration of effects, device-related serious and non-serious adverse events and procedure-related complications, the probability of a harmful event, duration of harmful events, risk from false-positive or false-negative results for diagnostics, uncertainties due to the design, conduct, and analysis of the study, characterization of the disease, patient tolerance for risk, patient perspective on benefit including the value the patient places on treatment, availability of alternative treatments, and risk mitigation. Along with each factor, there are questions for reviewers to consider. On the Worksheet, reviewers also must provide a summary of the benefits, the risks, and the other factors and a conclusion of whether the probable benefits outweigh the probable risks.

According to the Agency, FDA will increase the transparency of the benefit-risk decision-making processes by describing the worksheet analysis in the publicly released PMA Summaries of Safety and Effectiveness Data and the decision summary review memos for de novo decisions.

* * *

On May 1, 2012 CDRH reviewers will begin applying the guidance to incoming PMA and de novo submissions and to submissions already under review. According to FDA, CDRH will train medical officers, review staff managers and device reviewers on the guidance to assure the guidance is applied consistently to submissions and petitions. FDA is also developing external training modules to help industry and device sponsors understand how CRDH will apply the guidance.

Written comments and suggestions on the final guidance may be submitted at any time for Agency consideration.