On February 3, 2016, the Food and Drug Administration (FDA or the agency) released three guidance documents on human factors and usability engineering for medical devices. Applying Human Factors and Usability Engineering to Medical Devices1 is a final version of a 2011 draft guidance and will also supersede one of the agency’s earliest guidance documents on human factors and risk management.2 The other two newly released draft guidance documents address the types of devices most likely to require pre-market human factors testing3 and human factors considerations particular to the development of combination products.4 Together, the three new guidance documents reflect a recent FDA trend of placing greater emphasis on human factors (HF) considerations and assessments in medical device pre-market submissions and, at times, in the post-market setting.
The role of human factors in medical device development starts with the design control provisions of the Quality System Regulation, specifically 21 C.F.R. § 820.30(g), which requires design validation to “ensure that devices conform to defined user needs and intended uses.” Human factors analysis considers a device’s intended use, users, and use environment to improve the quality of the user interface (all points of interaction between the user and the device, and all sources of information transmitted by the device, labeling, training, physical controls, and display elements), such that errors occurring during device use are reduced. FDA believes that incorporating human factors testing and usability engineering in device design and development processes can simultaneously: (1) address potential safety problems before a device reaches the market, avoiding the need for costly measures later; and (2) lead to modifications that enhance device appeal and ease of use. The three recent guidance documents proceed from these core principles and provide additional agency insight around some of the nuances of HF testing.
Final Guidance on Applying Human Factors (HF) and Usability Engineering (UE) to Devices
FDA has noted that it received over 600 comments on the 2011 draft human factors guidance. Though the agency reports that the guidance was reformatted and restructured in response to the comments, the substantive content and overall framework remain largely consistent with the original draft.
The guidance emphasizes that human factors and usability should be considered both during the design development phase as well as during validation. Device firms should consider the particular characteristics of their device users, use environments, and user interfaces, and should employ both analytical and formative methods to identify, evaluate, and address associated use-related hazards to ensure safe and effective use. The final document serves primarily to clarify FDA expectations regarding particular testing methods, user populations for testing, training of test participants, appropriate sample sizes, reporting test results, and collecting HF data as part of a clinical study.
The guidance sets forth three steps to performing a successful HF/UE analysis:
- Identify anticipated use-related hazards and initially unanticipated use-related hazards, and determine how hazardous use situations occur;
- Develop and apply measures to eliminate or reduce use-related hazards that could result in harm to the patient or user; and
- Demonstrate whether the final device user interface design supports safe and effective use by conducting human factors validation testing.
The overall process should be iterative: outcomes of preliminary analyses inform modifications to device design and labeling; these are tested for a reduction in use-related hazards, potentially leading to further changes to the device; and finally the finished device is validated to demonstrate that all use-related hazards have been sufficiently controlled or eliminated.
Preliminary evaluations, which involve both analytical and empirical approaches, generate a list of critical tasks that users should perform correctly for use of the device to be safe and effective. It is helpful at this stage to identify any use-related problems that have occurred with devices similar to the one under development; these may be identified in customer complaint files, journal articles, or relevant websites such as FDA’s Manufacturer and User Facility Device Experience (MAUDE) database. Firms may also employ heuristic analysis, i.e. evaluation of a device’s user interface against applicable design principles, rules, or guidelines. Finally, data from user experiences interacting with the device or a prototype (e.g., interviews) provide additional information to support effective product development and modification.
Once the use-related device hazards have been fully identified, risk management strategies should be applied to eliminate or reduce the likelihood or severity of these hazards. FDA emphasizes that design modifications to the device and its user interface are the most effective means to achieve this end. Secondarily, manufacturers should implement protective measures (e.g., a low battery alert) and modifications to device labeling and/or training.
Finally, human factors validation testing must be conducted to demonstrate that the use-related hazards were successfully addressedand that no new hazards were introduced by the modifications. Such testing aims to show that the device can be used by the intended users without serious use errors or problems, for the intended uses and under the expected use conditions. Therefore, the validation test(s) should be structured as follows:
- Participants represent the intended (actual) users of the device;
- All critical tasks are performed;
- The device user interface represents the final design; and
- Test conditions are sufficiently realistic to represent actual conditions of use.
Testing is typically done under simulated use conditions, but human factors data can alternatively be collected under actual use conditions or as part of a clinical study. The appropriate sample size will depend on the purpose of the particular validation test and the device involved, but FDA recommends a minimum of 15 participants per distinct user population. Generally, test participants should also reside in the U.S. to ensure that the results are not skewed by different clinical practices that exist in other countries or language differences that change how labeling/training are understood. FDA encourages manufacturers to submit a draft protocol to the agency for feedback before conducting the testing.
The agency recommends that validation testing collect a combination of objective (i.e., observation) and subjective (i.e., participant opinion) data to generate a complete picture of any areas of weakness. The results should be carefully analyzed to determine the root cause of any use errors or problems that occurred during the test, which information will help determine the additional risk management measures that should be implemented. If results include use errors on critical tasks or participant feedback indicating difficulty with critical tasks, the company must act on this information, applying further risk management strategies and then re-testing to demonstrate that the modifications reduced the risks to acceptable levels. Manufacturers must also demonstrate that any significant residual risk is either impossible or impractical to reduce, and that the benefits of the final device outweigh these risks.
Labeling and training play an important role in mitigating human factors risks. In addition to the final guidance document, the agency specifically recognized this in a September 2015 public workshop on medical device patient labeling, where FDA commented that device labeling is part of the user interface and discussed (among other topics) how to design labeling to make it more accessible to the intended user population, as well as iterative testing with small representative user groups to identify problems. Once labeling changes are made, follow-up studies should be conducted to confirm their effectiveness. Nevertheless, the new guidance recommends that design changes that directly address risks be implemented first, as the effectiveness of modifying labeling or training is dependent on the labeling being available during device use and the user accurately remembering knowledge gained during training.
The final marketing submission should include a report describing the human factors testing conducted. Appendix A of the guidance outlines a sample report.
FDA will hold a webinar on the final guidance on February 19, 2016. Information is available at http://www.fda.gov/
Draft Guidance on Highest-Priority Devices for Human Factors Testing
In addition to finalizing the guidance on human factors design and validation, FDA also released a draft guidance describing the types of devices most likely to require human factors testing in a premarket submission. The 2011 draft guidance stated that a manufacturer should perform human factors testing if:
- the risk analysis indicated a moderate to high risk of use error;
- the device is being modified due to problems with use;
- a special control requires submission of human factors information;
- FDA guidance for a specific device type recommends such testing and the manufacturer cannot justify forgoing it; or
- FDA directly requests such testing.
FDA’s new draft guidance builds on these general guidelines, providing more concrete advice as to when human factors data must be included in a medical device premarket submission. Specifically, FDA identified a number of device types that, according to the agency, “have clear potential for serious harm resulting from use error” (e.g., duodenoscopes with elevator channels, insulin delivery systems, and robotic catheter manipulation systems). Human factors data should be included in premarket submissions for the listed devices unless the submission does not involve any changes to users, user tasks, user interface, or use environments from those of the predicates. It should be noted that FDA is likely to interpret changes triggering the need for human factors testing quite broadly, such that any differences in the look and feel of a user interface may require supporting human factors validation data. Human factors data may also be required to support clearance or approval of other device types on a case-by-case basis or as communicated through guidance documents or special controls, and such data should be submitted if the manufacturer’s risk analysis indicates that users performing tasks incorrectly or failing to perform tasks could result in serious harm.
Though the guidance was issued in draft form, in our experience it generally reflects the agency’s current thinking and mirrors recent requests we have seen for human factors testing with these types of devices. For example, approval of a PMA for an insulin pump with continuous glucose monitoring has typically required human factors testing, and a review of the agency’s PMA database confirms that FDA continues to expect companies in this sphere to address usability risks before their products reach the market.
In addition, in July 2015, FDA hosted a public workshop on Robotic-Assisted Surgery Devices that focused largely on human factors issues. Specifically, the event addressed FDA’s expectation that human factors validation be conducted for this type of device to ensure the device is user-friendly, can be set up appropriately, can be operated properly and without fatigue by the designated users (e.g., surgeons), has a straightforward graphical user interface, and comes with proper training.
In addition to PMA-approved devices, FDA continues to commonly request human factors testing for devices reviewed through the de novopathway as well. For instance, de novo petitions for over-the-counter (OTC) aesthetics devices often include labeling and/or self-selection studies to show that intended users can understand the product’s labeling and correctly determine whether it is appropriate for them. Such studies may be accompanied by validation that representative users can correctly complete all critical tasks related to preparation and use of the device, and may be conducted either as part of, or separate from, a major clinical study. We have also seen HF testing required for other types of de novo devices (e.g., devices associated with IV administration) to confirm that device design has adequately addressed potential use error hazards that could result in patient harm.
While, ideally, all potential use errors would be addressed pre-market, sometimes unanticipated problems are discovered only after a device is released to the field. FDA may also request human factors evaluations to address safety issues identified during the post-market phase. For example, FDA recently ordered three major duodenoscope manufacturers to evaluate users performing reprocessing procedures after reports that diseases were being transmitted through improperly cleaned endoscopes.5
Public comments on the draft guidance will be accepted through May 3, 2016 at Docket No. FDA-2015-D-4599.
Draft Guidance on Human Factors and Related Considerations for Combination Products
Combination products, such as drug delivery tools combined with the associated therapeutic, often raise human factors considerations. In recognition of this, FDA has issued a new draft guidance describing the use of human factors principles in the context of developing combination products that include a device component. Depending on the specific combination product, there may be a greater potential for use errors or requirements for more complex testing; however, the ultimate goal (to achieve a product user interface that is safe and effective for use by the intended users, uses and environments) is consistent with the more general human factors guidance discussed above. The draft guidance proceeds through the same types of considerations as discussed in that guidance, such as:
- identifying critical tasks and use-related hazards associated with the product;
- eliminating risks inherent to the product design;
- applying additional risk management strategies (labeling modification, training) to mitigate remaining use-related hazards;
- conducting validation studies (most likely simulated, but perhaps with actual clinical use);
- evaluating identified use errors or problems to determine the root cause(s), the potential for harm or compromised medical treatment, and whether additional measures to eliminate or mitigate hazards are needed; and
- assessing any changes through a new validation study before submitting a marketing application.
The guidance identifies two groups of combination products for which human factors data will generally be expected: (1) products for use outside the health care environment or by laypersons (e.g., home-use products); and (2) products having a device constituent part for which human factors data should be submitted (i.e., the device component is included in FDA’s list of high-priority devices for HF testing or otherwise triggers a requirement for such testing; refer to the draft guidance discussed above). For all other combination products, the agency recommends completion of a risk analysis and assessment of the use-related risks to determine whether HF studies are necessary. These considerations are equally applicable in designing a new device as in modifying an existing device, and in preparing a premarket versus an investigational application.
As with standalone devices, the agency recommends an iterative process to analyze potential use errors and make the changes needed to address them. The guidance also notes that once results of human factors analysis are available, they must be taken into account in determining whether further improvement is needed. Lastly, the guidance explains that human factors validation studies should typically be conducted, on the finished combination product, prior to conducting major clinical studies.
Public comments on this draft guidance should be submitted to Docket No. FDA-2015-D-4848 by May 3, 2016.