- On December 21, 2017, the RAND Corporation submitted to Congress its report on the federal bid protest system and its impacts on DoD contracting.
- RAND provided detailed quantitative analysis on protests filed with GAO and COFC, and we reported some of RAND’s related observations in a separate update on January 3, 2018.
- We provide here a summary of the RAND Corp. report’s recommendations. RAND’s recommendation for enhanced debriefings is consistent with reforms already adopted in the NDAA for FY 2018.
As we reported on January 3, 2018, the RAND Corporation submitted to Congress a detailed report in late December summarizing extensive research on the impacts of the federal bid protest system on Department of Defense (DoD) contracting. The RAND report, "Assessing Bid Protests of U.S. Department of Defense Procurements: Identifying Issues, Trends, and Drivers," was commissioned by Congress a year ago in Section 885 of the National Defense Authorization Act (NDAA) for FY 2017.
As we reported last year, the FY 2017 NDAA required DoD to contract with an independent research entity to carry out a comprehensive study on the prevalence and impact of bid protests on DoD acquisitions, including protests at contracting agencies, the Government Accountability Office (GAO) and the U.S. Court of Federal Claims (COFC). The report is now available on the RAND Corp. website.
RAND provided detailed quantitative analysis based on its review of data on protests filed with GAO and COFC, and on January 3, we reported on some of RAND’s most important observations, compiled from the report’s Summary Findings and Observations. The RAND report, which is more than 100 pages long, presents many detailed findings and observations on specific topics as called for under Section 885.
Here, we provide a separate summary of RAND Corp.’s recommendations for policymakers and DoD leadership. As stated in RAND’s Summary Findings and Observations, these recommendations are:
- Enhance the quality of post-award debriefings.
- Be careful in considering any potential reduction of the GAO decision timeline.
- Be careful in considering any restrictions on task-order bid protests at GAO.
- Consider implementing an expedited process for adjudicating bid protests for procurement contracts with values under $0.1 million.
- Consider approaches to reduce and improve protests from small businesses, such as improving debriefings, requiring protests to be filed by legal counsel, or providing legal assistance in filing.
- Consider collecting additional data and making other changes to bid protest records to facilitate future research and decisionmaking.
Recommendations From the RAND Report
RAND Corp.’s recommendations for policymakers and DoD leadership are presented in more detail at pages 69-73 of the report, and we discuss them in more detail below. According to the preface, “these recommendations are intended to inform future changes to the bid protest system.”
1. “Enhance Post-Award Debriefings”
The report reflects that the quality of post-award debriefings is a “major concern from the private sector.” As stated by the report, the poor quality of post-award debriefings is a frequent cause of bid protests:
The consensus among companies is that the quality and number of post-award debriefings vary significantly. The worst debriefings were characterized as being skimpy, adversarial, and evasive or as failing to provide required reasonable responses to relevant questions. In desperation, unsuccessful offerors may submit a bid protest to obtain government documents that delineate the rationale for contract award. The bottom line is that, in most cases, too little information and evasive/adversarial debriefings will lead to a bid protest.
The report noted the recent reforms adopted in Section 818 of the FY 2018 NDAA, “Enhanced Post-Award Debriefing Rights.” RAND’s conclusions are consistent with and support the reforms in Section 818. According to the RAND report,
The general expectation is that these provisions should substantially improve the quality and usefulness of post-award debriefings by providing additional transparency into the underlying DoD competition process. Disappointed bidders will gain greater understanding of the evaluation and award process and can better analyze any potential protest grounds before filing a protest.
As we reported on November 10, 2017, the newly adopted provisions in Section 818 require disclosure of a redacted source selection award determination for an award greater than $100 million, and for awards greater than $10 million, an option to request this disclosure for small businesses or nontraditional contractors. These provisions also provide enhanced opportunities for follow-up questions after a debriefing. We noted that “these provisions are based on the basic principle that better debriefings should improve communications and reduce protests,” and that it “remains to be seen how these procedures will be implemented by DoD agencies.”
Regardless of whether the reforms in Section 818 reduce the frequency of bid protests, they should, as discussed in our update on Section 818 and in the RAND report, improve the transparency of the underlying DoD competition, which is a critically important purpose and effect of the bid protest system.
2. “Be Careful in Considering Any Potential Reduction to GAO’s Decision Timeline”
RAND found that most GAO protest actions are resolved within 30 days, and that 70 percent of cases are resolved within 60 days. But RAND also found that “cases that go to decision (merit cases) typically take 90–100 days (which is the allotted time for GAO decisions). These cases are typically more complex and are not easily resolved.”
RAND discussed proposals (reflected in the pre-conference Senate version of the FY 2018 NDAA) to reduce GAO’s decision timeframe to 65 days. While the proposal was not adopted in the NDAA, it may be considered by Congress again. RAND expressed the concern that “shortening the timeline for GAO to close a protest might not leave enough time for it to develop these more complex decisions.” RAND concluded that “in all, the data suggest that it will not be easy to reduce the GAO timeline to 65 days for all protests.”
As we previously reported, GAO opposed this provision of the Senate version of the FY 2018 NDAA. The congressional compromise in the final post-conference NDAA reflects that RAND and GAO are correct on this issue, and that GAO and the parties to protests need, and should have, the full 100-day window to enable GAO to properly resolve all of the complex and most complicated cases that go to decisions.
3. “Be Careful in Considering Any Restrictions on Task‑Order Bid Protests at GAO”
The RAND report notes that the threshold for DoD task‑order protests has recently risen from $10 million to $25 million (a reform adopted in the NDAA for FY 2017). RAND’s quantitative study found that task-order protests account for approximately 10 percent of GAO protest actions, and that task-order protests “are generally more likely to be sustained or have corrective action compared with other types of protests.” RAND concluded that “this result suggests that task-order protests fill an important role in improving the fairness of DoD procurements,” and “[w]e recommend caution in considering any further restrictions on task-order bid protests.”
4. “Consider Implementing an Expedited Process for Adjudicating Bid Protests of Procurements Valued Under $0.1 Million”
RAND found that a significant number of protests at GAO and COFC concern procurements under $100,000:
A surprising result (at least to the authors) from the analysis is that roughly 8 percent of GAO protest actions and nearly 4 percent of protest cases at COFC concern procurements with a declared valued under $0.1 million. An interesting policy question is whether the costs to the government to adjudicate these protests exceed the value of the procurements themselves and thus are not cost-effective.
But RAND also recognized that “cost-effectiveness may not be the most important criterion in adjudicating bid protests. Transparency and fairness of government spending might be the overriding consideration, with cost-effectiveness being secondary.”
RAND recommended that “streamlined processes” be considered for protests under $100,000 (or another “suitably low value”)—“perhaps processes analogous to how traffic tickets are adjudicated in traffic court or how cases are adjudicated in small claims court,” which RAND suggested could include rulings from the bench at COFC, and require alternative dispute resolution at GAO.
5. “Consider Approaches to Reduce and Improve Protests from Small Businesses”
RAND found that more than half of the protests at GAO and COFC are from self-identified small businesses. RAND questioned whether this was the optimal allocation of resources for the bid protest system:
While small businesses are awarded more than half of DoD contracts, such contracts represent only 15‑20 percent of total contract dollars. This disparity raises another cost-effectiveness question: Should more than half of protest activity be focused on less than 20 percent of contract dollars?
As stated by RAND, the relative amount of protests by small business “suggests that any improvements to the bid protest system should also address small businesses.” RAND noted that the “loser-pays” pilot program adopted in Section 827 of the FY 2018 NDAA excludes businesses with annual revenues under $250 million. But RAND’s implied suggestion, that this program could be extended below that threshold, is unlikely to receive support from small businesses or the greater contractor community, which has already expressed concerns over the fairness of the pilot program. Small businesses are likely to defend their rights to protest and oppose any expansion of the pilot program, or limitation on their access to the protest system.
RAND further concluded that small businesses may protest more frequently due to poor quality of their debriefings:
Furthermore, the fact that small businesses are generally less successful at GAO (but not at COFC) suggests that small businesses’ reasons for protesting differ from larger businesses’ reasons (a feature that was corroborated in our discussions). Sometimes, when debriefings are uninformative, small businesses lodge protests to gain understanding of why they lost a procurement. To the extent that this is the case, the changes to the debriefing process discussed earlier should help to eliminate some small-business protests.
As noted above, Section 818 of the FY 2018 NDAA provides, for small businesses or nontraditional contractors and for awards greater than $10 million, the option to request a redacted source selection award determination, and also provides enhanced opportunities for follow-up questions after a debriefing. It remains to be seen how well these provisions will be implemented, and whether they will reduce protests by small businesses. But to the extent they improve transparency for small businesses, these reforms should improve the bid protest system. Any future reforms that further improve the debriefing system could benefit all participants, including large and small businesses.
RAND also noted that small businesses are more likely to have their cases dismissed for lack of jurisdiction or for being legally insufficient, and suggested options for better legal representation for small businesses (such as a requirement to file through counsel, or the provision of legal assistance through the Small Business Administration). While small businesses are likely to oppose any new barriers to their access to the bid protest system, they would likely welcome any increased assistance from SBA, as well as any further improvements to the debriefing process that could be adopted in future NDAA reforms.
6. “Consider Collecting Additional and Making Other Changes to Bid Protest Records”
RAND recommended several changes to data collection and reporting in order to improve future decisionmaking. RAND offered suggestions to GAO and COFC with the goal of improving available data. For DoD agencies, RAND recommended the following, focusing on data relating to agency-level protests and reasons for corrective action:
- Collect agency-level protest data and provide an annual report (similar to GAO’s) that summarizes protest activities (at a minimum, elements such as the number of cases filed, time to resolve a case, case outcomes, the reasons for sustained protests, accommodations made, procurement value, whether the protestor subsequently wins the contract, whether the protester is a small business, and if the protest subsequently appears at GAO or COFC.)
- Collect and summarize reasons for corrective action.
RAND’s suggestions reflect a relative lack of data relating to the agency-level protest system, and to agency decisions to take corrective action. An improved understanding of the agency-level protest system, and agency decisions to take corrective action, could potentially lead to more protests filed directly with agencies, instead of with GAO and COFC, resulting in both cost and time savings to agencies and protesters. An improved understanding of corrective action may also lead agencies to a better understanding of past mistakes, and help them avoid repeating those same mistakes in the future. With better information on agency‑level protests and corrective action, agencies can better understand aspects of the procurement process that they can improve in the future, which could in turn lead to fewer protests.
While protesters in many cases may doubt the agency’s ability to render a neutral decision on its own procurement process and decisions, an agency-level protest is sometimes the most effective means for a protester to bring certain issues directly to the agency’s attention, in the most cost effective manner for both the company and the agency. Agency-level protests are a vital and useful aspect of the bid protest system, and improved data collection will hopefully lead to improved utilization of these procedures, including by small businesses.
RAND’s Summary on Views of the Protest System
We reported on RAND Corp.’s observations based on its quantitative analysis on January 3, and the RAND report contains extensive detail and data that will illuminate future discussions on bid protest reform and current practices. RAND’s Summary Findings and Observations also included a summary of its qualitative analysis, which “found substantial differences between how DoD and the private sector view these issues.” Not surprisingly, DoD agency personnel primarily offered a critique of the current system:
In our discussions, DoD personnel expressed a general dissatisfaction with the current bid protest system. They believed that contractors have an unfair advantage in the contracting process in that they are able to impede timely awards with bid protests. These personnel also stated that the protest rules encouraged this behavior by allowing protesters to make excessive numbers of “weak” allegations, by permitting contractors too much time to protest, and by virtue of the amount of time it takes to resolve cases. In addition, there was a commonly held belief that a contractor is more likely to file a bid protest if it is an incumbent that has lost in a follow-on competition. . . .
RAND also found competing views on the other side:
These DoD views were contrasted with views expressed in our discussions with representatives from private-sector companies, trade associations, and private law firms regarding the impact of bid protests on their corporate decisionmaking. Overall, the private sector views bid protests as a healthy component of a transparent acquisition process, because these protests hold the government accountable and provide information on how the contract award or source selection was made. . . .
That said, a major private sector concern was the quality of post-award debriefings. . . . It became clear over the course of our study that too little information or debriefings that are evasive or adversarial may lead to a bid protest in most cases. The private sector also observed that the acquisition workforce was insufficiently staffed and could benefit from additional training. That workforce was cut massively in the 1990s and is still in the process of rebuilding.
While RAND highlighted these competing views in its Summary Findings and Observations, its report also revealed some areas of apparent consensus as well as additional areas of concern for the private sector and DoD, some suggestions for additional reforms advanced within DoD, and current efforts to improve debriefings and to improve dialogue with industry underway within DoD agencies.
As noted in our January 3 update, the RAND report’s findings and observations will likely lead to proposals for further reform of the bid protest system, following the recent reforms adopted in the FY 2018 NDAA. These recent reforms included the enhanced debriefings and bid protest pilot program that we summarized in our November 10, 2017 update.
It remains to be seen how these latest reforms will be implemented, and what future proposals may be advanced based on RAND’s findings, observations and recommendations. Since the adoption of reforms in the FY 2018 NDAA, the government contracting industry has expressed concern over the fairness of the bid protest pilot program. But the RAND report reflects support for improved debriefings within both industry and DoD. The enhanced debriefings adopted in the FY 2018 NDAA may be a starting point for future improvements to the bid protest system.