The latest discovery dispute in National Day Laborer Organizing Network v. U.S. Immigration & Customs Enforcement Agency (“National Day Laborer”) has resulted in Judge Shira A. Scheindlin of the Southern District of New York directing the government, once again, to supplement its response to a Freedom of Information Act (“FOIA”) request. __ F. Supp. 2d __, 2012 WL 2878130 (S.D.N.Y. July 13, 2012). As discussed in previous issues of this Update, the government has recently seen challenges to its FOIA responses based on the form of production and adequacy of searches, and the court has repeatedly held the government to standards approaching those required in civil discovery. See SDNY Ruling Requires Government to Produce Metadata in Response to FOIA Requests, Electronic Discovery Update (Apr. 2011); Update: SDNY Judge Scheindlin Withdraws Ruling Requiring Government to Produce Metadata in Response to FOIA Requests, Electronic Discovery Update (Nov. 2011); Judge Takes Another Bite at the FOIA Apple, Electronic Discovery Update (Apr. 2012), all available at http://www.kramerlevin.com.
The most recent National Day Laborer opinion scrutinizes the practice of self-collection by client-representatives, and provides some guidelines for parties considering executing their own electronic document searches and collections, suggesting some caution where parties are relying on keyword searches alone during discovery. While the decision concerns the FOIA statute and the government’s obligations arising thereunder, the decision may also be instructive with respect to civil e-discovery.
The core dispute in National Day Laborer concerns a FOIA request seeking information about Secure Communities, a program that enlists the help of states and localities in enforcing federal immigration law. National Day Laborer, 2012 WL 2878130, at *1. After two years of discovery disputes and the production of tens of thousands of records, the latest installment of National Day Laborer sees the parties cross-moving for summary judgment on the adequacy of the government’s searches pursuant to the FOIA statute. Id. As part of its effort to establish that the searches were adequate, the government submitted declarations describing which agencies and offices conducted searches and how employees searched for records. Id. at *2. Plaintiffs criticized the government for inadequately describing search methodology, failing to search certain custodians’ records, and conducting searches that were inadequate in and of themselves. Id.
Search Terms & Custodian Self-Collection
In order for the government to prevail on a summary judgment motion under the FOIA statute, “an agency must show beyond material doubt that it has conducted a search reasonably calculated to uncover all relevant documents.” Id. (citation omitted). Judge Scheindlin emphasized, “it bears repetition, the government will not be able to establish the adequacy of its FOIA searches if it does not record and report the search terms that it used, how it combined them, and whether it searched the full text of documents.” Id. at *11. In the past, a court could be satisfied without such specifics because it was sure that “searching parties were actually looking at the documents with their eyes.” Id. at *10. But, as Judge Scheindlin noted, “[t]hings have changed.” Id. When it comes to email, specifically, a custodian executing an electronic search typically only reviews documents retrieved by search terms and is not, as in the past, looking at the entire universe of potentially responsive documents. Id. Also, as Judge Scheindlin has previously opined, small mistakes (such as typos or spelling errors), differences in Boolean connectors, or search terms can yield dramatically different results. Id.
The decision also provides guidance concerning the persons who should conduct the search for responsive documents. Judge Scheindlin wrote that recent scholarship and case law show that “most custodians cannot be ‘trusted’ to run effective searches [of their e-mail] because designing legally sufficient electronic searches in the discovery or FOIA contexts is not part of their daily responsibilities.” Id. at *11. She continued: “Searching for an answer on Google (or Westlaw or Lexis) is very different from searching for all responsive documents in the FOIA or e-discovery context.” Id. Notably, these observations are said to extend to “the discovery . . . context” generally, even though the case concerned FOIA standards. Id.
Additionally, Judge Scheindlin stressed the importance of “careful thought, quality control, testing, and cooperation with opposing counsel in designing search terms,” but cautioned that “there is no guarantee that using keywords will always prove sufficient.” Id. at **11-12. In fact, research has shown that “the use of keywords without testing and refinement . . . will in fact not be reasonably calculated to uncover all responsive material.” Id. at *12.
Endorsement of Predictive Coding
Also of note, the decision added another influential voice of support to the use of predictive coding technologies in the discovery process. Such technologies, when trained by high-level reviewers using seed-sets, can retrieve responsive documents with potentially greater accuracy than human review alone, and with far more efficiency. See Courts Continue to Address Technology-Assisted Review and “Predictive Coding”, at 1 of this Update. Judge Scheindlin noted that “[t]here is increasingly strong evidence that [k]eyword search[ing] is not nearly as effective at identifying relevant information as many lawyers would like to believe.” National Day Laborer, 2012 WL 2878130, at *11(citation omitted). Though not contemplated by the parties in National Day Laborer, Judge Scheindlin suggested that “parties can (and frequently should) rely on latent semantic indexing, statistical probability models, and machine learning tools to find responsive documents . . . [which] can significantly increase the effectiveness and efficiency of searches.” Id. at *12 (footnote omitted).
Ultimately, the Court ordered the parties to agree on a list of search terms and methodologies so that the government could (1) search documents of previously omitted custodians and (2) run additional searches for a selection of previously searched custodians — as a device to test the adequacy of the government’s previous responses. Id. at *13-14. Notably, Judge Scheindlin did not bar self-collection by lay custodians nor mandate the use of predictive coding software in place of search terms, despite discussion on both issues.
National Day Laborer serves as another guide for litigants in the seemingly mine-ridden field of e-discovery. It provides thoughtful guidance — in the specific context of FOIA, but nevertheless making observations of broader application — relating to the use of search terms and self-collection of ESI by unsupervised lay custodians. It also adds to the growing discussion around the use of predictive coding technologies in place of search terms and other retrieval methods. Moreover, the entire National Day Laborer oeuvre highlights the potential benefits of cooperation during the discovery process. Here, the government might have avoided several rounds of motion practice, as well as the additional searches that were mandated, had there been more dialogue among the parties concerning custodians and search terms from the outset. In many cases it may be best for litigants and their e-discovery counsel to engage in a collaborative and iterative discovery process, consulting on search terms and methodologies as warranted, and documenting key decisions and other steps that may later need to be explained to an adversary or the court.
Summer Associates Daniel Lennard, David Mayo, Anna Schoenfelder, and Alexander Traum.