On April 26, 2012, in a ruling that had the potential to profoundly influence the use of predictive coding in the Southern District of New York, Judge Andrew L. Carter, Jr. instead took a measured and cautious approach. Judge Carter rejected plaintiffs' objections to Magistrate Judge Andrew J. Peck's much discussed endorsement of predictive coding in Da Silva Moore v. Publicis Groupe SA & MSL Group, No. 11 Civ. 1279 (ALC) (AJP) (S.D.N.Y. Feb. 24, 2012) ("Peck Opinion"), but did so on procedural grounds and stopped short of embracing predictive coding as a panacea for the mounting challenges of e-discovery.
Magistrate Judge Peck's ruling was the first to address, and endorse, predictive coding (also known as computer-assisted review). Predictive coding is an advanced technology and process used to manage the review of electronically stored information ("ESI"). It usually begins with an initial review of samples of ESI by lawyers with in-depth knowledge of the issues in the case to set baseline expectations for responsiveness and define a "seed set" of responsive documents. The seed set is used to train the predictive coding system. The system, using its built-in algorithms, analyzes the lawyers' decisions to predict how other, unreviewed documents should be coded. A key component of this process is its iterative approach in which the system offers lawyers additional sets of documents so that the lawyers can further "teach" the system and improve its predictive ability. As described by Judge Peck, this iterative process continues until the lawyers' review and the system's predictions "sufficiently coincide," Peck Opinion at 2, at which point "the system has learned enough to make confident predictions for the remaining documents." Id.
In this gender discrimination case, defendant MSL Group was eager to use predictive coding in an effort to reduce the burden and expense associated with reviewing three million documents for possible production. Plaintiffs, with very little to produce, were reluctant to agree to defendant's predictive coding protocol for fear that it would limit their access to discovery. After hearing from the parties and their e-discovery experts, Judge Peck favored the defendants' predictive coding methodology and ordered the parties to submit a joint protocol. Pursuant to the protocol, the parties are to engage in at least seven rounds of "training" the system to find responsive documents. After the parties reach agreement on how the initial seed set is to be coded (relevant, non-relevant, etc.), the computer will return another set of documents for lawyer review. In each round, the specific documents that comprise the seed set (excluding privileged ones) and the relevance coding applied by defendant's counsel must be provided to the plaintiffs for examination and feedback.
Although Judge Peck's order stated that plaintiffs had agreed to use predictive coding, plaintiffs argued before Judge Carter that they had not so agreed. On April 26, 2012, Judge Carter issued his Opinion and Order, Da Silva Moore v. Publicis Groupe SA & MSL Group, No. 11 Civ. 1279 (ALC) (AJP) (S.D.N.Y. April 26, 2012) ("Carter Opinion"). He noted that in objecting to Judge Peck's order and the protocol, plaintiffs argued, among other points, that (1) "the predictive coding method contemplated in the ESI protocol lacks generally accepted 2 reliability standards," Carter Opinion at 2, and (2) "that the use of such methods violates Fed. R. Civ. P. 26 and Federal Rule of Evidence 702[.]" Id. In addition, the plaintiffs filed a separate motion asking Judge Peck to recuse himself from the action based on his dealings with one of the lawyers for the defendants, his prior writings on the topic, and his participation on panels sponsored in part by vendors who market predictive coding software. That motion is still pending.
Judge Carter, relying in large part on the deference afforded to federal magistrates, rejected plaintiffs' challenge and adopted Judge Peck's order. Critical to Judge Carter's decision appears to be his view (also noted in Judge Peck's initial order) that many of plaintiffs' concerns about the reliability of the predictive coding software were premature and could be better dealt with as discovery advances. As Judge Carter observed:
If there is a concern with the relevance of the culled documents, the parties may raise the issue before Judge Peck before the final production. Further, upon receipt of the production, if Plaintiffs determine that they are missing relevant documents, they may revisit the issue of whether the [predictive coding] software is the best method. At this stage, there is insufficient evidence to conclude that the use of predictive coding software will deny Plaintiffs access to liberal discovery. . . . Id. at 3-4. If the method provided in the protocol does not work or if the sample size is indeed too small to properly apply the technology, the Court will not preclude Plaintiffs from receiving relevant information, but to call the method unreliable at this stage is speculative. Id. at 4.
Judge Carter's opinion is the latest in this new frontier of electronic discovery. Just last week, in Virginia state court, Judge James H. Chamblin ordered, over plaintiffs' objection, the use of predictive coding in Global Aerospace Inc. v. Landow Aviation, L.P. No. CL 61040 (Vir. Cir. Ct. Apr. 23, 2012). In another closely watched case, Kleen Products LLC v. Packaging Corporation of America et al., No. 10 C 5711 (N.D. Ill.), Magistrate Judge Nan Nolan is expected to issue a decision on whether defendants should be ordered, over their objection, to redo a document review using predictive coding.
Judge Carter's order – like that of Judge Chamblin in Virginia –provides support for the use of predictive coding in large volume ESI cases. Time will tell whether the predictive coding movement gathers steam. It is too soon to know whether disputes over relevance and coding decisions likely to arise between parties as part of predictive coding protocols like that in Da Silva Moore will make it hard for parties and courts to realize the potential time and expense savings that are behind the predictive coding movement.