Take Away: In Part 2 of our look at a recent publication from the Sedona Conference titled “Commentary on Achieving Quality in the E-Discovery Process”, we look at why there’s a need to measure the quality to determine whether your E-Discovery process has been reasonable.  

In Part 1 of our look at the Sedona Conference’s Commentary on Achieving Quality in the E-Discovery Process we looked at how the application of project management processes could help define and control the E-discovery process. In Part 2, we want to again look in detail at what the commentary has to say about measuring quality, and why this is vital to the success of your process.

In the first section of the commentary we notice that the Sedona Conference makes an effort to take the common business practices regarding project management and apply them to E-Discovery. In this section, they take another common business concept, quality measurement, and discuss why it’s important that we learn to measure the quality of our discovery efforts.

The commentary lays out four reasons why quality is important. Poorly done E-discovery processes can miss key evidence, could include privileged documents inadvertently, isn’t defensible, and can be more costly when you have to go back and do it again. All of these are valid reasons to be considering quality as you go through your E-discovery process.

Sedona also has a recommendation about quality control, versus quality assurance. In their view quality control, meaning that “quality” is engineered into the process up front, is superior, and more feasible in most cases, compared to quality assurance, which involves looking back over the process at its conclusion. In the context of litigation it is easy to see why taking steps to get it right up front makes much more sense than a post-hoc review that has only downside potential for the client. After the fact, CQI-type programs carry a special risk of being discovered and exploited by current and future litigation adversaries.

In a nutshell, the benefit of measuring quality while the discovery process is on-going is to enhance the defensibility of your processes, should they later be challenged, while informing lead counsel of corrective action that can be taken to mitigate any problems before they show up in a motion for sanctions . There should be, at minimum, someone checking to make sure your search results make sense, that the files you’re putting into a piece of technology to be reviewed are also coming out of that system for production, and that the reviewers and coders are doing what you expect as you review documents. There’s nothing worse than spending days, weeks or months with a team of reviewers only to find out that not all of them were using the same taxonomy to identify privileged documents, causing your identification of those documents to be incorrect.

As a practical matter, having quality control built in to the process allows you to catch problems when they happen, and correct for them, rather than having to redesign an entire process and start from scratch each time. That can be a significant savings of time and expense.

Sedona identifies five measures of quality:

  • Judgmental sampling
  • Independent testing
  • Reconciliation techniques
  • Inspection to verify and report discrepancies
  • Statistical sampling

Those with an accounting or engineering background are no doubt familiar with these sorts of measures and their role in measuring quality at varying points in a process. The commentary admits that there are a number of barriers to the adoption of these sorts of quality measures to E-Discovery. Differences in particular cases make a single-cookie cutter approach impractical for all cases, fueling confusion as to what a reliable quality plan must entail. “Reasonableness” remains the bearing point. Perfection is neither required nor economically feasible. A sensible, well-informed application of these quality measures commensurate with the needs of a particular case can help a party comply with their obligations.

You can download your own copy of the Commentary from the Sedona website.

In Part 3 we will look at specific ways the commentary talks about applying quality measures in E-discovery.