Probability sampling is a well-established technique used to reduce the costs in lengthy and complex construction disputes. Properly applied, it is a truly innovative tool of truncating evidence in construction disputes. But, what about non-probability sampling?
The TCC judgment of Amey LG Limited v. Cumbria County Council  EWHC 2856 (TCC) considered and accepted the proposition that disputing parties may, by way of non-probability sampling, prove the liability and quantum of a claim. The case provided necessary guidance for parties that intend using either probability or non-probability sampling for disputes where parties would ordinarily be called upon to prove the liability and quantum of each individual defect pertaining to the construction works.
Amey LG Limited and Cumbria County Council concluded a services agreement in terms of which Amey would provide maintenance services on various highways throughout Cumbria. Amey was also required to provide patch work on the highways. The contract term was seven years, commencing on 1 April 2005. During the contract, there were approximately 36,000 individual works instructions. These included the laying of around 2,200 surfacing beds, with a combined area of around 4.2 million m², and the issuing of around 1,700 separate patching instructions.
The contract was terminated. Amey commenced proceedings to recover the money that Cumbria had deducted in one of Amey’s monthly payment applications. Cumbria rejected the claim and raised a number of counterclaims. In one of the counterclaims, Cumbria alleged that the standard and quality of Amey’s patch and surface works was defective.
The judge accepted that it was quite impracticable for Cumbria to have visually inspected and/or undertaken core testing of all the patches laid by Amey. To have expected Cumbria to conduct such inspections and tests would have taken considerable amounts of time and costs. In principle, the judge accepted that Cumbria was entitled to advance a case based on sampling. The judge also accepted that there was no legal principle to the effect that only random probability sampling was an effective method of advancing this case, and that representative sampling was also an acceptable method.
The issue was whether a process of genuine probability random sampling could be used to show the nature of Amey’s defective works and thus allocate a cost to the defects.
The statistical experts in this case could agree on the key principles relating to sampling and extrapolation in a joint expert statement.
The experts found that probability sampling required the use of a method that identified elements for selection through a randomised process in which each element had a known and non‐zero probability of selection. This random process assured independent selections. A properly drawn random sample could be used to extrapolate a point estimate of a statistic. With a properly drawn probability sample, confidence intervals could be calculated to account for random error due to sampling using known statistical principles. The 95% confidence level was a widely used and generally accepted level of confidence for samples using probability sampling. The primary benefit of probability sampling was that it precisely measured uncertainty inherent in the results and those measures of uncertainty could be derived from accepted statistical analyses.
The experts also found that there may be circumstances where it was not feasible, practical or theoretically sensible to do random sampling, and that instead one could use non-probability sampling. Non-probability sampling relied on subjective judgements about which elements were sampled, rather than relying on a random process that assures the selections were made independent of subjective judgements.
With non-probability samples, the accuracy and precision of estimates could only be determined by subjective judgement. There was therefore a risk that the findings were not valid because of bias in the selection process. The risk of bias did not, however, mean that it was certain that bias would present, but it was an issue that should taken into account when using non‐probability sampling. Non-probability samples could be considered representative if it was established that they were not biased by the subjective selection process. If bias was identified then adjustments would be made to form a representative conclusion.
There was an issue between the experts as to whether it would have been practicable to undertake genuine probability random sampling in this case. Based on the evidence, the judge found that it would have been possible to do so, but equally, that there would have been so many variables that to create rules which would make it a genuine probability random sampling exercise would probably have proved to be a difficult, time-consuming and costly exercise.
The judge accepted that in those circumstances it was not unreasonable in principle for Cumbria to seek to rely upon representative sampling. The key issue, however, was whether Cumbria could demonstrate that it was sufficiently representative to enable the court to place reliance upon it.
In accepting that non-probability sampling was available to parties as a means of evidencing construction defect claims, the judge found that:
- the exercise should be the subject of very careful advance planning to ensure that it could be defended as properly representative;
- those conducting and supervising the exercise should take care to ensure that any issues which arose during the exercise that might cast doubt on its representative nature were identified and addressed in the same careful way;
- the aim was to get as close to probability sampling as was reasonably possible, to remove subjective contamination, and thus to remove or reduce as far as practicable any actual or potential bias; and
- insofar as it was not possible entirely to remove bias, to recognise that it was still present and take appropriate steps to address it.
In short, the judge found that Cumbria failed to demonstrate that the sampling exercise undertaken on its behalf was a sufficiently reliable exercise, free of bias, to justify the court in making the finding as against Amey that there was a systematic and endemic failure in its performance which led to the damages claimed.
The principles laid down in this case are nonetheless relevant if you bring representative defects claims under construction contracts. But, be warned. If you opt to prove liability and quantum by way of non-probability sampling, you must cater for and precisely deal with all sources of bias that may taint the reliability of the conclusions reached in respect of your sampling process.
The authors acknowledge and thank Boitumelo Tsotetsi for her contribution in the writing of this article.