In the film Minority Report, Tom Cruise, as the head of a “pre-crime” unit, apprehends criminals based on foreknowledge provided by psychics. This results in low crime levels, but also in a world where knowing the future limits choice (and implicitly freedom) and access to justice.
The reality of case prediction practice is very different from this dystopian vision.
Today, the goal of predictive analytics for civil litigation is to combine rigorous data and statistical discipline with the expertise of the litigation practitioner. It is not to replace the role of the litigator with a robot lawyer that publishes the answer for the client. The result is improved prediction, advice and outcomes for the litigator and their client.
This is key. Litigation lawyers already predict outcomes, almost always at the demand of their client - not just on the question of their chances of success but also as to how long it will take, how much it will cost and what impact a particular judge, barrister or witness will have on the outcome.
The objective of case prediction analytic is to enhance this, not replace it.
Using rigorous statistical models and accurate, extensive and relevant data, it is possible to produce accurate baseline predictions which practitioners can tailor and refine before turning into client advice. There is extensive research to demonstrate that working this way delivers dramatically improved forecasts of outcome and, consequently, better decisions. In Superforecasting: The Art and Science of Prediction, Philip Tetlock and Dan Gardner show how the use of robust baseline data and statistics, coupled with self-critical human analysis, achieved predictions 60% more accurate than control groups.
How does it work?
Technically, predictive analytics has three aspects: statistical/data analysis, modelling and human-led review and adjustment. On the statistical side, data scientists take multiple data points (broadly speaking in the thousands), across as many variables as possible. They then analyse the data variables to uncover patterns in historic results which correlate with an outcome. These patterns and variables are then modelled to generate a prediction which can be tested against previous and future cases with matching variables. In litigation these are often relatively obvious. For example, the nature of the claim (contract or tort), reliance on factual evidence or points of law, remedy sought, arguments deployed, and the barrister and the judge involved can all have a statistical influence on the outcome.
In summary then, the degree and relationship between multiple variables determine the predictive outcome. Is the judge more influential than the reliance on expert evidence, for instance? It may also be dependent on one of many other variables. This complex combination of variables is essentially the formula that is applied, to forecast future outcomes. In reality though, this is relatively complex mathematics which non-experts have little chance of being able to explain to a client. Few clients will invest substantial funds in a piece of litigation based on a mathematical model forecasting a future event that may not eventuate for some time and which they don’t understand.
How then does this help practitioners?
In current litigation practice, the litigation lawyer will tend to use the historic pattern of results as the basis for beginning to build their own forecast, which they can use to guide client advice and manage their expectations. If you imagine that all cases fall on a spectrum of predictive success starting with <1 percent (almost no chance) to <99 percent (almost certain), cases that sit at the top and the bottom ends tend to be easier to predict. As you move to the middle it becomes harder to forecast with a high level of confidence. For example, it is easy to predict that a poor claim is very unlikely to succeed, but once a claim has some merit, success becomes more difficult to determine. A claim with a 40 percent chance may well succeed, even though it will, on a probabilistic basis, fail 6 out of 10 times. Determining that outcome is very difficult, but knowing that it is 40 percent as opposed to 60 percent is meaningful.
For the practitioner, recognising that a claim in front of them has a 40 percent chance of success (based on past cases with the same features) gives them a starting point for their own prediction. Does the claim require the judge to reject an oft-employed precedent? By how many percentage points would that alter the baseline? Does the reliability of the factual witnesses move it closer to 50 percent? Is a commercial interpretation of a contract key to success, so taking it down a few percent?
The more carefully the practitioner undertakes this exercise, the more accurate their forecast becomes, particularly if they self-critically challenge their prediction and assumption for each component of the case they analyse.
At the end of this process, the litigation lawyer has the basis for sound advice to the client. (Of course, they also gauge the ability to pay, appetite for risk and motivation of the client to stay the course.)
Used in this way, and with rigorous data and analytics to support them, a lawyer’s ability to accurately forecast and then effectively guide and advise a client is meaningfully increased. This is especially so where the client is data-led in their attitude. For example, it helps to persuade a CFO, perhaps reluctant to pursue expensive litigation, to know the probabilities, not just of the outcome, but of the duration; if you can turn that information into data they can utilise in their own financial and risk models.
The value of predictive data and analytics does not end there. For many cases the process of determining strategy and argument relies on a combination of litigator experience and case law research. Litigation analytics can help the user find the best arguments, tactics and insight, by focusing on outcome and on the factors that impact outcome.
Summary judgment application in front of Walker? The data tells you that just over 69 percent of applications fail in front of Walker J, as opposed to just 31.5 percent on average for that court as a whole. This data point is key; it may well make you alter your decision. But of as much value is to be able to readily access the relevant judgments that sit underneath, to determine why they failed.
The analytics can also give an edge or anchor point for settlement negotiation. If 66 percent of negligence claims fail at trial, the defending party introducing that data point would put considerable pressure on the claimant to persuade them that they would still win in court. The analytics can also potentially form the basis for agreeing the settlement value of a claim and coming to agreement.
As a decision support offering, litigation analytics is already widely used in the USA. A recent FT article suggested around 75 percent of the top 100 US firms already make use of litigation analytics services. There are broadly two main provider groups: the large legal research providers including Bloomberg, Thomson Reuters and LexisNexis (Lex Machina) who have provided analytics add-ons to their core offerings; and a range of agile litigation analytics businesses offering new solutions for US jurisdictions, with companies like Premonition, Gavelitics and Ross Intelligence offering algorithm-powered litigation support.
A key driver of this growth is the increasingly data-led decision making that large corporates and their legal functions use in order to drive more aligned business plans and decisions. This has created a demand for data to support business and risk management strategies where the executive leadership can balance the risks (financial, regulatory and reputational) and resource investments they make.
Unsurprisingly this same driver for more data oriented legal decision-making in commercial organisations is driving increased interest in the UK.
Accessibility of data
While the demand in the UK is growing, and we have seen considerable interest ourselves, the availability, consistency and usability of case data is substantially lower than for US courts. The UK courts system is not set up to enable fast and straightforward production, publication and distribution of court information, including judgments and other court documents.
In addition, this variability in the availability and accessibility of data makes the use of advanced machine learning technologies more difficult. The lack of consistent templates, styles and content structure makes it harder for these tools to work and learn as quickly.
The incorporation of litigation analytics data into the key litigation decision-making and research processes will become widespread as clients seek support for their data-led decision making and litigation lawyers look for every area of advantage and a competitive edge. The ability to find relevant data more quickly will also help to even out some of the resourcing imbalances you can find, especially when smaller or less well-resourced legal teams go up against the big firms.
New skills for litigators
As in other areas, while data brings value and insight, relying on it is not without risk. Broadly, there are two types. The first is in the misinterpretation of data and analysis and its inaccurate or incorrect utilisation. Firms and lawyers need to master key areas of data and statistical analysis in order not to fall into this trap. The smarter litigation analytics firms are supporting their clients with training and providing helpful gradings, scores and guides to assist with this. Importantly, this investment in new skills and competencies will have a benefit in improving the forecasting accuracy of those that engage with it.
The second risk lies in the tendency to shortcut decisions by assuming the data provides the full answer. Once the data suggests a claim has a 70 percent probability of success, that chance can become a fait accompli. Without careful management, we risk short-changing claimants and defendants and removing their access to justice. It also means that potential cases on which the law may well need to be challenged may never be brought to court because the likelihood of success is too low.
It will require the thoughtful application of focused insight to ensure this doesn’t happen. Automated tools spitting out answers provide little real value.
This is why our company’s focus is on marrying the analysis and insight the data can reveal with the skills, instincts and talents of the litigator. Practiced this way, these risks are not only manageable but effectively identified in advance. As a result, the art of litigation is enriched and this adds to the quality of decision making and advice that the litigator can provide their clients.
As a way of enhancing the skills, and abilities of the litigator and the value of the service they provide to clients, litigation analytics has a very promising future.