INSIDE AND OUT is a new semi-regular column, where corporate counsel discuss technology law issues with outside counsel. In this installment, Adam Losey of Foley & Lardner talks with D. Casey Flaherty, corporate counsel at Kia Motors America, whose opinions expressed herein are entirely his own.

D. Casey Flaherty: I am a critic of magical thinking. The technophobia that helps fuel the legal profession’s change aversion is often paired with a techno-utopianism that leads lawyers to expect any sufficiently advanced technology to be indistinguishable from magic. A core idea in magic is that only the magicians are supposed to know the mechanics of the trick. The audience is more than just passive, their satisfaction is predicated on their eternal ignorance. Similarly, the user of a magical technology is never to be expected to do anything – e.g., learn to operate the software or gain an understanding of how the device functions.  

Though I am critical of magical thinking, I remain susceptible to it. In particular, I have found that I am sucker for pitches that involve “unlocking” the “hidden” value in large data sets through the magic of analytics. I imagine feeding BigData into a Waston-like device and then having the machine prove and disprove all manner of implicit assumptions I did not even know I was making (of course, since it is my dream, the machine also ends up confirming all sorts of pet theories I hold that run counter to conventional wisdom).

Yet, I submit that a healthy skepticism about the magical promises being made about BigData is not a good excuse to ignore that there is considerable, obvious value to be unlocked by analyzing large, discrete, known data sets. My most recent article revolves around the idea that law firm data on billable time is just such a data set.

As in-house counsel, budgets are of major importance. Though I have quite a bit more work to do, I have started to get a better handle on what my regular matters should cost depending on degree of difficulty and stage to which they advance. But I continue to lack the data necessary to make informed judgments about those matters that are outside the norm.

Uncommon matters are fairly common. That is, though the nature of the individual matter may not appear regularly, there is a relatively consistent stream of irregular matters. These matters are irregular to me and my company. But they are not irregular to the firms we hire to handle them. That the firms handle such cases is the reason for their hiring.

Yet, despite very real experience and expertise in handling these engagements, many firms do a poor job of supplying budgets – even after they’ve been retained (i.e., it’s more than just a sales pitch). My pet theory is that their budgeting performance is largely attributable to an insufficient respect for data. Instead of looking at the huge volumes of pertinent, empirical evidence they have compiled, these lawyers prefer to base their budgets on their own impressions of their own experience. Limited quantification skills then combine with all sorts of inherent biases – e.g., planning fallacy, anchoring, motivated reasoning, etc. – to the detriment of both client and counsel.

I do not expect most lawyers to be able to crunch the numbers themselves. But I do expect firms to invest in that capacity. Indeed, as I point in the article, I am not even advocating for any particular form of analytics. I simply want my lawyers to provide me evidence-based budget projections with ranges and distributions broken down by phases and tasks. I then want my lawyer to make explicit assumptions and identify key variables in predicting where within the ranges the tasks and phases will fall.

Adam, is that too much to ask?

Adam Losey: No for the evidence-based budget projections with ranges and distributions.  Yes, to some degree, in having the granularity you want- i.e., broken down by phases and tasks- but I certainly think it is in a law firm’s interest to have the kind of data you envision.  The rub is in getting good data, and analyzing it in a useful way.

Providing evidence-based budget projections and sticking to a budget should be the norm.  Sure, sometimes wild events happen that can make a budget go haywire.  You can unexpectedly have more depositions than you’d expect.  A spoliation issue could pop up.  Still, it is perfectly reasonable to expect a law firm to give evidence-based budget projections with ranges and distributions for fees and costs.

The issue comes in peeking behind the curtain to phase-and-task based granularity.  We do track, by phase and task, and it is tough to do correctly.  Human beings, and especially lawyers for some reason, have difficulty categorizing the same kinds of tasks consistently- one man’s “Analysis/Strategy” is another man’s “Briefing” is another’s “Budgeting” in the task-code world.

This is the same problem commonly seen in document review in litigation.  It is well established that human assessors will disagree in a substantial number of cases as to whether a document is relevant, regardless of the information need or the assessor’s expertise and diligence.

Plus, busy lawyers will frequently “hug” an individual phase or task code.  In the document-management world for example, a typical document category is that of a “document[,]” and many will save everything as a “document” rather than taking the time to sub-sort into all the various sub-categories of documents.

It is a huge time and effort commitment to take the time to make the value-judgment and plug in a phase and task code for every billing entry, which translates into a big real-dollar cost to a law firm to engage in that type of tracking.  So, the data isn’t always reliable or useful- regardless of attorney diligence- and it is difficult to compile and sort the good from the bad with the typical existing methodologies.

Plus, we don’t see a huge demand to see granular data behind budgeting estimates. Typically a client is more concerned with the budget ranges- not how we specifically calculated those ranges.  Most potential clients (present company excluded) would have no desire to see granular data.  They just want the ranges and to ensure we stick to them, especially in a fixed-fee arrangement where our firm bears the risk of under-budgeting.

So there isn’t much marketplace-side demand for this granular data at the courting stage, outside of some individual client billing practices and outside counsel guidelines that accompany an engagement.

As you put it, the value of a large data set oft comes from its size— a small or medium-sized firm that occasionally handles a type of specific work (i.e., a firm that has a one-time handling of a software licensing dispute) wouldn’t spend the time or effort to track data associated from something they aren’t likely to do again.

However, for a firm that has established core or specialty practices, it makes perfect sense that if the law firms keep billing records (I don’t know of any law firm that doesn’t keep billing records) there is already a large data set regardless of any tracking by phase or task code.

The trick is then turning the morass of data into something useful, and I think there must be a better way than relying on the end-user to take the time to plug in subjective opinions as to phase and task.

Instead, I know there must be a way to parse years of billing records (which are by their nature text-searchable, as well as chronologically sortable, and custodian specific) using some sort of computer assisted review to get the kind of granular data you envision- separate from the phase and task code world.

Given that fixed fees are becoming more common (pushing the risk of loss on law firms) I agree with you that this kind of self-knowledge is crucial for large law firms to survive.