Moneyball (both book and movie versions) portrays a baseball team’s general manager, Billy Beane (Brad Pitt in the movie), who eschewed conventional wisdom and instead relied on data analytics to build a successful team on a limited budget. But sabermetrics is no longer confined to sports.
The Atlantic, in a feature story on this crossover in its December issue reported the following: “Last year [Billy Beane] appeared at a large conference for corporate HR executives in Austin, Texas, where he reportedly stole the show with a talk titled “The Moneyball Approach to Talent Management.” That trend is confirmed in additional articles in the New York Times (how Google is “building a better boss” and “building better workers”); the Washington Post; Forbes (here, here, and here); and Bloomberg Businessweek.
Commercial employers are following the lead of sports employers in using or experimenting with people analytics – the collection and analysis of data about applicants and individuals in the workforce. Exploiting the available data (and there is a lot of it) to determine what makes an effective sales person or call center worker and then applying it to new hires and current employees holds a promise of increasing retention rates, expanding market share, improving customer service, and boosting innovation. In short, people analytics may make your business better. But, as with some much else in the human resources world, it is not without risk.
There are three core problems with the use of people analytics:
First, finding the right metric is crucial. For example, Billy Beane viewed getting on base as a better metric than either batting average or RBI. Yet the wrong metric is worse than worthless: soccer teams that focus on possession or on shots, for example, are missing the forest for the trees according to Professors Anderson and Sally in their book The Numbers Game: Why Everything You Know About Soccer Is Wrong.
Second, when an employer uses people analytics at the hiring stage, it is necessarily excluding those applicants its models show might turn out to be unhappy, unproductive, or otherwise unlikely to succeed. That makes practical sense – why waste resources on employees destined to fail? But this has the potential to run smack into a disparate impact claim. Simply stated, any screening device that produces a statistically significant disparity between men and women, whites and blacks, etc. creates liability unless it is justified by “business necessity.” Griggs v. Duke Power Co., 401 U.S. 424 (1977) (holding that education requirements and written tests created such a disparate impact).
Third, when an employer uses people analytics with current employees, it is typically doing so to collect information on what makes a successful engineer, manager, etc. and then make decisions based on that information. This, of course, raises the same disparate impact risk as when directed at applicants except now the decisions are promotions, demotions, compensation setting, etc. based on data analytics. And with current employees, there is also a potential privacy problem. Employee performance metrics are certainly fair game, but what about all of the non-obvious data that an employee generates every day and that an employer can now collect and analyze? For example, is it okay to analyze emails to determine “dedication” by comparing after-hours traffic? What about measuring decisiveness by tracking the time from when an email is opened to when it is answered?
There are, in turn, three keys to addressing those potential people analytics problems before they become thorny legal issues:
- Analyze First. Know what you are trying to accomplish and what metrics are actually predictive. For example, the widespread use of credit checks in hiring is predicated on an unproven assumption that there is a correlation between dishonesty and low credit scores. Such “conventional wisdoms” are the antithesis of what Billy Beane attempted in baseball and what Professors Anderson and Sally advocate in soccer.
- Minimize Disparate Impact Risk. There is a greater risk of a disparate impact claim when analytics are used in the hiring stage, so scrutinize and control that usage. For example, using an analytics-based factor as one, non-determinative component of a comprehensive hiring process makes the application of disparate impact analysis less likely. For decisions related to ongoing employees, it may be worth historical testing: if applied last year, would this have produced a disparate impact?
- Recognize Privacy Rights. If you are going to collect data from employees to drive employment decisions, give notice (which eliminates any reasonable expectation of privacy) or – even better – get consent. And don’t bury those provision in fine print in some sign-off on IT usage policies. If your metrics are worthy of using, then their use is worthy of explaining.
People analytics will not entirely replace “gut feeling” decisions, but its use is increasing. Done well, it holds promise. Done poorly, it recalls an earlier mantra of the information era: GIGO (“garbage in; garbage out”).