By Steve Gibson, William Henderson, Caren Ulrich Stacy, and Chris Zorn*
For more than a century, baseball has been been known as America’s pastime. Change comes slowly to the game, and tradition is sacred. But as the recent film Moneyball details, one key part of the baseball edifice—the way that teams select and develop their players—has undergone a revolution, thanks to the data-driven approach famously employed by Billy Beane, the general manager of the Oakland A’s.
Beane’s core insight was that the engrained biases of scouts and coaches caused baseball teams to misidentify and misprice talent. With careful attention to data, even a small market team could clobber the competition. Over time, nearly all major league teams have borrowed from the Oakland A’s playbook and won games, even championships, using the Moneyball philosophy.
Is it possible for law firms—also tradition-bound institutions—to benefit from a Moneyball approach to talent? Corporations such as IBM, Xerox, and Proctor & Gamble have done so (well before professional sports), why not law firms? As experts in quantitative analysis, legal labor markets, and law firm recruitment, we explored this exact question with multiple law firms. Our findings might surprise you.
Far and away, the most consistent finding is that there are huge gains to be made by focusing on traits or attributes that are actually correlated with performance.
A Moneyball analysis explores the association between performance and several dozen success traits that can be observed on a lawyer’s résumé or transcript. These range from traditional success criteria such as grades, law review, clerkships and law school rank to non-traditional criteria that many firms overlook or give less weight to—blue or pink collar work experience, advanced degrees, publications, participation in team sports, etc. Using this wide range of biographical data, Moneyball analyses reveal that law firms are often systematically overvaluing some attributes, ignoring others that really matter, and generally making bad tradeoffs in both entry level and lateral lawyer “drafts.”
Consider grades, law firms’ primary filtering device when making hiring decisions. It turns out that grades (like home runs in baseball) do matter for success, but that there are other Moneyball factors that typically matter more and, as suspected, some that matter less. When a firm focuses excessively on grades—which only explain a small fraction of associate performance—it often systematically overlooks better performing candidates.
What are the factors beyond grades that are positively correlated to success at firms? The answer depends on the firm. In some Am Law 100 firms, lawyers with blue and pink collar work experience tend to be among the top performers. While at other Am Law 100 firms in the same profit bracket, associates with blue and pink collar backgrounds tend to underperform. Similarly, with educational credentials, JD-MBAs have thrived at one firm, but at another competing firm only a Masters or Ph.D. in a science or technical field is correlated with success. Performance of homegrown versus lateral associates varies by firm. There are also “non-factors” at many firms that don’t rank as positive or negative predictors of success, including participation in moot court, mock trial competitions, and clinics.
The lawyer success profiles vary significantly across firms. Although AmLaw 100/200 firms compete over market share, just like the Yankees and Red Sox compete over wins and losses, the law firms and lawyers seem to be playing different sports—one is rugby, another polo, a third lacrosse, etc. The data support the existence of distinctive firm personalities and cultures, which are likely the product of a variety of factors, including the dominance of particular practice areas and the principles that have guided the firm’s growth. Contrary to the conventional wisdom of law firm recruiting, many firms may be better off scouting for different success factors and, in turn, hiring different candidates than their so-called peer firms.
Firm culture, however, it only one piece of the puzzle. For a law firm to apply Moneyball principles to its lawyer hiring and development process, it has to confront a simple but difficult question at the very heart of its business model—how do we keep “score” of excellent performance? For example, does the firm measure success simply in terms of billable hours, or through lawyer competencies such as initiative, problem solving and interpersonal skills, which place a lawyer in higher demand both inside and outside the firm?
Some law firms (consciously or unconsciously) make billable hours the key to increased compensation or advancement. Yet, a firm does not win market share or get more challenging and price-sensitive work by amassing hours. The abundance of client work for most law firms is the result of proactive, client- and team-focused behaviors that consistently deliver value in excess of cost. When a firm fails to measure these behaviors and instead relies on billable hours and subjective evaluations that are not competency-based, the Moneyball factors tend to bake-in the biases most prevalent within the partnership.
Bias among brilliant equity partners? Yes, it happens. A good example is attitudes toward law school pedigree. The data suggests that, in several firms, a subset of partners who attended elite law schools often give higher performance ratings to associates who also attended elite law schools—even when non-elite associates are statistically identical on every other measure. In contrast, when looking at the same group of associates, partners who did not attend elite law schools observe no performance gap.
Depending upon the firm, indulging these biases can lead to problems. While some partners have a love affair with academic credentials, they are not always strong predictors of success. In fact, we have observed instances where undergraduate honors and law review (after controlling for law school grades) have been strong negative predictors of essential associate competencies or success traits. This may sound counterintuitive, but it is consistent with other research on the traits of high performing lawyers.
*The authors are associated with LawyerMetrics Inc, a law performance consultancy
