Home Unfiltered Articles Players Baseball Prospectus
Basketball Prospectus home
Click here to log in Click here for forgotten password Click here to subscribe

Click here for Important Basketball Prospectus Premium Information!

<< Previous Article
Premium Article NBA Finals (06/25)
Next Article >>
Drafting by Similarity (06/27)

June 27, 2012
Making the Right Choice
NBA Drafts

by Neil Paine

Printer-
friendly
Contact
Author

The economists Richard Thaler and Cade Massey once wrote a pretty famous (in these circles, at least) academic paper analyzing the NFL Draft, wherein they found that teams drastically overestimated the value of high first-round picks. Instead of those selections delivering the most surplus value--defined as future production per dollar of salary-cap room spent--as predicted, Massey and Thaler estimated that picks at the end of the first round and the beginning of the second round were the ones that created the most bang for teams' bucks.

One component of the Massey-Thaler study that I don't believe applies anymore is the fact that, at the time of its writing, there was no scale for the salaries of NFL rookies, which is a key item that was changed when the league and players' union renegotiated the CBA last summer. A large portion of the surplus value found by Massey & Thaler was driven by the fact that high picks were paid like superstars before playing a single down, while late first/early second-round selections were paid much more modest salaries despite carrying only slightly less expected value than top picks. With a rookie wage scale, this is no longer an issue for the NFL (and hasn't ever been one for the NBA).

The other component, though, is one which hasn't changed for any league: the fact that decision-makers seem to vastly overestimate their ability to rank talent from top to bottom. The typical NFL team's overconfidence in its scouting skills--essentially its ability to predict how much future value a prospect will produce--leads it to significantly overvalue the right to choose one player over another within the same generalized tier (which is all a high draft pick gives you).

As Massey & Thaler point out, teams are not making "regressive predictions" when they haggle for the right to choose early in the draft. In other words, they're not taking into account the wide variety of possible outcomes the high pick could deliver, and therefore are incorrectly managing risk. In the words of Massey/Thaler:

"[T]o be regressive is to admit to a limited ability to differentiate the good from the great, and it is this skill that has secured NFL scouts and general managers their jobs."

According to the Massey/Thaler paper, NFL teams are nowhere near as accurate as they think they are when it comes to predicting future returns from a given pick, particularly at the top of the draft. But is this true of NBA teams as well? First, think to yourself about the NBA draft--in hindsight, what are the odds that a team with the #1 pick actually takes the most productive player available? What about the #2 pick, or #3, and so on, all the way through the first round? Remember your guesses for later.

In the meantime, to put some actual data behind this, I looked at the career totals of three very different value metrics for NBA players: Daniel Myers' VORP (based on a boxscore regression of Regularized Plus-Minus stats), John Hollinger's Estimated Wins Added (EWA; based on his Player Efficiency Rating metric); and Basketball-Reference/Justin Kubatko's Win Shares (based on Dean Oliver's Basketball on Paper efficiency ratings). All three have different biases, and admittedly none are perfect, but they do give you a solid picture of the value a player has produced over the course of his career, and as we'll see, they tend to agree in the big picture.

Anyway, I took those career numbers for players drafted between 1989 (the first draft to feature the modern two-round structure) and 2007 (the last draft where we can remotely say that a certain player's career is likely to be better than another's). 19 picks were made for each current first-round slot, 1-30, and we can track how often the teams with each pick chose the most valuable player available according to each metric's notion of career value.

Remember how often you predicted teams pick the best player available when given the right to choose? Here are the actual answers:

Pick # % (VORP) % (EWA) % (WS) Total Model
1 26% 32% 21% 26% 30%
2 21% 26% 21% 23% 30%
3 32% 32% 32% 32% 30%
4 21% 32% 32% 28% 30%
5 53% 42% 37% 44% 30%
6 11% 5% 11% 9% 30%
7 21% 16% 16% 18% 31%
8 21% 26% 26% 25% 31%
9 47% 42% 42% 44% 31%
10 47% 32% 37% 39% 31%
11 32% 26% 32% 30% 32%
12 16% 11% 21% 16% 32%
13 42% 42% 47% 44% 33%
14 32% 26% 32% 30% 33%
15 37% 37% 42% 39% 34%
16 32% 47% 37% 39% 35%
17 47% 58% 47% 51% 36%
18 37% 37% 32% 35% 37%
19 21% 26% 26% 25% 38%
20 37% 37% 26% 33% 39%
21 63% 63% 63% 63% 40%
22 32% 26% 32% 30% 42%
23 74% 58% 68% 67% 43%
24 53% 58% 58% 56% 45%
25 42% 37% 42% 40% 47%
26 47% 53% 47% 49% 49%
27 58% 74% 63% 65% 52%
28 47% 53% 47% 49% 54%
29 47% 37% 47% 44% 57%
30 53% 53% 53% 53% 60%

("Total" is the percent picked correctly across all metrics; "Model" is a smoothed-out probability for the pick using a simple multinomial model.)

Basically, when given his choice of every player in the draft, your typical NBA general manager makes the "right" call somewhere between a quarter and two-thirds of the time. In fact, after smoothing everything out, having the right to any top-10 pick means you whiff on the best possible player about seven out of every 10 tries!

This isn't to speak of the magnitude of the mistakes; sometimes they're small (Carmelo Anthony over Chris Bosh at #3 in 2003 was pretty close to a wash, metrically speaking) and sometimes they're massive (taking Darko Milicic second over Anthony, Bosh, and Dwyane Wade that same year). But in terms of the pure ability to identify the player whose career will turn out better than all others simultaneously on the board, NBA GMs are succeeding less than 40 percent of the time across all first-round picks--and their batting average gets worse the higher the pick is.

That's why there's such a tremendous risk involved at the top of the draft, because with so many theoretically good players available, there's no real way to differentiate which ones will be better than others. In that sense, high NBA draft picks are what Nassim Taleb might call "Black Swan opportunities," circumstances that arise when random (unpredictable) events have an enormous impact on the course of history. We know that hitting a home run on a high pick is nearly a requisite if you want to win a championship, but this data also shows that it's pretty random as to whether you knock it out of the park with that pick or not. In spite of that unpredictability, the success or failure of that pick almost always has serious long-term ramifications for your team and the league as a whole.

One might be tempted, then, to call the NBA Draft a system where nobody knows much (beyond sorting prospects into vague tiers), everyone thinks they know far more than they do, and the outcome of a largely random process dictates the long-term futures of all parties involved. That's probably too harsh an assessment, but it's still a sobering thought to keep in mind when watching the proceedings on Thursday night.

Neil Paine is an author of Basketball Prospectus. You can contact Neil by clicking here or click here to see Neil's other articles.

2 comments have been left for this article.

<< Previous Article
Premium Article NBA Finals (06/25)
Next Article >>
Drafting by Similarity (06/27)

RECENTLY AT BASKETBALL PROSPECTUS
State of Basketball Prospectus: A Brief Anno...
Tuesday Truths: March-at-Last Edition
Easy Bubble Solver: The Triumphant Return
Premium Article Bubbles of their Own Making: Villanova, Temp...
Tuesday Truths: Crunch Time Edition

MORE FROM JUNE 27, 2012
Drafting by Similarity: 2012 Comps

MORE BY NEIL PAINE
2012-06-27 - Premium Article Making the Right Choice: NBA Drafts
2012-05-30 - Market Advantage: New York vs. L.A.
2012-05-21 - Most Dominant: Ranking the Champs
2012-04-18 - Coach of the Year: Quantified
More...


Basketball Prospectus Home  |  Terms of Service  |  Privacy Policy  |  Contact Us
Copyright © 1996-2014 Prospectus Entertainment Ventures, LLC.