Do Draft Grades Mean Anything?

The short answer is yes.

Hello, sports fans. TC Fleming here. Football Outsiders released their annual Draft Report Card Report this week. They've been doing those since 2004. Theirs is an interesting study. They look at eight respected national writers who give out draft grades and average those to generate a consensus on each team's draft. You can see the logic: one guy saying 'I think this Packers draft class stinks" means essentially nothing. But taking eight people who have spent a lot of time getting familiar with these prospects and seeing if their collective wisdom can tell us something, that has possibilities.

I was honestly surprised they hadn't looked into how these draft grades correlate to the production of the draft classes after the fact. I had some time, so I checked into it.

To measure the production of those draft classes, I used the Approximate Value (AV) stat from Pro Football Reference. It's a stat trying to provide one uniform scale to measure performance across all positions, so it's going to have its flaws (eg ranking Joseph Addai as the 10th-best player in the 2006 draft). It fits our purposes though. We're just looking for a wide-angle look at whether or not teams drafted players that contributed to their team.

Pro Football Reference has an exceptionally handy page on each draft where they list the AV that each player accumulated while with the team that drafted. I pasted those into Excel and added up the totals for each draft for each team. I decided to use only the 2005 through 2009 drafts. Anything more recent than that and I was afraid we'd be dealing with too many guys on whom the book is still out.

While I was compiling this list of AV totals for each team's draft classes, I noticed that there was a large gap between the totals from 2005 and the totals from 2009. That makes sense, since the 2005 players have had more opportunities to accumulate value. I was afraid this would add another variable and artificially disrupt the correlation factor. So I graded on a curve. I added together the total AV for each year's entire draft, averaged those numbers together, divided that average by the total AV for each year's draft and then multiplied the team totals by that factor.

In the end, the Approximate Value for a team's draft showed a correlation of about .36. It's on a scale of zero to one where zero is no relation and one is a direct relation. So collectively, draft experts know something. It's better than throwing darts. Draft grades don't say a lot, but you can't brush them off either.

What does this mean for the Cowboys? Well, they ranked dead last in this year's report. In the five years that I studied, the lowest-ranked team had a below-average AV total four times. In the lone outlier, Tennessee was docked by most experts for over-drafting a small school running back with their first pick, but Chris Johnson proved effective enough to make it a fairly valuable class. In every other instance, the experts were more or less right to say those classes were lackluster. The average draft class in this study produced 83 points of Approximate Value. If you look only at classes that experts ranked in the bottom five, that average drops to 59. Ranking so low does mean something. When experts scoff at your drafting, there is legitimate cause for concern. The lowest-ranked class rarely turns out to actually be the worst class, but it almost always turns out to be a bad class.

One bit of trivia: the Cowboys own both the highest and lowest for AV totals for a single draft class. The 2005 class that produced five regular starters (DeMarcus Ware, Marcus Spears, Chris Canty, Marion Barber and Jay Ratliff) is the most productive on record. And looking through these, I was surprised at how few regular starters teams pick up in a single draft class. When you consider that there are 22 starters for each game and that football careers are so short, I would figure that there would be a team every year that found five starters in one class. Not the case. The 2009 class, on the other hand, famously produced nothing of value. The AV total for 2009 was 15. That is the lowest AV total in this study, but I would argue it's not the worst draft class. The Lions in 2005 had a much better selection of picks than the '09 Cowboys and produced only 18 AV points. That was the year they picked an out-of-football Mike Williams who was cut after two seasons. Shaun Cody was their second rounder, and he's become a decent player with Houston. However, he started just 12 games in four years while playing on some really bad teams. So I'd call that Cowboys '09 class the second-worst.

Jake and I talked about this little project on our podcast if you'd like to hear a little more.

And in the interest of showing my work, here's the spreadsheet I put together.