How good are pro sports teams at picking good players?

Factual Questions
1

More specifically: has anyone ever done a statistical analysis of how much more effective first-round, top-flight pro sports recruits are than unheralded free-agent picks? Every NFL game I watch, I’m amazed by how many players come from no-name colleges, and/or were picked late in the draft. The Cowboys’ winning QB from yesterday, Tony Romo, came from Eastern Illinois U and went undrafted, but he jumped over first-round Michigan QB Drew Henson in the lineup. And there are of course plenty of Ryan Leafs or other bust picks to throw doubt into the selection process.

So, is it a crap shoot? Are teams really spending their money wisely with high-dollar first-round contracts when a 7th-round player might just as easily make his way to the starting lineup? Am I just noticing the exceptions to the rule more?

2

An interesting question, and I wish I had an answer for you. I’m not sure how you could objectively measure the “effectiveness” of the players for your correlation, but maybe an easy place to start would just be to analyze the draft position of all Hall of Fame members in each sport? Although that might be confounded by the year they were drafted, e.g. maybe the NFL draft worked differently in 1985 than it does today.

3

From my personal observations, if the team is based in Cleveland OH then the executives picking the players generally can’t choose a good one unless he’s a lead-pipe cinch like LeBron James. :smiley:

4

It is a crap shoot, Because players are poked, prodded, run through the paces, and watched from a relatively young age the teams generally know enough about a player to make an informed decision. Nonetheless, there have been some notable exceptions. Ryan Leaf, Sam Bowie, Tony Mandarich, Brian Bosworth, Akili Smith, the list goes on and on, but Jack Lambert, Johnny Unitas, etc. either got drafted low or not at all.

Every few years the analysts do some sort of retrospective on how the big names turned out, and the teams are right about as often as they are wrong. But it’s not because they did something wrong. Often it’s because of something else, like the player’s out-of-control behavior, some sort of drug abuse, or a major career-ending injury, all of which could happen with any player picked in any round.

It is also worth noting that when you get to the lower rounds the players selected almost never make the team, or if they do they have minimal impact. The last guy picked is called “Mr. Irrelevant” in the NFL draft, and none of them have had any sort of real impact in their careers. Therefore, it stands to reason that the teams must be doing something right.

The selection process is not quite a random chance sort of thing, but yes, things can go horribly wrong, and that is often a random thing.

5

Sam Bowie has gotten a bad rap for 20 years. Portland knew when they picked him that he’d had injury problems here at Kentucky, and that it was going to be a crapshoot as to how he’d hold up over an NBA season.

Sam Bowie wasn’t a draft bust, he was a bad pick at #2 because Jordan was available.

6

According to the book Moneyball , no, teams choose players on all sorts of irrelevant factors such as “gut feel” and do not end up with the best players for the money they spend.; the author (Michael Lewis)

7

Look. I know you’re a prime time Kentucky homer, and I know that that colors your thinking on matters like this, so I’ll say this gently: nobody gets taken at number 2 in the draft if a series of debilitating injuries is expected. Even without Jordan going behind him he was a primo bust. That just makes it look worse.

8

I think the question is a bit flawed in its fundamentals.

“Success” at the major league level means “being better than the majority of players who have already made the major leagues.” So by definition there’s not much room for a player to be a success.

MLB has 750 players on its rosters for the bulk of the season. This means you have 375 players who are by definition above the median talent level. BUT: the MLB teams draft 1500 player a year. So it’s literally impossible for the teams to have a “high” success rate in the draft, overall. You can only hope to have a better rate than your neighbors. It is a zero-sum game, because if I draft a good prospect, then you can’t have him.

As has been pointed out, almost all of the players in MLB went in the top of the draft. What with unexplained mental flameouts (Rick Ankiel), freak injuries, unexpected talent limits (never learned to hit the outside slider), most of even the top half of the draft fails to be above the median talent level. This only stands to reason, as the MLB is the ten-year All-Star team of the previous ten drafts, as determined in real play, not on paper.

I remember that the Dodgers at least used to play their AAA team in a series of exhibitions, and would of course win every series. How could Albequerque have a prayer? Most of their team was never going to collect a big league check, ever.

I read Moneyball, and I think Beane has a point: Stop getting out. Just don’t get out, and we can keep the inning going. I mean, what would you give for a player that averaged 6 piteches per at-bat (wearing down the pitcher) and had an on-base percentage of .800, even if almost all of it was on walks? You’d give a lot of money, is what. Runners end up getting scoring chances.

Of course, Beane points out that doubles are worth more than singles, etc (and yes, I realize he’s building on years of statistical analysis by other people). I’m sure Beane would love to have Vladimir Guerrero who is a notorious bad-ball hitter, even though VG goes against the “be careful and pick your high outcome pitch”, except that Beane knows he has to work with a budget that can’t afford Vladimir. In 2006, he had the third-highest OBP of the players with the 30 most at-bats, and the third highest slugging average in that group (bases per official at-bat). Both those numbers are below his career average. He’s a monster.

9

I don’t have time to do a search in The Economist archives, but a couple of guys did do research on this to some extent. They were economists and found that teams actually got their best value in the second and third rounds of the NFL drafts. Money wasted on one first rounder (even if he pans out) could be better spent on a couple of seconds and thirds who will contribute.

But its not an exact science. And it a lot of ways, it is very much a crap shoot. Some good coaches are able to trim the odds a bit and that’s one reason they are great coaches. But how does someone really know who is going to work hard and who is going to run his mouth and complain? Look at a few dozen college players and tell me which ones have the character to work hard in practice. Or tell me which early draft pick will get star struck and go off the deep end? Which troubled kid will finally bear down and decide to be a superstar. Which no-name kid will have a desire to play so bad that he puts in overtime studying plays and working out? Which small-school player got his stats because of big-time talent or a lack of any serious competition.

I still harp on Cheesesteak Andy picking Freddie Mitchell over Reggie Wayne, Chad Johnson, Steve Smith, Alge Crumpler, Chris Chambers and TJ Houshmanzadeh. Smith had a lot of off the field problems. Houshmanzadeh was not a number one in college. So there were knocks against some of the guys. Wayne was fortunate to have a guy like Marvin Harrison to learn from. I guess we underestimated Freddie’s mouth while overestimating his ability to find open field.

10

The answer to this question is, “YES”.

Tons of it. In every single sport.

The amount of math and statistics applied to baseball, basketball, football is incredible nowadays. Not just performance on the field, but it’s application to draft picks.

Here’s an article by “Football Outsiders” about the value of draft picks:

http://www.footballoutsiders.com/ramblings.php?p=2530&cat=10

It references a study done by economists at the Duke and Chicago schools of business which you can find (PDF WARNING) here.

Football Outsiders is a site with lots of statistical analysis about football players and teams. They have interesting ways of determining such things as how much of a run is attributable to the line or the player. They rate players based on their "defense adjusted value over average’ instead of just raw statistics.

Baseball is loaded with this stuff, anf they’re recently finding ways of applying it to basketball. There was an article last year that none of the commentators understood, but they disagreed with it in part because it indicated that Allen Iverson wasn’t as helpful as everyone thought he was. (he was way down the list of effective players the year that the sports writers voted him MVP)

IMO, I don’t think it’s fair to call it a crap shoot. Undoubtably, first round draft picks historically have been better than lower round draft picks. The real question for teams is whether the money paid to them is worth it, and whether other teams over-value draft picks too much, allowing you to exploit their mis-valuation.

11

I don’t have the book in front of me, but I’m recalling a passage from C.S. Lewis’ autobiography, in which he describes the first boarding school his father sent him to. Lewis thought it was a hellhole, one of the worst imaginable schools on Earth, and wondered how his father could have chosen it. At last, he explained it thus: if his father had gone with his first instincts, he would surely have picked a good school. Or, if his father had done a little bit of research and study, he would undoubtedly have picked a good school. Instead, his father studied and re-studied, considered every angle and then re-considered every angle, looked at every school from 50 different points of view, until he talked himself into the worst possible choice.

And, smart as they are (all of them know waaaaaay more about football than I ever will), NFL scouts and general managers often act like Lewis’ father. They get so caught up in physical stats and bench press stats and psychological profiles… they make the process so very complicated that they overlook the most important question of all: can this guy play football?

I mean, it’s very impressive if a defensive tackle from TCU is 305 pounds with 1% body fat, can bench press 450 pounds, and can run the 40 yard dash in 4.5 seconds. But if a guy with those kinds of physical attributes was NOT an utterly dominant player in college, you have to wonder why. Far too often, scouts don’t wonder why, because they’re too busy drooling over the kid’s physical attributes.

I mean, here at Texas, I saw offensive lineman Leonard Davis play for years. He had all the physical assets an NFL coach wants in a lineman… but I saw how rarely he opened big holes for running backs, and how often undersized opponents got past him to put pressure on Chris Simms, and wondered how the Cardinals could have drafted him so high.

12

The NBA dropped their draft to just two rounds because they realized there was little point in going any further. The kids are watched from an early age and their skill level evaluated, debated and dissected continually. The number of jobs available is so limited that if you are not in the top 60 considerations your chances are between slim and none.

Certainly, there are busts among those choices and on rare occassions a free agent will get a job. The other thing that teams consider is that if a player hasn’t made it through that small window of opportunity there is always someone younger coming up. They turn their eyes towards the up and comers.

13

You’re reading the wrong Lewis.

The major point of Michael Lewis’ Moneyball is that scouts looking at players and going, “can this kid play baseball?” are at a huge disadvantage to the guys looking at the kid’s actual stats.

Slightly apples/oranges here, of course.

Football gets more complicated than baseball in some regards. College performance is highly “competition dependent” whereas a 4.4 40 is pretty objective.

But, you can’t measure a guy’s “instincts” in the combines, and that’s a major part of what makes players like Ray Lewis so great (well, his speed, strength and size help too, but there are plenty of guys with those attributes who are lesser players).

14

Basically, it is a crap shoot, but there are ways where loaded dice can be detected. After all, it’s more likely a 7 will come up in craps than any number, and if you have to pick just one number, it makes sense to choose 7 every time, even though you’re going to lose fairly often.

Getting away from the analogy, there is no certain way to predict how any young player will do. Statistical analysis does not predict a thing (even those who try to predict admit they are fudging the numbers, if you read the fine print). Scouting the player is a little better, but not by much. A combination is useful, but can’t guarantee anything. And it’s even harder to predict personal issues that might derail a player (e.g., Dwight Gooden, who looked like he was on the way to a Hall of Fame career before getting involved in cocaine).

A team tries to take everything into account: statistics, physical ability, personality, etc. They are probably better than average on finding talent, but there is still plenty of room for error. In baseball, for instance, a hitter may not be able to learn to hit a curveball. A pitcher may not be able to learn to throw one. An injury may wreck their career.

At the same time, a borderline player may improve over time. Mike Piazza was drafted in the last round by the Dodgers, and then only because their manager, Tommy Lasorda, was a family friend. Piazza kept working on and improving his game and now he’s finishing up a hall-of-fame career.

It’s especially difficult in baseball because of the long development time. In football and basketball, you have a better idea, since Division I play requires a level that’s close to the pros going in. The colleges are the minor leagues, and you’re just picking the best on what would be the AAA level in baseball. If baseball worked so that the minors were independent and the major leagues drafted from them, the result would be more like those sports.

But it all boils down to judgment, and it’s easy for even the smartest people to misjudge talent. So you keep picking what looks like a 7 to you, since that’s what’s most likely to come up.

15

No. Sam Bowie WAS a bust. But to continue the hijack, the team that really blew that draft was Houston. With the benefit of 20/20 hindsight, the Rockets, who drafted first and selected Hakeem Olajuwon and who already had Ralph Sampson (who was highly regarded at the time) should’ve traded Sampson to Portland for the pick, and then used the #2pick to select Jordan. A team with Hakeem and MJ would’ve ruled the league for 10 years.

16

Esp. considering that Sampson ran off the cliff 3 or so seasons later and was almost as bad a
bust as Bowie all things considered.

17

I don’t know if the majority of MLB players come from the top half of the draft. That’s because there are so many players from the Dominican Republic and Venezuela who are not subject to the draft. Some teams just sign players there by the bushelful, send them to academies they have in those areas, and wait to see who pans out and who gets to sent back to the sugar fields (or corresponding metaphor for poverty in Venezuela).

18

If memory serves, the Lakers were at one point willing to trade James Worthy straight up for Sampson. If only the Rockets had taken THAT deal!

19

A large part of Worthy’s value was finishing a pass from Magic (which is why basketball is
a team sport). Without Magic I don’t think his scoring average and his value would have
been nearly as high. I still make the trade tho as Houston.

20

Another thing that complicates this is that a player’s success is often situational. Tom Brady is an excellent QB with three rings, but he only ever got his shot because of Bledsoe. He could just as easily have been drafted somewhere else, like say Green Bay, and still be an unknown backup because Favre has played every game.

Or take Leaf vs. Rivers. Leaf is a world-famous bust and Rivers looks great, but maybe if Leaf had backed up someone like Brees for a couple years things might have worked out.

Also I wonder how much draft order is sort of a self-fulfilling prophesy. As mentioned, lots of late-round picks never even make the team, and part of that has to be that teams use similar criteria for evaluating who to draft and for evaluating who to keep on the team.