This post is a reply to some comments made in this thread. They concern the website (and published works of) Football Outsiders (FO), which does advanced football statistics and analysis – I’ve been pimping them here and elsewhere for a few years now. I missed the linked thread the first time around, and found it by accident tonight when I ran a search for my screen name. Since the original thread is a few months old and this post is quite long, I decided to start a new thread rather than resurrect the old one.
Aside from the immediate topics of this thread, if anyone has any comments, questions, or criticisms of FO, I’d be interested in addressing them.
Ok, on with the show:
They really don’t. They make lots of predictions for each team, as well as hundreds or fantasy football projections every year, so of course you’ll be able to point to instances in which they were way off. OTOH, there also plenty of times that they predicted seemingly unlikely things to occur, and they did, for exactly the reasons they gave.
For instance, after 2003, when the San Diego Chargers were coming off a miserable four year stretch in which they had a record of 18-46 and a lame duck QB, FO’s projection system said that the Chargers would completely turn it around in 2004 and have one of the best offenses in the league (as well as an improved defense). I think it’s safe to say that they were the only ones making such a prediction, and they were absolutely right. Their system noticed that San Diego’s 3rd down performance was way out of line with their 1st and 2nd down performance, and that 3rd down performance tends to regress to a team’s mean, which has a disproportianately large impact on their overall success.
Or consider the 2005 Atlanta Falcons. Halfway through the season they were 6-2 and tied for the best record in the NFC, yet FO had them ranked as the 17th best team in the league. They caught a ton of flack for it, including a major flame war in the discussion threads brought on by some visiting Falcons fans. But they were absolutely correct – their stats (DVOA in particular) noticed that Atlanta had faced weak competition and had had great luck recovering fumbles (they had recovered 15 out of 19 fumbles in their games – more on fumble recovery later, btw). Over the second half of 2005, the Falcons went 2-6 – their level of play remained the same (at the end of the year they were still ranked 17th in DVOA), but they faced a tougher schedule and, more importantly, their fumble recovery luck was completely reversed: they recovered just 5 out of 18.
Other noticeable predictions, all from this year: Oakland would have one of the best defenses in the league (they improved from 23rd to 9th), Philly would bounce back, and Minnesota would get a lot worse. To be fair, they also predicted that Seattle would be the best team in the league – I don’t think anyone knows why they regressed so badly (even when healthy they played mediocre football).
First, the article you refer to isn’t like the official canon of Football Outsiders – it’s just a guest column by a reader who used their numbers to do a lot of research and draw his own concluions. Second, FO does now differentiate between run support and pass defense plays in their defensive player stats. Third, FO is always quick to point out that a lot of their numbers are in their infancy, with much room for improvement. Finally, even the writer of the guest column in question isn’t taking those numbers as gospel: “. . . The second possibility is that there is important information that is being missed by the PFP numbers. And until we can explain what is going on with Kelly “Lockdown” Herndon, we can’t accept Bailey’s PFP numbers as being a truly accurate reflection of his play.”
KC Joyner puts out his own book and writes for ESPN.com, so don’t hold his flaws against FO.
[Mocking parody of FO:] “And anyway, if Bailey is so good, why is Denver’s defense not good against the pass? Incidentally, we rate Denver as 4th against the pass, which doesn’t matter. Idiot.”
Come on, the article gives a perfectly good explanation of why you might discount Bailey’s contribution to Denver’s good pass defense: “Then again, if Bailey was so effective, why doesn’t it come through more in Denver’s DVOA numbers? Yes, Denver was fourth in the league in passing DVOA, but a closer inspection of the numbers provided in Pro Football Prospectus 2005 shows that Denver’s high ranking was almost entirely due to its tremendous ability to cover the tight end (-43.6%, 2nd) and running backs (-48.8%, 2nd). Their DVOA versus #1 receivers was only 7.4%, a substantial improvement over the year before Bailey’s arrival, but still below average. Denver’s DVOA against #2 receivers actually got significantly worse in 2004, dropping from –9.1% to 14.4%, which both highlights the strangeness of Herndon’s PFP numbers and throws into doubt the notion that Bailey’s presence allowed the team to roll coverage effectively to the other side of the field.”
I wonder, are these numbers online somewhere in some kind of searchable format?
Sadly, no. FO’s website needs a major upgrade. They are at least taking a step in the right direction: some sortable stats.
I’d be interested to see who their numbers indicate is the cream of the crop; all those really low rankings for Bailey do surprise me, and they do fly in the face of how good I believe he is.
Their numbers from their latest book suggest that, last year, Shawn Springs was the best CB, while Adrian Wilson was the best Safety (by a lot). They also indicate that Champ Bailey was excellent.
Ellis Dee, on Fumble Recoveries
To pursue the anti-FO hijack, they piss me off to no end with their “it’s random so therefore meaningless” bullshit. Fumble recoveries are random because teams aren’t consistent in their ability to get them across seasons. Guess what? So are interceptions, and so are field goals, and so are any number of things in football. This in no way demonstrates that they are random or “unachieved” in any way. . .
. . . Look at Tiki Barber. The Giants have recovered half his fumbles throughout his career. (Oh no, 50%…must be RANDOM!)
The thing that some people mis-understand about FO’s stance on fumble recoveries is that it’s not based on the overall 50% recovery rate or the fact that there’s a lot of variance in the number from one year to the next. It’s based on correlation coefficients, which consistently show that there is no correlation between a team’s rate of successful recovery one year and that same success the next. Likewise, there’s no correlation between a team’s recovery rate in the first and second halves of the same year.
Just about any other football stat, despite plenty of variance, will show a meaningful positive correlation from one year to the next – teams that intercept lots of passes (or to cause or avoid lots fumbles) in Year X will tend to do the same in Year X+1, in spite of lots of inevitable exceptions. OTOH, over the past few years there has actually been a small negative correlation between inter and intra-year fumble recovery rates. It seems silly that good fumble recovery teams are likely to turn into bad recovery teams in the future – more likely there’s just no connection at all.
You can spend practice time stripping balls. You can also spend practice time recovering loose balls. These are skills that can be honed. You can have a more disciplined team that has a nose for and/or swarm to the ball. You can have guys well trained to fall on the ball instead of trying to pick it up and run. You can have your running back alter the way he holds the ball. You can improve your OL so your QB doesn’t repeatedly cough it up from getting killed all day. All these factors play a causative role in your fumble recovery production. According to FO, though, fumble recoveries are unearned because they are all and always completely random.
Two things: First, you have to seperate fumbles from fumble recoveries. Causing and avoiding fumbles are repeatable skills that show postitive correlations from one year to the next. Once the ball is on the ground, however, the numbers indicate that no NFL team is any better at recovering it than any other.
Second, a fumble recovery is indeed the product of hard work – teams presumably run loose-ball drills and instruct their players to use this or that technique to recover the ball – but it is still a non-predictive event. A team’s success (or lack of success) at recovery over one period of time tells us nothing about the likelihood of that team’s success over a subsequent period of time.
Even worse, they admit that individual fumbles aren’t random. (Behind the line, more likely the offense will recover. Downfield after the catch, more likely the defense will recover.) But when added together, these non-random events combine to produce seemingly random totals. GAH! That randomness is a limitation of your stats, not a reflection of reality!
FO accounts for a fumble’s type and location on the field when measuring the likelihood of recovery. Their position on fumble recovery has absolutely nothing to do with the fact that overall recovery rates are about 50/50 between offense and defense. If every one of an offense’s fumbles occured in the secondary, and that team lost 70% of its fumbles, FO would say that they’ve had normal recovery luck, since that’s about how often fumbles downfield are recovered by the defense.
Other “non-predictive events” include blocked kicks, as well as fumble and interception returns. In these cases, it’s not that no team is better than any other, it’s that the events are so rare and, in the case of turnover returns, so dependent on circumstances beyond the team’s control. Because of these factors, whatever differentiation in skill that exists is unable to break through all the random noise, and is thus useless for making predictions about the future.