I don’t even know how to ask this so I’ll give it a go and someone can correct my terminology later.
Someone I know of has developed a mathematical formula to compare hockey goalies across time. It’s all rather interesting but something isn’t sitting right and I need to know if it’s my lack of knowledge or if there is really a statistical problem.
Here are his terms and how they are arrived:
Marginal Save Percentage: The difference between a goalie’s actual save percentage and the league average save percentage not including the goalie in question. This is supposed to compare the goalie in question to a theoretical average goalie.
Workload: The number of shots faced by a goalie had he been on an average defensive team. This is arrived at by multipying the league average of shots faced per minute and the goalie in question’s playing time (in minutes).
Unadjusted Goals Saved: Multiply the previous two numbers together to see how many more saves the goalie would have made when compared to the league average.
The last number is then adjusted to an 82 game schedule for comparison purposes.
My problem is that the goalies are compared to the league average for each year. A good goalie playing with weaker associates throughout the league would stand out and have an artificially high ranking because of his weaker rivals.
This seems to be grading goalies based on a curve (correct terminolgy if necessary), then comparing the results against a different curve to determine who is best. While I think it is good to see how goalies compare within a given year, can this be used to realistically compare a goalie from 1974 to a goalie from 2003?
No, for the reasons you mentioned: a decent goalie with lackluster competition in a given year will come off looking better than a star goalie with star competition.
And that’s ignoring issues of rule changes that can make comparison tough.
As I see it, your last sentence is the reason for doing this whole year-relative comparison (rather than just use some easily-computed stat like saves or saves/attempts). Rule changes from year to year may make a goalie’s job easier or harder, so you can’t just count save rate. If you had some good idea of how much the rule changes affected things, you could adjust the stats directly. Since you don’t, you instead try to estimate these by using league averages as proxies for a constant level of performance. This, of course, will fail if the league average-goalie-performance is not constant, just as you noted.
If individual goalies had constant performance, you could use goalies with overlapping careers in a “ladder” to compare current goalies with past goalies (just like the cosmological distance ladder). Sadly, the errors in this method are undoubtedly too large to be useful.
But that’s OK, since I really just wanted to mention cosmology in a hockey thread.
I don’t skate over your telescope so don’t bring your stars to my rink (I wouldn’t be able to play with them anyway).
The problem seems to be that there are too many assumptions made for this formula to be anything but a WAG. It just so happens to be a WAG with math behind it rather than a scouting report. IMO, I would take a scouting report over these numbers.
I not only think that grading against a curve is appropriate for comparisons across eras, I think it’s the only meaningful way to do so. The pool of NHL goalies in any given era provides the only non-trivial standard against which performance can be compared. If Glenn Hall was 30% better than the goalies of his era, and Patrick Roy was only 20% better than his, than Hall was the better goalie by the standards of his time.
Were those standards as good as later standards? Of course not. Athletes at every position in every sport are stronger, better nourished, better conditioned, and better trained than those of even a generation ago. It’s for exactly that reason that the league curve should be used for comparison.
I do take issue with the “workload” factor, however. That assumes that a goalie who logs a save percentage 20% better than the league average in an era when teams are averaging 45 shots per game is better than a goalie who accomplishes the same in an era when teams are averaging 35 shots per game. By measuring saves as a percentage rather than a per-game number, you’ve already taken that into account.
The conclusion that you draw does not seem to follow that line of reasoning. The only comparison that I can see being allowed for is to say that Hall was further ahead of his peers than Roy was of his. It does not seem to allow for the conclusion that Hall > Roy.
Granted. However, for the same reasons that you just stated, we can not make a valid comparison between goalies of different eras. Roy may have been 20% better than his peers but, had he played in the 70’s, been 40% better than Hall and his peers. There is no way to tell. Roy’s peers were adapting to his style of play and thereby narrowing the talent* or, more specifically, statistical gap between goalies. Imitation may be the highest form of flattery but in this case it works against a stellar goalie’s rating. Further, had Roy played in the 70’s, he would have followed the 70’s conditioning programs, or lack thereof. Had Hall come to play in the 90’s, he would have been in a lot better condition. There is no way to quantify this in order to compare different eras.
*I am using the term “talent” in a broader sense as I do not consider it an athletic talent to be able to wear oversized gear and let the puck hit you without really having to move (i.e. J.S. Giguere)
We’re starting from the same premise, but coming to a different conclusion. You’re saying “Absolute comparisons across eras are impossible, so don’t try.” I’m saying, “Absolute comparisons across eras are impossible, so use a relative comparison as the next best thing.”
This seems like an appropriate place for this quote:
“As far as the laws of mathematics refer to reality, they are not certain, as far as they are certain, they do not refer to reality.” - Albert Einstein
(I wouldn’t be able to play with them anyway).