Just a note on your last point (emphasis mine): the men on base would effect the ERA’s of the pitcher(s) who allowed them to reach, whose ERA’s would likely improve if they were relieved by the best pitcher possible. The “closers’” ERA would still correlate to the quality of the batters they actually face +/- a fudge factor for the added stress of actually being inserted into tenuous situations.
Snipping this bit and chiming in to point out that David Cone won 20 games in 1998. We all remember David Wells because he pitched a perfect game, won 18 games and was overall amazing (lost only 4 games, only walked 29 batters in 214 IP) in 1998, but Cone was a bona fide ace that year.
The key point still stands though: the 1998 Yankees were an incredibly balanced team. Well-managed, no egos, bunch of ham-and-eggers, and most refreshing to many Yankees fans was that the Orioles had a higher payroll that season. 
Just looking at his K rate from year to year that looks purely like a fluke. He’s struck out more men than that every other year of his career, usually a LOT more. He only pitched 61 innings in 1998 (a career low) so it’s just random chance more than anything else.
Rivera only has one pitch but his command of it is as good as I’ve ever seen. Rarely does he miss his target badly. Lots of pitchers could get by with just a fastball if their control is good, and indeed some have. Really solid control men with sharp two-seamers like Bret Saberhagen, Roy Halladay, or Curt Schilling have been known to go long stretches where they just threw 90-95% fastballs. If you can get it over 90 and hit the right spots, location can be just as deceiving to the hitter as changing speeds. If you don’t have the control to hit your target, though, the ball will end up over the left field fence.
Other pitchers have been successful with just one pitch as well; I saw Mike Timlin throw a lot of innings where he really didn’t use anything but his slider. When he had it controlled and moving nicely he’d blow through three guys faster than you could get to the kitchen to get a beer.
Interestingly, Rivera earned the closer’s job as a result of his performance over the 1996 season, a year before he learned the cutter. Back then he just threw fastballs and the occasional slider.
Good point, I was just trying to point out that their numbers would change and I overstated my position. Stepping back to look at it again, I can’t see why their ERA’s would go down, but I can see many reasons they’d go up (pitching with runners on, a rise in intentional walks to reinstate the force, etc.)
jimmycolorado of course you’re right about Cone; I got the team numbers from Bill James’ book and mistook the Win Shares column for actual Wins. I will say that Cone’s ERA was 3.55, and Hernandez led the team at 3.13 with only 141 IP.
I don’t think anyone’s going to bring up any of those Yankees as a prototypical Ace in the Big Unit, Rocket, Maddux sense of the word.
Cone was very close. He wouldn’t be a terrible choice for the Hall of Fame.
Though checking baseball reference, he was pretty close to the end of his career by 98.
Wells also had 8 complete games and 5 shutouts that year, so they really did have co-aces. That team had a great bullpen and a deep bench, and for good measure Shane Spencer came up and hit 10 homers in 67 at-bats (including 3 grand slams) that September. What a team.