So do the spoken poetry bits on all the classic Moody Blues albums put you on Edge?
Since the OP specifies “good songs”, not “great songs” …
I love, love, love the drumming and bass on most of Toni Basil’s “Mickey”. I also like her vocals for much of the song. I cannot stand the ballpark organ, nor Basil’s delivery of the chorus vocals “Oh Mickey/What a pity/You don’t understand …”.
Billy Ocean’s “Loverboy” is a song I always thought would be a fantastic instrumental. Great beat, nice bass and lead guitar work. I’ve got no problem with Ocean’s voice at all … I just think this song is very strong instrumentally, and the lyrics kind of knock it down a few notches.
There are several albums that are prevented from being %100 perfect by an annoying part in one song.
One example I can think of off the top of my head is “Chester the Molester” by Sloan. The annoying part is that they repeat the title phrase over and over again. I understand that it’s meant to convey the skeeviness of the character (who might not be a molester but is definitely creepy,) but its an annoying phrase in itself, with all its sibiliants.
I’ve said this before, but the opening two or three notes to Heartbreaker off Led Zeppelin II don’t belong there. Others claim that it is correct and they simply start off the measure in a weird place, but it will always sound like a blooper to me.
I was gonna mention this. Also cut the endless "na na na na"s from Hey Jude and you’d have a pretty good song.
Well, no. They put me off Edge. 
I would like to nominate the unintelligible bridge sung in falsetto in Taxi by Harry Chapin.
You, too?
My husband almost dumped me over this, but I don’t like the weird slow middle part of “Don’t Fear the Reaper.”
It needs more cowbell.
Huh. I’ve always found that part of the song to be its emotional peak. Never thought it stuck out in any way.
I agree about the HOOGA-CHAKA in “Hooked on a Feeling.” I’m still puzzled as to what that has to do with the rest of the song.
I’m most likely going to be in the minority on this one, but I hate the piano part tacked on to the ending of the original “Layla” by Derek and the Dominoes. I’m not sure why I hate it so much, being someone who plays the piano, but it just seems very incongruous to me, and lasts too damned long. Maybe if they were out after 16 bars, I wouldn’t mind it so much, I don’t know.
I get a bit annoyed by the long wandering intros to many of the songs on The Teenage Fanclub’s Bandwagonesque. It doesn’t ruin it for me, but just doesn’t seem appropriate on what’s essentially a (very good) power pop album. Sure, do it live if you want, but on the studio recording, cut it.
Journey’s “Lovin’ Touchin’ Squeezin’” suffers from the same defect. Once the song reaches this point I always change the station. Guns n’ Roses could have also cut out the repetitious ending of “Paradise City”, same for Pearl Jam’s “Alive.”
I really like the Eels song “Trouble With Dreams”, however, after he’s done singing, and it just trails off into screechy synth junk, I get incredibly bored. So these days I just skip ahead when it gets to that part.
Worst offender in my playlist:
“Absolutely Curtains”, last song on Obscured by Clouds (Pink Floyd). When the weird atonal bleating at the end kicks in, it’s time to hit the advance to next track button. Yeesh! Pink Floyd often did weird shit on their albums, often finished up an album side with something not-very-listenable (“Seamus”, — arrooo!; “Several Species of Small Furry Animals”), but this track is way over the top. Or under the bottom or something.
I agree about the song w/r/t the album – it’s an annoying blot on the album. (And to make it even worse, it makes the similarly sounding “Day After Day” by Badfinger seem good in comparison, making me feel guilty about not liking the Floyd.)