Here’s my favorite baker’s dozen of artists whom I have complete (or 98% near) catalogues of and whom can do no wrong:
Siouxsie & The Banshees
Garbage
The Cardigans
Sarah McLachlan
Portishead
Led Zepplin
Hammerbox
Goodness
Toad The Wet Sprocket
Alice In Chains
Soundgarden
Tool
Jane’s Addiction
My shameful admission:
Blondie. My knowledge of Blondie music is vast. Encyclopedic, even.
Nick Drake
bob hund
Radiohead
Primal Scream
Sonic Youth
Velvet Underground
Belle & Sebastian
Air Supply
John Denver
My Life Story
I’ve nearly 200 CDs from various artists and genres in my collection. Can’t even name all of those… But play me a few bars from a JD song and I can name it… Sad I know… I know.
My list is short, too…VERY short…
The EAGLES
Thank you, thank you very much…
I assume you mean every song with no chance of being stumped by one obsure tune on an obscure album somewhere. In that case, Steely Dan.
Hmm… Off the top of my head:
Metallica
Faith No More
Alice in Chains
Iron Maiden
Joy Division
Nick Cave
The Police
Radiohead
Nirvana
UnwrittenNocturne is in an EBM band, and interface2x like Depeche, NIN, and Mesh. I may possibly be in love.
I’m working on VNV Nation. I may become an Icon of Coil addict, too. I usually fall in love with individual songs rather than entire albums.
Let’s see…
The Smashing Pumpkins - which includes a lot of weird unreleased stuff
Sublime
The White Stripes - and I feel like I need to say I was in love with them before “White Blood Cells” their breakthrough, came out
Nine Inch Nails
Weezer
Bob Marley - though I must say, his earliest stuff isn’t worth remembering
Mine is worst. ABBA. I know every ABBA Song, including the non-English ones.
The Indigo Girls
Sublime has a lot of “weird and unreleased” stuff. Are you sure you have all your bases covered? 
Yeah, I’ve tried VNV, but I still don’t have Advance and Follow and I’m not paying the $200 it usually ends up at on Ebay. Every other song they’ve released since then I know inside out (with the exception of the new song on the Honour 2003 single!).
As for Icon of Coil, they’re pretty hit and miss with me. I have both full lengths in addition to Seren and One Nation Under Beat (which is primarily filler material). Their cover of Pet Shop Boys’ “New York City Boy” rocks, though.
Oh, and if Unwritten Nocturne gets to do it, then I will to - I know all of the songs from my former band Interface 2x. Even the ones that sucked. Which was most of them.
Icon opened for VNV at the Toronto concert.
Do you like Wolfsheim? Sorry, I shouldn’t hijack. Maybe we need our own EMB/goth thread.
Might as well throw my two cents in:
The Alkaline trio
Botch
The Dillinger Escape Plan
The Who
Less Than Jake (Except for the newest album)
Fugazi
Refused (Except for their really early stuff)
Weezer
Harvey Danger
and Sunnyday Real-estate
I have encyclopedic knowledge of Operation Ivy (including the E.P.s) and Green Day. I know all of Rancid’s stuff except their first record.
Hüsker Dü (Including sub-categories Bob Mould and Sugar)
Jane’s Addiction
The Police
Sisters of Mercy
Kate Bush (pre-Red Shoes)
Peter Gabriel
Butthole Surfers (pre-Weird Revolution)
The Who
and…um…
Duran Duran (pre-Notorious)
Oh, and fallapart … any time you wanna do a mano-a-mano Who Trivia Smackdown, just lemme know little bro’.
The Cure
The Smiths
(most of ) New Order
[notice a trend here…?]
My Final Hijack Until There’s a New Thread 
I can’t really get into Wolfsheim - it’s the vocals. I’m strangely concerned with vocals in the music I listen to. If I don’t like them, everything the band does is screwed. Icon of Coil just barely squeaks by for me. Admittedly, I’ve only heard a couple of Wolfsheim tracks, so I might just need to give them more of a chance.
p.s. Last year when VNV came through Chicago, Assemblage 23 and Haujobb opened for them. It was awesome - at the end of the VNV set, they called A23 up onstage and they performed A23’s song “Naked” together.
But, do you know the complete lyric to the Ballad of Spiro Agnew?