Though I wouldn’t suggest he’s the greatest, I wanted to at least throw G. K. Chesterton’s name into the thread (for his Father Brown stories as well as others, such as those collected in The Paradoxes of Mr Pond or The Man Who Knew Too Much). Sure, he wrote mainly short stories; and sure, his stories are contrived and artificial, with a love of the bizarre and seemingly paradoxical. But he was great at coming up with mystery ideas, and I haven’t found anyone who writes mystery stories quite like his.
Classic who-done-its has the be Dorothy L Sayers and Ngaio Marsh. Honourable mentions to Emma Latham (John Putnam Thatcher) and Ellis Peters (Brother Cadfael)
On thriller side I’d go for Ed McBain and John D MacDonald.
I nominate G. K. Chesterton, who not only wrote about a hundred excellent stories, but also the incomparable essay A Defense of Detective Stories.
And I also give a shout-out to Zelda Popkin, not because she wrote particularly good mystery novels, but because she’s my great grandmother.
I like Agatha Christie and Arthur Conan Doyle the best.
Rex Stout.
In an Agatha Christie story, the first character introduced, who couldn’t have done it, did it.
While there are several other names mentioned in the thread who I particularly like, (James, Stout, Hammett, Parker) I feel the need to add Ross MacDonald and his intricate Freudian plots to the list.
I’ll second John Dickson Carr
Also Poe, Hammett, Doyle
Rex Stout.
Many people wrote great cozy mysteries; others wrote great hard-boileds. Stout did both.
Many created singularly unique super-genius detectives; others went with smart alecky, every-man gumshoes. Stout did both.
Many had aforementioned super-genius detectives but needed to create an all-admiring, much-lesser-brained sidekick to prop him up. Stout’s bench was far deeper, and made both his lead characters require one another; in fact, the so-called sidekick is really a co-lead. (I’d certainly have read a mystery with just Archie and no Wolfe – see In the Best Families for an example where we don’t see Wolfe for quite a long time.) Even the second and third tier cast members have plenty of wit (Saul Panzer and Lily Rowan ain’t slouches) and even the antagonistic police–usually depicted as bumbling Keystone Kops, hopelessly corrupt, or fawning admirers of the geniuses–aren’t always shown up; Inspector Cramer and Sgt. Purley Stebbins are treated with respect even when in opposition to our heroes. Stout respected his audience and thus his characters. Even though his wasn’t the best or well-shaded characterization by a long shot, his players, even the stock characters among them, often displayed surprising depth or quirks.
The Nero Wolfe/Archie Goodwin team lasted from the Depression and age of the elegant locked-room mysteries to the cynical Watergate era, taking on everyone from socialites to communists to Red-baiters to industrialists to corrupt cops to beatniks to J. Edgar Hoover himself. With 70+ stories under Wolfe’s strained belt buckle, I agree that some of the mysteries themselves are a bit forgettable (and it’s funny that Before Midnight is mentioned above, because that’s my least favorite and possibly the least memorable of all!). But pound for pound, dollar for dollar, I think Stout is the most balanced, well-crafted, re-readable, clever, and just darn fun mystery author of 'em all.
Also I’ll 54th Agatha Christie, because what she lacked in real characters she more than made up for in ingenuity. And Robert B. Parker, Robert Barnard, Ellis Peters, Ellery Queen, Ngaio Marsh, and Ed McBain. Has Margery Alligham been mentioned? Let’s toss her in too. 
I’ll throw in one who’s probably not considered a mystery author, but whose plots do have elements of the ‘caper’ genre: P.G. Wodehouse.
(Can I throw in a negative vote? I’m virulently anti-Sayers. I detest smug Wimsey and his fucking magistra Vane. I wish she’d answered “No, it doesn’t piacet me at all, you pretentious twit!” :D)
Raymond Chandler.
For me, the title of “Greatest mystery writer of all time” has to be one of Doyle, Christie, or Hammett. But I’ve been burning my way through Earl Derr Biggers’ Charlie Chan books lately and they’re all top-notch, so I feel compelled to nominate him as well.
Biggers lays out the clues so the reader can solve the mystery, or at least come pretty close. He also throws out a bunch of entertaining red herrings to keep you guessing. The only drawback is that there are only six Charlie Chan novels and I’ve already finished five of them.
Earle Stanley Gardner, Winner of the 1962 Edgar Grand Master Award, has always been a favorite. John D. MacDonald is my nominee for the greatest. Rex Stout would have to be runner up.
I’ll probably go with P.D. James, but I also love Robert Goddard. If we’re allowed to go with crime writers, however, I’ll take Donald Westlake writing as Richard Stark (the “Parker” novels, especially the first series, from the late fifties to the early seventies). A few of those actually do read as mysteries, in that Parker has to figure out what went badly wrong with his heist.
John Farrow (Trevor Ferguson) also produced a couple of good crime/mystery novels (the “Ice” novels).
Not a nomination, but Umberto Eco basically inadvertently invented the “Medieval Mystery” subgenre.
I used to wonder if Brother Cadfael was a sort of simplified and more accessible version of Brother William, but according to their copyright dates, Ellis Peters was writing her monkish mysteries before Eco’s novel came out.
(I won’t nominate Peters, however, since she also does that irksome nice-young-couple-above-suspicion thing.)
I was thinking it was strange that no one had nominated Ross MacDonald. His “The Underground Man” is a classic. “The Way Some People Die” is also, IMHO, really powerful.
Huh. I just assumed that that was the case. Ignorance fought, thanks.
Just a bump to see if there are any more nominations before I start the poll, probably on Sunday.
Nicholas Blake aka Cecil Day-Lewis father of Daniel Day-Lewis.
Robert Bernard
Some favorites that may have been overlooked.
Lawrence Block
Michael Connelly
Douglas Preston/Lincoln Child
Lawrence Sanders
First, another nomination for G.K. Chesterton. He is really one of a kind. The other four are obvious:
Arthur Conan Doyle
Raymond Chandler
Dashiell Hammett
Agatha Christie