I’m sure I’ll think of one later, but the first one that comes to my mind would be Robert Mitchum in “Ryan’s Daughter” because he was one of the most virile men in Hollywood and plays a cuckold, and does a good, convincing job.
Someone elsewhere mentioned Andy Griffith for “A Face in the Crowd”, but that was his first movie, so he didn’t have a “type”
Eric Laneuville, who played the likeable orderly (and later Physician’s Assistant) Luther Hawkins on St Elsewhere, was also a stone-cold killer on Hill Street Blues. Aside from Henry Fonda in One Upon a Time in the West, I’ve never seen an actor go from playing a good guy to a bad guy so convincingly.
One of the reasons that I’ve long liked Brad Pitt as an actor is because he may be renown for his looks but he does not always play the pretty lead, and is in fact good as the quirky weirdo, a la 12 Monkeys
Fred MacMurray. Most of his early career he played happy-go-lucky romantic leads. Then Billy Wilder cast him in Double Indemnity and he was a hard-nosed insurance agent and film noir killer.
MacMurray went back to genial comedies, but Wilder cast him again as Mr. Sheldrake, the smarmy boss having an affair with Shirley MacLaine.
These were both completely against type. Evidently MacMurray didn’t like that sort of role and he quickly went back to being a genial good guy. But those two performances are among the most powerful in film. Wilder evidently saw that MacMurray’s type made a great heel.
In Lost in Space, Gary Oldman does not play a typical Gary Oldman-style psychopath. He plays Dr. Smith. In several scenes, if you close your eyes, you can visualize Jonathan Harris playing the scene, because Oldman mimics Harris’ vocal mannerisms perfectly.
Mulholland Falls is a murder mystery with several actors playing against type. The more stereotypes you have about the actors, the less likely you are to spot the real killer.
Tim Allen in Red Belt, written & directed by David Mamet. Rather than his usual improv wacky comedy, he was playing a very boring, straight-laced action star, kind of a Kevin Costner/Bruce Willis kind of guy with no sense of humor. It made for a funny bit in the “making of” part of the dvd:
Allen: “I’m used to just taking the script and kind of improvising my way through a scene.”
Mamet: “If he can improvise better than I can write, I should go home.”
In the 1990s, John Ritter starred in a made-for-TV movie about a man who’s going through a divorce that is amicable, until his STBX starts dating a guy who the kids instantly dislike, and he turns out to be an unemployed psycho stalker.
The unemployed psycho stalker is played by Henry Winkler - and he was PERFECTLY cast for that role!
Buddy “Jed Clampett” Ebsen played the mastermind behind a gang of traveler’s check thieves in another episode of Five-O. It started out with the cold-blooded murder of the drivers of an armored car.