Well, there was this nice little con job pulled by aliens in a Twilight Zone episode titled “To Serve Man”.
It’s not a movie and not a novel, but the episode of Community Conspiracy Theories and Interior Design starts off with Jeff being a conman faking a class he took, only to discover by the end there are a series of cons being played. I just watched it again and still can’t really follow the ending.
For novels …
Well, there’s The Talented Mr. Ripley, and sequels. I’m not quite sure it fits – Ripley is a sociopathic liar and killer who strikes out at people who realize he’s living a fraudulaent like,
My as to what the OP is looking for I think would be the Dortmunder series by Donald Westlake, John Dortmunder is a criminal mastermind with really bad luck.
A similar series The Burglar … by Lawrence Block. Bernie is an antiquarian book dealer by day and a cat burglar by night, and much comedy ensues.
Both these series have movies made of one or more of their books.
I enjoyed what I have seen of Hustle, but I found Leverage to be much more enjoyable. I just looked the characters better I think.
My recommendation is The Art of the Steal. Kurt Russell and Matt Dillon, and a very rewarding movie about a long con that I loved. I saw it streaming on Netflix and I believe it is still available there.
Con men and magicians share much in common. Therefore, I nominate The Illusionist. And I should include the story it was based on, Eisenheim the Illusionist.