I always liked “By Jove!” Maybe you can bring it back in fashion.
I prefer “Holy Terror, Batman!” (and variants thereof)
“For the love of Jeff” from The Simpsons always does good in a pinch
“Cthulhu ftaghn !”
Or,“By that imaginary God bastard !”
Or just throw back your head and let out a Klingon Death Scream. 
I think “bless you” is stupid. No one says “bless you” if you burp or fart, yet we hold on to it for sneezes.
Cheese Crisis!!
My Nanna always says “Oh my Godfather!”
Presumably, he’s dead, so no one gets hurt.
She has also been known to exclaim “Sugar!”
dpwned!!!1!
Personally I have no problem in taking the name of a being I don’t believe exists. It’s part of my culture and language. “thank Christ”, “for God’s sake”, whatever. Don’t care. Though for added sweariness, you can substitute “fuck” for the deity: “thank fuck”, “for fuck’s sake”, “what in the name of fuck?” etc.
Now come on, people. We have to parse the nuances here before we can offer him some substitute expressions.
For instance, there’s the college freshman’s “Oh god” on realizing she left her book in the car, and ohmigod it’s a five minute trek away. That would best be translated “Sheesh” or “For crying out loud.”
“What in god’s name” brings to mind an angry forty year old father facing his teenager’s latest irritating but not extremely threatening aberrance of behavior. In this case “What the hell”, if you don’t mind a distantly religion-connected word, or “for the love of Mike, what do you think you’re doing”, or “what the fuck!”, as several have offered here. “Frick” makes a useful substitute for “fuck” where a degree of cleanliness is required. I like “What the frickin’ hell is going on here!”. When I lived in New York City, I observed that “frickin’” was the only word New Yorkers spoke slowly - as in, "So I goes to him and I says right to his face, “What the [slow down] frickin’ hell do you think you’re doing driving that taxi right over my foot???”
But then there is a third situation in which the use of “oh god” may be frequently heard, often in repetitive phrases. What’s a poor atheist to do?
For these I suggest “Yes yes yes yes yes yes yes right there right there oh”, or, for the eloquent, “That makes me so hot I can hardly stand it”.
Oh Good Grief! always works.
What On Earth?! Is a good substitute.
[QUOTE=gabriela]
But then there is a third situation in which the use of “oh god” may be frequently heard, often in repetitive phrases. What’s a poor atheist to do?
This is where one word steps up. BUGGER! (Bugger, bugger, bugggggggggger!)
I like “crappers.”
Oh, crappers
What the crappers
You stupid crappers
I harken back to The 2010 Year-Old Man, a comedy album by Carl Reiner and Mel Brooks. The 2010 year old man had a caveman buddy named “Phil.” He was the strongest, baddest caveman around. Then one day he got eaten (or something). The 2010 year-old man says, “There’s somethin’ bigger than Phil!”
Yes because it would sound stupid to say bless you if you burp or fart, is it reserved for sneezes 
I’m going to start saying “bless you” for farts and burps, and see if it really sounds stupid.
burps and farts aren’t supersonic carriers of icky microbes that can kill you. although i am taken with the idea of blessing people who pass wind. this could be a fun new trend.
i tend to go with:
oh my.
heavens to betsy.
good heavens.
oh dear.
good grief!
oh wonder unutterable!
By Toutatis!
Thundering typhoons!
Blistering barnacles!
There was a REASON people said “bless you” for sneezes. Those reasons never applied to burps or farts. Granted the Plague is not so much a problem now, but still. Burps and farts were never a sign of anything greater than a full stomach. (Or IBS, but then who believes in THAT?? That’s way out there with that God fellow…)
Oh, and how 'bout:
Great Googelly Moogelly!
Oh, for the Love of Mike!
Oh, Cripes! (That’s short for Jesus Cripes, Son of Gosh)
Always wondered why my favorite never really caught on …
Lucy
[Exits stage left]
Jeepers!
Golly!
My goodness! (or Goodness gracious [me]!)
Cripes!
Blast!
And don’t forget:
Feck!
Drink!
Girls!
If a priest can say it, why can’t you? 