Everyone does it. So as you grow up you just pick it up.
I know an atheist that often says 'Oh My God" what’s that all about?
I don’t take the Lord’s name in vain, out of respect for those who would find such things highly offensive. What amazes me, however, are the number of “Christians” I meet who do take the Lord’s name in vain.
Basically, as has been mentioned before, using religious words as a part of swears has become so commonplace that most people don’t even think about what they are saying. You no more have to be Christian to say, “Good Lord!” than you have to be Jewish to call somebody a putz.
Barry
I say “Oh, God” “Oh, ye Gods” “Oh, gods” and “Jesus Effing Kee-rist on a crutch”, even though I’m an athiest. It’s not like you have some kind of athiest committee that’s going to take away your athiest points if you say “Oh, god”.
It’s just a saying that is universally understood in our culture to mean you find something horrible, impressive, unique, or worthy of note.
It’s use of “God” is irrelavent to the understood meaning. When (s)he says, “Oh My God” you know the meaning, regardless of theology.
Just cuz I gave up God doesn’t mean I gave up his nifty way of explaining shit,
Steve
I swear because if there is in fact a god, I want him to be good and pissed off at me. I could really use a good smiting.
On the contrary, you’re already at -47 points. I, on the other hand, am well on my way to getting a George Carlin gravy boat!
Because ‘Madelyn Murray O’Hair on a crutch!’ just doesn’t have the same ring to it …
Damn. Maybe I can go gay and get a toaster out of the whole mess.
Atheist/Agnostic checking in. I say Jesus Christ (sometimes I say jchrist - his userid, were he to logon to our network) like it’s going out of style.
One question because I first was confused by the OP’s assumption that one’s belief or otherwise should have anything to do with swearing:
Does ‘swearing’ in English-language usage necessarily involve profanity (in the strict sense of invoking God or other sacred/anti-sacred concepts in a profane context)?
I.e.: If someone expresses their feelings in colorful language but without using any religious reference, this cannot constitute swearing?
Ummm…
Because I bashed my toe.
Because Bellamy missed another sitter (soccer)
Because the bus left before I got to the bus stop
Because I’m ranting in the Pit
Because I’m singing along with the South Park movie
Why the **** do you swear? 
As I tell people when they ask, in a tone of absolute horror, WHY? I pierced my nose:
Coz it PISSES OFF people like you!
'Nuff said.
I’m agnostic. I could say “Sacred Being or Being(s) or perhaps nothingness - I’m not really sure – on a cracker!” Or I could say “Christ on a cracker!” The latter is easier to remember and has a “k” sound alliteration (“k” being, of course, one of your funnier consonants.)
Besides, most of the profanity I use has nothing to do with religion, unless you’re in a very sick and dirty religion.
Mostly just habit. I’ve tried to cut down on god-related swears, but I learned two things: one, already mentioned, is that when you swear “god-” something or other, people know what you’re talking about, so it communicates the same meaning regardless of your personal opinions on the deity. Secondly, swearing creatively with “Jesus” or “Christ” is the most fun. There are lots of Christians, ergo there’s more of a chance for a fun reaction. My girlfriend (who is not an atheist) is fond of “Jesus Christ on a toasted bagel!” I use that sometimes - I was proud to coin “Christ on a Croissant!” today - but other times I’ll try something like “Jesus Christ with a steel dildo!”
For the same reason I say “goodbye”, a contraction of “God be with Ye”. It’s a popular convention that has lost any religious meaning.
Because they believe in the ineffable mystery of Shit and the redeeming power of Fuck, while fearing ShitBuggeryBollocksandShaft and being in awe at KnackersandPiss.