My friend sometimes exclaims ‘Oh, Christ on a pony!’ I’ve also heard ‘… on a pogo stick!’
It’s always appealed to me because of its absurdity.
Why do we say it?
I know that many non-obscene expletives are euphimistic for dirty things (‘darn,’ ‘flippin’,’ ‘crikey’ etc) but I can’t think of a single reason for this one.
Any ideas?
I’ve never heard the first one. Nor the second one, for that matter, although it might explain the phrase “Jumpin’ Jesus Christ!” 
I have heard the phrase “Christ on a crutch”, but that’s probably for the alliterative value and doesn’t actually refer to our fractured lord (should you believe in the guy’s divinity in the first place).
I suspect the phrases you are referring to are done purely for the comedic effect of the phrase, although “Christ on a pony” doesn’t exactly crack me up.
Just seems strange that I’ve heard two, from different places (I think I may have read one on these boards, actually), and today I read something about ‘You waltz in here like Jesus on a donkey’ which sounds almost similar.
Hm.
Jesus did ride into Jerusalem on a donkey just days before his crucifixion according to the gospels so I’m not sure why that is so absurd. I think the phrase implies arrogance rather than just trying to be funny. “Jesus H. Christ on a fucking rubber crutch,” said by Tommy Lee Jones in Natural Born Killers, is more for humor of the phrase.