I’ve always wondered about that too. In reality, there’s no such thing as ice cold water. Ice cold water can’t be water.
Maybe, one of these days, when you have a few hours of spare time and your lips aren’t too tired, you could try reading some of the OTHER posts in this thread.
Anyone want some water warm ice?
It’s not an oxymoron, it’s an example of hyperbole.
That was my thought. “Icy hot water” would be more of an oxymoron. Water and ice aren’t contradicting terms, since ice is a form of water, not the opposite of it
Whatever is considered cool in Vegas, stays cool in Vegas! 
Why is it hot as hell and cold as hell? Make up your minds! And what’s up with airplane food?
And if it such a terrible place why is there always a hell-of-a-lot-a stuff.
Ah, come on. You expect English to be LOGICAL? 
Ice cold water is an expression, pure and simple, just like “lose your head”, “put the word out”, etc.
J.
All of a sudden, my father’s voice popped into my head:
“They need ice water, and you know where.”
Never knew what he meant by the statement but I presume he meant hell.