Assuming, of course, it’s free of microbes, bacteria, and other assorted microscopic beasties?
Well it might have dangerous non-biological contaminants (water dissolves a lot of stuff), but other than that, it would be fine.
Why wouldn’t it be? It’s the same chemical compound. As long as it’s pure H2O without potentially toxic dissolved minerals or compounds it should be fine.
No possible single answer. Sure, it would be drinkable if it was pure H[sub]2[/sub]O. But the planet could have a suboptimal amount of other things for human health. Perclorates. Lead. Mercury. Arsenic. Ammonia. Any non-organic nasty substance that can firm on Earth can form on another planet.
Oh, and I didn’t think to mention that the vast majority of water on Earth isn’t drinkable.
Only with a very very very long straw.
Very good! 
It is virtually assured that there will be no “microbes, bacteria, and other assorted microscopic beasties” that are in any way compatible with Earth life beyond being able to break down simple carbohydrates or simple lipids. Any alien microorganisms are highly unlikely to be able to colonize a human being beyond a topical infestation.
As Colibri says, pure H[SUB]2[/SUB]O is purified of all toxic elements and compounds, it should be potable regardless of the source.
Stranger
“Liquid water”?
What if conditions favored a high percentage of Deuterium?
I’ll have my Antarean brandy on the rocks, please.
Heavy water is only mildly toxic.
So you’d need to live on it for a little while before it became a problem.
Don’t worry–if you have a deuterium problem, these guys can help you. (Coincidentally, run entirely by dodgy dudes.)
Or ice-nine?
(I know it’s from a book that I haven’t read, but am familiar with the concept.)
Evidently not, since ice-9 isn’t liquid, as specified in the OP.
Also, fictional water is not really drinkable.
In Arthur C. Clarke’s book 2061; Odyssey Three, an astronaut drinks water made from melted comet ice. The only result is a case of diarrhea.
I personally wouldn’t drink untreated comet ice. It’s known to harbor other, possibly toxic chemical compounds, including ammonia and (IIRC) cyanides. But Clarke evidently thought they’d be too diluted to cause problems.
Oh, it’s drinkable. It’s just not good for long-term sustainment of life.
“Here: if you have a planet, and I have a planet, and I have a straw — There it is. That’s a straw, see? Watch it. — my straw reaches across the stars, and starts to drink your planet, I drink your planet! Thhhp, I drink it up!”
I can already see bottled Space Water on grocery store shelves and in hipster cafes.
For humans. For animals with better salt-excretion systems, like marine mammals, seabirds, sea turtles, and fish, it’s fine. (Although some of them don’t drink seawater much, getting most of the water they need from their food.)
Straws are bad for the planet