I use it, and have also wondered whether, semantically, it makes any sense. It always seems to be in contexts like this:
Person: This wine is horrible.
Me: Yeah, no, I don’t like it either.
So the “Yeah” is agreeing with them, and the “no” is there because I am about to make a negative assertion. I don’t know, something like that.
I say a lot in the same context given in the OP. And I hate myself every time.
Think of it like this
Director: And I was thinking that maybe we could hire Johnny Depp for the part?
Producer: Oh that’s an interesting thought, but we were going to go with Tom Cruise.
This is the same as
Director: And I was thinking that maybe we could hire Johnny Depp for the part?
Producer: Yeah no we were going to go with Tom Cruise.
Also typing this out made me realise that it is actually “yeah no” and not “yeah, no”.
Huh, I thought it was a Kiwi tic. I know I say it all the time, although I never hear myself say it until someone points it out.
This is kind of like “I know, right?” It’s just an agreeable noise you make to fill in the nanosecond before you know exactly what you want to say.
It’s not like that. And neither is “I know, right?” IME.
“I know, right?” occurs in two forms to my ear. The less-expressive form just means “yes,” or at most “yes, for sure,” strictly as a response.
The more-expressive form is a speaker’s interjection within or following something they are saying, as an acknowledgment that a listener has understood (and conveyed by laughter/expression/body language/whatever) the particularly outrageous/ridiculous/wonderful/whatever nature of something. It serves to simultaneously emphasize, and share a moment of marvel at, what was said before, like combining an emphatic “for real!” with an amazed “can you believe it?”
I’ve heard this fairly often, mostly from women in their 20s. The “yeah” means “I understand what you said” and the “no” means “But you are wrong/I disagree.” The example in the OP seems off to me, it would make more sense if she were disagreeing that the skiers lip balm was preferable:
Me: I know, that’s why I don’t like those. I use this skiers lip balm instead, it’s easier to use.
Her: Yeah, no. That stuff tastes gross.
I use it in both of these contexts and meanings.
Person A: “Don’t you think we should [implement some idiotic policy.]”
Me: “Yeah…no.”
or
Person A: “Don’t you think we should [implement some policy that’s not idiotic, just not the best idea.]”
Me: “Yeah [to imply “I hear you and hear your rationale”], no [because of this reason].”
I think it is normal to answer a two part question with “Yes and no.” Then go on to explain.
But I have never heard the “Yeah, no…” phrasing of an answer.
I noticed that I sometimes say this, and I think it’s in contexts like these. It’s not sarcastic. And I can’t remember ever hearing anyone else say it, but maybe it’s just not noteworthy.
Just as a footnote, there is actually a word for this in German:
Njein.
Directly translated as “No Yes” Combination of “nein” (no) and “ja” (yes).
I’m in New York – originally from New Jersey – and I think use that expression fairly frequently. Might it be a regional thing?
I’ve often wondered myself why I say it. I use it to indicate agreement, so the “no” part makes no sense. I think ThelmaLou got it right in post #26 – it’s just a verbal tic. At least it is for me.
I only use it as a sarcastic response to a dumb suggestion someone just made.
If I heard it in the context that the OP talks about and it was a friend doing it, I would abuse them horribly until they stopped. I’ve been married for six years so it’s hard to imagine a boyfriend saying it at this stage but if it were someone I was serious about, I would similarly tease and bug them until they stopped.
Exactly. 
I’m sure I’ve used it myself, in the way Lamia describes.
“Where should we go after work? I’ve heard the Red Lion is good.”
“Yeah, no, their ales weren’t very well kept last time I was there. How about the Wellington?”
My aunt says, yeah, no, so anyways…
Maybe it’s a mutation of “ya know?”
I think Ben Stiller says the “yeah no” bit in a lot of his movies.
Didn’t Phoebe say that, or variations thereof, a lot on “Friends”?
“Yeah, uh-huh —no.”
I voted for other; I went through a phase where it had rubbed off on me, compliments of Phoebe Buffay, as noted by Swallowed My Cell Phone.
Now? Not so much 