Bad Santa question

No factual answer, I guess, just opinion.

Why was Willie so sullen and quiet to Fetish-girl’s conversation when he was a raging womanizer the rest of the movie? I thought this scene, and how he reacted to her, set the tone for his character regarding women/sex and was mildly surprised to see he was successfully pursuing quite a bit of tail.

Based on my personal experience with alcoholics, his drinking/mood doesn’t seem like a viable excuse.