I’ll be bold enough to say that the dialogue is a large part of why many people do take to Joss Whedon’s work. Hell, I know that’s why first drew my attention to Buffy all those years back. It isn’t the only thing he has going for him, though.
Joss can take a fantastical concept–Vampires, for instance, or a future in another solar system–and use interesting characters to make it feel as real and as vivid as our own. Now, as a caveat, this isn’t going to make it interesting to everyone; I don’t think there’s anything the entire world would agree on. And, even given interesting characters, some people just prefer more reality-based TV (not reality TV; shows that take place in the more-or-less real world).
Joss can also tell an interesting story. He’s very much a big-picture writer; even his one-off episodes (which are usually not written by him directly) tie into a larger theme. This makes his work very interesting, IMHO.
There are some drawbacks. Joss often makes decisions that polarize his fanbase in order to get a reaction–in his own words, giving the audience what it “needs”, not what it wants. There are a lot of people who resent investing time and emotion into these characters, only to see them die (one particular instance got a lot of flack a couple of years ago). To them, the deaths/shocking moments seem cheap, not deep.
Also, not everyone “gets” Joss all of the time, the same way that not everyone “gets” Monty Python all of the time, and, for those who don’t fall into the “get” camp, the zingers and kooky phrases can get really annoying, really fast. It doesn’t help that fans of Whedon shows tend to quote them a lot, which I’m sure gets old, too. Kind of like Monty Python.
Now, if you’ll excuse me, I’ll be in my bunk.