They do taste different. A stock starting with browned meat (and often roasted vegetables, as well) has a stronger, deeper, well, “browned” roasted flavor to it. For a non-meat analogy, it’s like the difference between toast and warm bread.
Now, that doesn’t mean one is better than the other. They’re just different. Typically, I actually do not brown my meat when making broth, as most of the soups I make I like to have a more “neutral” background flavor on which the soup is built. So for vegetable soups, bean soups, straight-up chicken soup, that sort of thing, I like a light, delicate broth.
For stronger soups where I really want to play up those dark, toasty, caramelized, maillard-type meaty flavors, then, yes, I’ll brown my meat. Something like beef stew, beef and barley soup, oxtail soup, French onion soup (although I usually prefer this one with a light stock, too, depending on if I want a really “beefy” version of it rather than an “oniony” version of it), etc.
I don’t know what you mean about not skimming browned meat. You still have to skim if you want a clean broth. Plenty of gunk still floats up, though perhaps not as much. One trick which I don’t do too often (but every so often I do) is to blanch the meat. So you take your meat, throw it in cold water to cover, bring to boil; boil for about 10 minutes. Take out the meat; dump the water, wash the meat, and start with your stock/soup recipe as normal. Most of the gunk will now be gone.
I, personally, don’t usually have an issue with too much protein scum or gunk. I’ll skim at some point, but it’s not a big deal to do.