You may be right that segregation in actual fact hurt them, but the large majority of actual labor unions throughout history (right up until the Civil Rights era) argued the opposite.
You’re making something up that, well, I won’t say doesn’t exist at all, but is far from a common or mainstream view among “the left”. A great example of this is Steve Jobs, who of course founded Apple, a favorite company among the left, who (as the stereotype goes) all took their iPads to OWS protests. And he was insanely crazily rich. But did people complain about it? Very few, because, for the most part, people don’t resent someone who started with relatively little, had ideas and talents, worked hard to create obvious items of value to society, and got rich for it. But Steve Jobs and the money he made is very different from Wall Street, Enron, whoever got rich getting bailed out in the bailout, etc. So even though Steve Jobs was obviously one of the 1% (heck, the 0.0001%), he’s not who the anger/envy is directed at.
(Sports and entertainment figures are yet another issue, and I think a lot of people, including myself, are somewhat bemused by the amount of money they make, but most people don’t get mad at THEM for it. Sure, Albert Pujols gets paid an amount of money which one would be hardpressed to really objectively defend given what he contributes to society, but heck, that’s not HIS fault, and he seems like a nice guy, so more power to him…)