Various sci-fi story universes have this idea as part of their set-up, The Culture by Iain M. Banks for example.
As he described it there was a little of what you worry about at first but then things settle down and its pretty much like clothing fashion for bodies, forms go in and out of style, most people are generically attractive and a lot take different forms to greater or lesser extremes to stand out, as a fashion statement, or just because they feel more comfortable looking a certain way.
Some people find a body they like and stick to it, others spend their lives experimenting. One character spends likes to spend five years as a female, five years as a male and five years as a neuter for example.
Personally I think it would be one of the greatest benefits we can imagine, both for individuals and society as a whole.
Take gender for example, a person would be male or female (or both, or neither) because they want to be, not because of a fifty/fifty coin toss they have no control over.
What a person is inside would quickly become more important than their external appearance, who they are, not what they appear to be.
A lot of ‘isms’ (racism, sexism etc) would go away, of course people being people we would probably come up with new ones…
Injury, disease and genetic problems would go out the window, if we can change form there should be no problems choosing or being given a healthy body.
Anyway, yes, personally I would love for this to happen. I think it would be fascinating and a lot of fun to change my form at whim. I really don’t see any downsides, at least none that can’t be overcome.