I’ll agree with all of the humidity comments. In the Toronto/Ottawa/Montreal areas the humidity in summer is just brutal and combined with 30C (90-ish F) temperatures long sleeves and long pant legs get clammy and binding if you spend any time outside. Even at night it’s bad. And yes, there is the eye-candy aspect of it.
Google the summer humdity levels for South Carolina cities sometime, and tell me we ought to be wearing caftans. Not a friggin chance, mate.
Now, in comparison, I spent a summer in Arizona, and I wore long sleeves and long pants (had to - it was a work conference) and when I was outside, wore a hat and bought sunglasses. In a place where the locals think it’s “too humid to bear” at 3% that was a totally valid set of clothing choices, and I didn’t feel uncomfortable.
I think a culture you might find instructive is the ancient Egyptian one. Check out their clothing choices - sure some of them are shown wearing robes, but they were gauze - for all intents and purposes, sheer. There are almost as many representations of people wearing just a skirt, or gathered short pants. They had to deal with heat and humidity, and they did it just like a lot of people in the south still do - wear less.