If you look at photos of men outside say 80 years ago they all wore hats without exception (e.g. see http://library.bloomu.edu/Archives/football/crowd.JPG ). Possibly no factual answer, but would I have been arrested, harrassed etc for not wearing one?
you would’ve been immediately socially outcasted and forced to live a life of solitude…
Thank god for the internet! 
One of my favorite family stories is about my grandfather, the rebel. He was an engineer; the building things kind, not the train kind.
It seems that when he was fresh out of college he and his buddies decided to go to work hatless. This would show how “modern” they were.
Their older coworkers ended up taking up a collection for the younger men so that they could afford to buy hats.
So at the very least, you would be hatted by your elders, who felt bad for your economic disadvantage. <)
From what I can tell, it’s pretty much the social equivalent of going around shirtless today (for a man, of course). There are certain, very informal situations where it might be OK or even expected, but most of the time it would just shock people to be able to see the top of your head. They might not have the nerve to say something to your face, but there would be lots of whispers, and you would not be invited to hang out in polite company.
As well it should be. I love hats on men. Bring back the bowlers, I say!
Without a hat? Oh, the danger! :eek:
http://www.ilkley.org/iguide/baht.htm
So there! 
I have often wondered the same thing - (re. hats, that is, not worms or ducks. )Well, rather less sensibly, I often wondered whether people were somehow so much colder that hat-wearing was actually needed, but that didn’t make a complete lot of sense.
I liek the tale of the older workers helping the “unfortunate” younger ones to buy hats!
(btw - that is not the complete song, but I am very sure it is out of copyright anyway)
You’d have been seen as odd, just as if you wore a fedora today. 
I don’t care. I wear 'em anyway.
I tend to agree with AskNott. Personally, I don’t understand why so few people wear hats these days. I usually wear a Stetson. Keeps the sun out of my eyes and off the back of my neck, keeps my head and fact dry(er) in the rain, keeps my head warm in the winter, and so forth. I just don’t feel as comfortable outside when I’m hatless.
While I agree that I like how hats look on me and on other men, I must say that you’d be teased mercilessly for being a hick if you wore a cowboy hat like a Stetson in Saskatchewan!
I hate having some so restrictive and tight on my head, personally. Even when baseball caps were extremely popular a decade or so ago (are they still? I never notice them much anymore), I never wore them.
I’m just now getting used to the feeling of wearing a toque and I wear it for purely thermal reasons. If it’s not cool out, it doesn’t get worn.
While shelving in the Fashion section at Border’s I found a book about the decline of the hat industry. Evidently hat makers blame JFK for the hat’s loss of popularity. He was inaugerated without a hat and fashion folklore has that as a trend setting moment.
The obligatory snopes link
Another Saskatchewanian, eh?
If you don’t have the obligatory farmer/trucker style hat or a ball cap, you are an outcast here.
I also love hats, something I have in common with my brothers and my father. Baseball caps in the summer to keep the sun from my eyes; other styles in winter (I don’t like knit caps) to keep my head warm. My favorite winter hat is a fleece hat with one of those tiny brims and a crown that snaps to it. On cold days, I can just put on my hat and feel warm enough to go outside.
Heh heh. Me, too.
I get sideways looks, but nothing beyond that… with one exception; a friend of a friend who’s not known for his social skills saw my grey fedora and guilelessly said, “Oh, cool hat - do you think they’re going to come back in style?”
It would depend entirely on your social circle, or caste if you will.
Being of the lowest caste, I never wore a hat and none of my circle did either. In fact I didn’t even own a dress hat. When I first went to work on the desert I wore a fake topee. We called it a Frank Buck hat for protection from the sun. However I soon abandoned that and ran around hatless for 30 years.
The expected happened and I began to develop skin lesions. I now wear a floppy, broad brimmen bucket hat, long sleeved shirts and long pants at all times when outdoors.
Stetson makes fedoras too, y’know.
I don’t have an answer for the OP, but I’m a Pork Pie hat guy myself.
As a multi skin cancer sufferer, hats are an integral part of my life, as are long-sleeved shirts and long pants whenever I go outdoors. Both my grandfathers were inveterate hat wearers, though my father and uncles were not.
thinking aboout it, it would probably be similar to someone turning up to work without a tie - it would be cause for gossip and snide comments and wondering what they were trying to prove.
Possibly in the UK - here, I don’t think anyone even looks twice if someone isn’t wearing a tie. Our business wear is a little more relaxed than the UK’s I think.