I’ve seen conflicting opinions on whether testosterone levels matter to personality and aggression, some claim yes and some claim no that it is just cultural or placebo effect. But where did the idea come from in the first place?
Men are well-known to be more aggressive than women; after all, if you look at violent crime stats, men are overwhelmingly represented, and since testosterone is seen as making a man a man, it makes sense that it would be associated with aggressive behavior. That said, this article mentions that some early studies found a reduction in aggression in castrated rodents (rats/mice) which contributed to this perception, but men given testosterone didn’t become more aggressive and it actually had the opposite effect - except in those who were sure they were given testosterone (belief bias). And here is a site that says neutering dogs doesn’t do much to reduce aggression (and in some cases may increase aggression, including in females), although it does reduce the intensity.
The most likely explanation for aggression in men is how their brains work, with measurable differences in the size of brain areas related to aggression.
It isn’t just a matter of opinion. It is backed up by thousands of studies across many different species.
Where did the idea first come from? The answer is livestock and pets. People found that they needed to castrate males of many species from dogs to horses to cows to make them controllable and less aggressive. They didn’t understand the mechanism of why it worked at the time but they still found the most effective way of suppressing testosterone production in males and using it to induce behavior changes. The same result was found in cultures where young human males were castrated to become eunuchs for social or political reasons.
World history? How many times has the known world been conquered by a women and her Amazon army?
Go to a farm and look at the difference in behavior between male and female animals.
But testosterone isn’t the only cause of aggression. Try telling a ewe that her ass looks fat in that wool dress and see what happens.
Here’s the Wikipedia entry on the history of testosterone, which overviews its discovery and the research that went into identifying “the characterization of the hormone’s effects.”
Also, if you’re interested in some anecdotes about it, I highly recommend listening to this episode of This American Life.
I know in my own case having a tumor that killed off my testosterone that when I started taking medication to reduce the tumor and the testosterone has come back I’ve gotten more aggressive. I wouldn’t say I’m looking to pick fights or anything like that, but I have noticed I’m more aggressive riding the motorcycle and things like that. I can also say that I’m more competitive at swim practice trying to beat people in the other lanes.
While I know that’s just my experience, it wouldn’t surprise me if it’s at least somewhat true.
I have to agree 100% with you. My body stopped producing testosterone because I am on long term opioid care for chronic pain. I now take testosterone replacement via weekly injections. The notable change between having testosterone and not is freaking amazing. When my body quit producing, before I was diagnosed, I lost 100% of my sex drive, gained weight, became very tired and unmotivated and I noticed that I was much less aggressive.
When they put me on replacement therapy the sex drive returned (thank God, my wife I don’t think is too overjoyed at the “new” me. I now have the sex drive of a 17 year old!), the fat left and I gained and am still gaining visibly notable muscle mass. In the beginning they were trying to get the levels right and for a while I was receiving too much testosterone. I became noticeably aggressive and easily agitated. I wouldn’t think twice about pushing a point across, even if it meant getting into a fist fight. Lucky it never went that far, but my disposition changed. I was never the aggressive type, I would always stand my ground and I wouldn’t back down, but rarely did I ever intentionally escalate a situation or bully my point across. However on the testosterone when I was getting too much I turned into Alpha male extreme. I can’t say I liked it very much and am glad they got the levels where they should be.
Experience.
But then, why should we pay any attention to empirical observation? 
**
Because my 'roid rage self says so, that’s why! You got a problem with that?** 
(Bolded for maximum testosterone induced aggression!)
This is a lot of what happened to me too. I lost a lot of sex drive, so much so that my ex and I went months and months without sex. Then she would get mad at me for not initiating it. Of course she’s gone and it wasn’t that. I also gained weight, a lot, I was up to 225 up from 190 or so. I was also tired a lot and unmotivated.
I chalked it up to having two young kids more then anything. Then when I started taking my medications a few weeks later and my swimming distance jumped by 20%. I got stronger and I slowly started to lose weight.
It was about two years after my ex left and I really hadn’t had many girlfriends, then when I found one I wanted sex all the time. It was a total change, and at the age of 39 I honestly do feel like I’m still in my early 20s some days.
Just in the last 4 months I’ve lost over 20 pounds and I really haven’t been trying. I’ve also gotten a lot more energy, so much so that some days I just have to do something. I’ve painted all the doors in my house one weekend, 22 of them, front and back, twice each. I’ve stained my deck. Torn out all the bushes around my house. It’s a massive change and I wouldn’t recommend someone going without.