While I have no medical knowledge to contribute to the answer, I must note that the expression : “Cold enough to freeze/shake the balls off a brass monkey” comes from the yesteryears of powder canons.
The monkey is a brass contraption which held a pyramid of 9 balls stacked neatly. When the temperatures would drop the metal would shrink, and sometimes knock over the canon balls as the diameter of the slots shrunk.
Hence, it was cold enough to shake the balls off a brass monkey.
This proverbial phrase was probably hobson-jobsonned into common language, stating that one’s finest part of the anatomy was being gambled in that wintry excursion.
A slight tangent…
In college, I hung out with a guy who had been in the Army. At one point he’d been stationed in Alaska; one on training maneuver he got penile frostbite. He went to the medic, who examined the offended organ, looked at my pal, and said “Looks like we’re going to have to amputate” waited a beat, then laughed at my pal’s shocked expression.
I once saw, for sale in a mail order catalog, an actual brass monkey who’s balls would fall off at a certain low temperature. I wished I could have gotten one, but as I recall it was for some ridiculously high price.
A search of Google Images for “Shot Garlands” shows cannon balls both above and below decks, on a variety of old warships. They were made of wood or brass. (On deck garlands were probably only stocked when the need for shot was imminent.) An explanation (that also deals with “Brass Monkey”) is provided in a comment half way down this Fine Scale Modeler web page.
Maybe Snopes could update their explanation, with images.
The explanation also doesn’t hold water from simple physics. The coefficient of expansion of brass is 0.00185% per degree K whilst cast iron is roughly half that. So a brass frame will shrink faster than the iron balls it holds, but the difference is such that the frame would shrink relative to the balls by only about 0.1% over a one hundred degree K range. Which is vastly wider than even the most desperate desert heat to Arctic cold. The idea that a brass frame would shrink enough to affect its geometry holding cannon balls is just fanciful.
I’ve seen similar ones that have exaggerated male genitals (some sort of fertility fetish I think). Not sure that such examples predate the reference of balls freezing off, but the ones I saw looked quite old.