We are probably guilty of being impatient and rude once in a while as we wait on someone else. If we keep up our attitude, we’re sometimes told to “Keep our pants on.” I’ve never had an urgent desire to remove my trousers when I’m impatient or mad, so why would this phrase somehow suggest to “Be patient?”
I’ve more often heard the phrase “Keep your shirt on,” the idea, I suppose, being that if you had lost patience, you would remove your shirt in preparation for fisticuffs. “Keep your pants on” may have come about as a “humorous” twist on the phrase.
I always heard shirt as a kid and pants later. I always assumed that shirt version had been cleaned up just as I learned as a kid “eenie meenie minee moe catch a tiger by the toe”