Your impression is wrong. Nearly all horse-riding relating clothing comes from incorrectly fitting clothes – and speciically when wearing jeans, having bulky seams all down the side of your leg that contacts the saddle.
I ride English in snug-fitting breeches, and have never chafed, ever. My SO reports the same when riding English in breeches, so I do not believe this is a M/F distinction.
Wearing panyhose while riding is not particularly advisable, but wearing stretchy, snug fitting pants with minimal seams – if your discipline allows – is (and chaps if it does not)