Just as Qadgop said, there are two classes of sex hormones - androgens and estrogens. Simply put, androgens are what make men look like men and women look like women. So, for example, if a female fetus is exposed to androgens inside the womb, it will develop male characteristics. Likewise, even a grown man if given estrogens, will develop breasts.
The main androgen for men is testosterone which is made in the testicles. Women also make androgens, both in their ovaries and adrenal glands but, generally, the quantity and potency of these androgens is rather low. The estrogen produced in their ovaries predominates in terms of sex characteristics. (Men also make these types of androgens in their adrenal glands, but their effect is dwarfed by the testosterone coming from the testicles.)
Androgens and estrogens are not produced by either sex in appreciable quantity until puberty (by definition!). Interestingly, in normal puberty, adrenal androgens are among the first of these hormones to rise, i.e. male hormones in both sexes. That’s why the growth of pubic and arm-pit hair are early markers of puberty in both sexes. Note also that it is the sustained production of these “adrenal androgens” throughout life that keeps women’s pubic and axillary hair growing. (Medical students are taught that a sign of adrenal disease in women is lack of such hair).
Besides being responsible for sexual characteristics and reproduction, sex hormones are major players in growth, bone maintenance (throughout life), mood, and muscle strength.
Some Guy: Some supplements contain lots of estrogens. This article showed how one herbal preparation was very potent in its estrogen effect.
As you mentioned, plants can produce so-called phytoestrogens. As the above study illustrates, these can have quite significant effects on humans. Also, IIRC, female rats whose ovaries have been removed can, nevertheless, still “be in heat” (i.e. undergo estrus) so long as they were given sufficient quantities of certain grain alcohols (these contain sufficient quantity of phytoestrogens to do the job).
Broomstick: I agree. Those very rare men who are resistant to estrogen have disordered skeletal mineralization and development (despite having normal androgen status).
shell: I just saw your post with the preview. Check out the linked abstract.