Back in the bible, there is a specific commandment to eat unleavened bread (matzah), but not to eat anything leaven. That’s why matzah is OK, it’s a specific exemption.
If you have a lot, you can always trade it for bricks or wool.
TriPolar:
Any grain that might have been in contact with water after it’s ground is suspected that it could have begun leavening. The grain used for Matzoh is carefully protected from water, in most cases, immediately after harvesting.
Matzoh made from other grains is available, though expensive. Spelt matzos are commonly used on Passover by Jews who have problems digesting wheat.
They knew that, but it’s the leavened grain rather than the yeast itself that is forbidden on Passover.