Hundreds or perhaps thousands of scientists have touched moon rocks (with human flesh) in the course of their authorized scientific research. As have millions of ordinary people (including me) who have visited one of the few places where you are allowed to touch a moon rock. For years my office was not 100 feet from that very rock, and I touched it at every opportunity.
Although NASA’s moon rock storage facility maintains the collection in a sterile environment, ISTM that the only reason moon rock samples are not permitted to be touched is to preserve them for possible biology-based experiments. AFAIK, geologists and other non-biological scientists can, and do, touch them without any risk of affecting their results.
All of the lunar material samples that were presented to foreign and state governments were 1) very small, and B) embedded in lucite, thus not directly touchable. But, as I noted above, I agree with your last sentence.