Can I stop being allergic to guinea pigs if I get more exposed to them?

One of our guinea pigs needed his nose wiped on a regular basis. The vet didn’t have any obvious answers as to why- the pig was just susceptible to colds (and it wasn’t life threatening- he lived to be 6 years old). An allergy to people would have never crossed my mind.