It's all about energy. Say I pick up a book from a table. The electrons in the atoms on the outer surface of that book come into "contact" with the electrons in the atoms on the outer surface of my hand, but neither group of electrons has enough energy to overcome the electrostatic repulsion (like charges repel) and knock things around.
Now, suppose I pick up a chunk of radioactive material. Suppose it's undergoing beta-decay, which means it's spitting out electrons. But these electrons have a much higher velocity (and therefore a higher energy) than the ones from the book, so they smash their way past the electrostatic replusion, and they may even knock an electron from one of my atoms, causing it to become an ion - positively charged. This means that whatever molecule the atom is in doesn't work anymore, and it either changes shape or breaks up. If that molecule happened to be a very important protein that I needed to fulfill some vital cellular function, then I have one less protein doing that function. If I sustain enough such impacts, the widespread lack of, say, oxygen processing or cell membrane repair will kill me.
The same argument goes for alpha particles (2 protons and 2 neutrons) or gamma rays (photons). That's why radios and microwaves and cell phones absolutely CANNOT give us cancer but ultraviolet radiation can. The radio and microwaves the appliances operate on simply don't have enough energy to knock anything loose, and no amount of intensity will give them that energy, while a single UV photon can start the ball rolling on skin cancer. This is what Einstein won the Nobel Prize for - the photoelectric effect.