there’s always the risk that something goes wrong during retreival and return to earth, and you end up with Dallas or Chiicago being flattened by a giant flaming lump of gold. The biggest value is having the matrial in orbit; fashioning a re-entry vehicle to bring back tons and tons of metal can get expensive. Then what, you have to launch the re-entry carrier again?
it takes so much energy to launch mass to orbit, the biggest value would be to have the mass already up there to work with; however jockeying mass into low earth orbit from the asteroid belt, and then docking with it would probably be almost as expensive.
The biggest value would be in smaller quantities or more rare material - gold, platinum, precious metals, rare earths - assuming you could find and extract them in industrial scales.
I guess if you accidentally crashed a container of toxic metals so you created a vapour cloud of it, you might gas an area of earth. Odds are it would disperse before causing major problems. Really, the biggest worry is lumps hitting the earth while still solid or worse, liquid. you probably don’t want to be dropping large quantities or radioactive material on earth either.