So there’s a lot of garden plants that it would be most unwise to put in your salad.
But is there any garden plant so toxic that you can get sick or worse from just touching it? You see various advice like “wear gloves when handling handling this plant”, usually with things like Foxglove, Lily of the Valley, Angel’s Trumpet. I saw a sign at a garden center requesting people use the provided gloves when picking out Hyacinth bulbs. What about smashing some leaves and then licking your fingers afterwords? What if your cat licks your fingers.
I think you are going to be hard-pressed to find a common houseplant that is dangerous to touch.
But, there’s always Euphorbiaceae, which have sap that ranges from irritating to deadly.
Oleander carries a lot of warnings. You often see statements like this:
http://garden.lovetoknow.com/shrubs/oleander-plants
That may be somewhat exaggerated, but the plant is very poisonous. Add that bees can supposedly make poisonous honey out of oleander flowers, and it can poison compost piles.
Oleander toxicity is, IMHO, incredibly overrated.
I have hundreds of feet (literally) of Oleander hedges, and the plants grow like weeds out here, and I’ve never once heard of anyone (or anything) being poisoned by them. They are supposed to be instant death to any animal that eats their leaves, but I’ve had horses, goats, and llamas pastured with them, with no issues.
I believe that WhyNot has written some about her unfortunate experiences with skin contact with datura.
Stinging nettle. While kayaking, my gf got out of her boat to pee. She brushed against a plant and experienced what she described as wasp stings. I took a picture and we figured out what the plant was.
There was an incident in my youth where a kid’s horse was poisoned on purpose with oleander. Someone chopped up oleander leaves and mixed them with the horse’s feed, apparently to remove the horse as competition in the show ring. The horse died quickly. There’s more info on oleander toxicity in horses here.
I’d say luck has played a large part in your experience with the plant.
Wild parsnip. It looks like Queen Anne’s lace, and causes huge blisters.
BTW, Snopes debunks the “oleander used as weenie roasting sticks” thing, which is what I was expecting when I said “somewhat exaggerated”. Snopes, does, however, repeat that ingestion of a single oleander leaf is reportedly enough to kill a child. Given the toxicity of the plant, it’s somewhat amazing that there aren’t a larger number of poisonings from it.
Re stinging nettle: I wouldn’t really consider it a garden plant. I don’t think many people cultivate it deliberately. You might have to deal with it as a weed, of course, and it does have its uses.
I wouldn’t describe the effect of stinging nettles as “getting sick or worse”, it’s localised skin irritation. Also, they make excellent soup if picked young.
Granted. I’m a little overly protective of my gf, I guess. 
I’ve gardened extensively over many years. I’ve handled foxglove, oleander and datura. I’ve stripped many leaves off Brugmansias (related to Datura) in preparation for bringing potted plants inside for a winter rest, without gloves, and without symptoms of any kind. Never a problem from skin contact, unless you count poison ivy.
The bit about not handling hyacinth bulbs without gloves sounds ridiculously unnecessary, though there are probably tiny percentages of people who might get a skin reaction to that or some other bulb/plant.
Euphorbia should be safe for just about anyone to handle, provided you don’t break or bruise leaves/stems and get sap on your hands (I’ve survived that too without incident).
Some parents and pet owners go bats over alleged plant dangers (typically related to ingestion), but provided you carefully monitor omnivorous toddlers and teach kids not to eat random stuff*, plant toxicity should not be a major concern in a garden sense.
*unfortunately, eating whatever they can find is a personality trait in Labrador retrievers. 