Oleander isn’t so toxic that you die if you simply put a leaf in your mouth. Your toddler will presumably have enough adult supervision that it would be immediately noticed if he had something in his mouth. According to the feds, you have to chew on the leaves, not merely put them in your mouth. The Urban Legend about the troop of Boy Scouts who died from merely using oleander twigs for wienie forks is, um, an Urban Legend.
If you don’t intrinsically hate the oleander, why not just put up a quick ‘n’ dirty picket fence on the inside of the hedge? Sink a few postholes, attach some picket fence panels (plastic or wood, your call), make like an oversize baby gate in front of it, and then when Himself is old enough to understand the legend of the Oleander Wienie Roast, you can tear it all down.
Which of course won’t remove the problem of oleander leaves blowing or being tracked into your house, but hey, he could die from being stung by a wasp or hit by a falling meteorite or space debris or something. You take reasonable precautions, yes, but then you just have to try to relax. You can’t raise him in a cocoon. 
Also, decoding garden-speak here: something that’s “less invasive” is still “invasive”. It’s just “less invasive” than that other invasive stuff over there, but it’s still “invasive”.
The “less invasive” bamboo is “less invasive” in the sense that the pod people were “less invasive” than the War of the Worlds Martians.
An example: Spearmint is invasive. Lemon balm is supposed to be a “less invasive” mint. I planted spearmint, which took over my entire back yard; I’ve had to beat it back every summer with those flamethrowers Ripley used on the aliens. And I planted lemon balm, which admittedly didn’t take over the entire yard, but I am still finding it popping up all over the yard in random places.
So the lemon balm was “less invasive” only in the sense that I don’t have to get out there every July with heavy equipment, but it still isn’t staying where I put it, dammit.