As several have stated, it seems to be a form of psychedelic skat.
Bands like Steely Dan and even Yes would use words (lyrics) for their sonic merit, and the lines didn’t have to make sense. Just one more example of the Beatles trying innovative ideas.
It is a fact that koo koo kachoo is an allusion to Walrus. As far as drugs go, “It’s a little secret, just the Robinson’s affair, Most of all you’ve got to hide it from the kids.” A lot of free-floating paranoia bait in that song. You had to be there.
I don’t know if it helps, but “Mrs. Robinson” was originally “Mrs. Roosevelt.” When Paul Simon got the commission for The Graduate, he used the song as is, changing only the name.