Question about Simpsons episode "Grampa vs. Sexual Inadequacy"

In this episode there’s a short sequence where an odd looking onlooker (he seems to be drawn and animated in a slightly different style to what was normal at the time) says “I’m not convinced! I’ve had bad luck with aphrodisiacs.” in a wobbly, comic sounding voice. The character’s appearance and the way he says his lines seems to be a reference to something else but it is unclear to me what, if any, other work this bit is referencing. Any ideas?

If you haven’t watched the episode in a while this still of the character might jog your memory.

A very good episode.

My opinion is that he was supposed to be so odd in appearance, voice, and behavior that it should be obvious that no aphrodisiac would be likely to help him. Thus the joke.

I’ll be happy to be proven wrong, though!

I thought the gag was, whatever alleged “aphrodisiac” he’d tried before is what left him so weird that talking like Ed Wynn is now as close as he gets to being normal.