That sounds plausible. Still, why only worms? Why not all the other stuff lying about in a rainstorm? Why wouldn’t it rain leaves, twigs and garbage along with the worms?
There are a bunch of reports of freak rains of various things including snakes, frogs, and nuts among others over the years in many different parts of the world. As far as I know, it is one of the few really strange phenomenon that is considered legitimate and not perfectly understood.
I recommend Charles Fort’s The Book of The Damned to anybody who wants an unscientific but in many way fascinating study into these matters. You might have to do a sanity check,* though, afterwards.