The number of baby/toddler toys that play that song is amazing, I have lost track of the number of times we have pushed a button in the shop, waited until the end and either thrown it back on the shelves or into the trolley depending on whether it is zed or zee at the end. Zed works fine.
I was born in the early eighties, and we sing it as zed where I attended pre-school and kindergarten. My sister is nearly 10 years younger, and they sung it as zee by the time she got to the same school. Wasn’t just her teacher, all three teachers were teaching it that way. My British mother was silently horrified.
I don’t think the rhyming aspect matters much, to kids. It’s the rhythm and sing-songiness that’s appealing and easy to remember. Small children don’t have a huge concept of rhyme, do they? I mean, as a kid, I was surely aware that D and G sound kind of the same, but after all that LMNO muddling through the middle and the Q and X at the end, I doubt my little brain was keeping track of the ‘ee’ sounds.
Now you’ve gone and made me picture a 10 yr old Australian kid singing the alphabet backwards. Do they do DUI tests on 10 year olds in Australia? I hear the beer goes down counterclockwise.
I prefer zed in the song - it says absolutely that ‘that there at the end of the alphabet now, we’re done.’ For this reason, kids often sing the zed with quite a lot of force.
Zed’s dead, baby… Zed’s dead.
Makes me wonder why you yanks don’t also say fee, hee, kee, lee, mee, nee, quee, ree, see, and wee! 
There are other alphabet songs. We had a thread not so long ago about the one referenced by jimm above.
The standard “Twinkle Twinkle” version is the more usual one though, even though it doesn’t rhyme here, probably because it’s a nice tempo for little kids. The British “rhyme Z with M” version scoots along a little too quickly for anyone who’s still a little shaky on which letters go where.
I learned a version (in the UK) which used the “twinkle twinkle…” tune but went like this:
A B C D E F G,
H I J K LMNOP,
Q R S T U V W,
X Y Z and oh dear me,
I can sing my A B C,
A B C D E F G.
Why “oh dear me” I have no idea. I guess the last two lines are just there to complete the tune.
Why would we? The rhyme stresses are on G, P, V and Z (and C in the repeat).