To add to the Busby Berkeley recommendations, there’s also Wonder Bar (1934), which has mostly “performance” musical numbers as well as some distinct differences from the more famous Warner musicals. Wonder Bar is set in Paris and has some dark, violent undertones that stand in contrast to the polished, Deco Moderne surroundings. (Be prepared for the “Goin’ to Heaven on a Mule” number, though; it packs most of the African-American stereotypes you can think of into a relative few minutes.)
If pre-rock popular music isn’t something you like, then most musicals from that era, as well as some into the 70s, won’t please your palate very readily. But if you do enjoy vintage pop, seeing it performed in a film musical can add dimensions to your appreciation; you learn about vocal styles and orchestral arrangements that were in vogue when the songs were written, sometimes by the same performers who made a song popular originally.
Here’s a number from Springtime in the Rockies (1942) that seems as prototypically 1940s-movie-musical as can be:
http://www.youtube.com/watch?v=MGgzmLmjwZk
“Run, Little Raindrop” is a performance number – the song itself has nothing to do with the plot of the film. The cheerful melody and lyric represent what many people dislike about musicals. Yet the little touches that Betty Grable and John Payne add to it accentuate the scenes that went before and what will follow (they portray Broadway co-stars who have an ostensible romance going, but they’re at each other’s throats because of his eye for other women).
All this may be deeper analysis that the number merits, but it gives some rationale into how movie musicals (one of them, anyway) worked at the time.