I love rainbows. I used to lecture on them frequently.
If you want good books on the topic, consult The Rainbow from Myth to Mathematics by Boyer and The Nature of Light and Color in the Open Air by Minnaert.
For my money, though, the best book on the topic is R.A.R. Tricker’s Introduction to Meteorological Optics. It’s the only source that directly attacks the Airy Integral. You might try my favorite reference on Odd Physics, as well, Jearl Walker’s Flying Circus of Physics. Walker himself published a really neat article on the Rainbow that has a full color diagram showing the various “orders” of the rainbow, with their angles and ranges.
Your question is a good one, and not sufficient attention is usually paid to it. What happens is that light is refracted in many different angles from that drop, but the angle for each color is an extremum for that color – that is, an awful lot of the light tends to emerge at that particular angle, and not elsewhere, so most of the color ends up 42 degrees or so from the anti-solar point (the point opposite the sun – inevitably, this is under the earth, unless you’re in an airplane or on a mountain). The points 42 degrees or so friom that point can all be arranged in a ring, so the rainbow is a portion of a circle. Because the refractive index differs slightly with color, the optimum angle actually differs from color to color. The colors DO overlap – and a lot more than you might think. This is a lot clearer if you graph the angle the light emerges from the raindrop at against the “impact parameter” (the point where it enters the rainbow), as in Tricker’s book.
So light does emerge along different directions, but it’s by far most concentrated and strongest along the rainbow direction.
What’s not so obvious is that even this explanation doesn’t really explain the rainbow properly. The rainbow is not really caused by “the raindrop acting like a prism and separating the colors”. It’s really an interference effect caused by light rays emerging at the same angle but with slightly different path lengths, so there’s constructive and destructive interference – if this wasn’t the case, there would be a sharp edge for each color, rather than the gradual trailing off. The width of the colors and the strengths of the different colors in a rainbow actually vary with the size of the droplets making up the bow, and in really fine fogs the bow is actually WHITE! (MInnaert has a nice chart correlating appearance with drop size). This wouldn’t be the case if the “refraction” explanation were all there was to it.
Young (who gave us Young’s modulus, and who started the translation of the Rosetta Stone that Champollion finished) was the first to propose this, and used it as another example of the wave nature of light to go alongside his “two slit interference” experiment (now called "Young’s experiment) back in 1803, but he messed up the math. George Biddel Airy, Astronomer Royal at Greenwich, finally got the math straight circa 1835.