The key concepts are that (1) light is an electromagnetic wave, (2) materials are complex arrangements of charged particles, (3) charged particles react to (and can produce) electromagnetic waves. Taking these three facts together leads to a rich suite of phenomena, including the OP’s observation that blue light slows more than red light when passing through familiar transparent materials.
The electrons in a material may be fairly free to move around (like in a metal) or they may be bound tightly by their host molecules (like in an insulator). But the situation is far from black and white. Relatively free electrons will have collisions; some electrons could become free with the slightest bump; others resist disturbance with zeal. The details depend on the elements in question and on their chemical and physical arrangements. The electrons in a mixture of hydrogen and oxygen, for instance, behave very differently from the electrons in water.
Now, an electromagnetic (EM) wave (such as visible light) consists of electric and magnetic fields oscillating in time and moving through space. (For simplicity, we’ll ignore the magnetic component here.) When an EM wave passes through a material, the electrons in that material feel the oscillating push/pull of the wave’s electric field. Blue light, for example, presents to the electrons an electric field that oscillates back and forth at a frequency of 630 terahertz (630 trillion cycles per second).
A free and massless charged particle would react to the electric field like a pawn, being moved one way and then back again with no will of its own. The electrons in a material are neither free nor massless. Also, the ones that are bound will have frequencies at which they are happy to oscillate and frequencies at which they will react as a dog being dragged to the car for a visit to the vet.
A further complication is that the motion the incoming field is inducing implies necessarily that there are charged particles being accelerated. This is exactly how EM waves are produced in the first place! Thus, the incoming EM wave causes the charged particles to produce their own EM waves. One must add together all the waves present (the incoming one plus the zillions of little new ones) to see how the observable “total” wave behaves.
Now for the rich suite of phenomena…
Consider water. There are ten electrons in each molecule. (There is some redundance in bound states, so the number of differently behaving electrons is smaller.) There is also the molecule as a whole, something I have left out up to now. That is, the molecule has a physical arrangement (H-O-H, but bent like a boomerang) that acts like a spring. An external electric field can push and pull on the spring, and the spring (like the individual electrons) has a range of reaction and resistance to this forced oscillation, depending on the oscillation frequency.
So, even though H[sub]2[/sub]O looks simple, here’s the net result for waves of various frequencies passing through: plots for water.
The top plot is the one of interest here. It shows the index of refraction of water for a wide range of EM wave frequencies. (The index of refraction is the constant of proportionality between the speed of the wave in the medium to the speed of light in vacuum. So, n=2.0 means the wave travels twice as fast in vacuum as it does in the medium.) The x-axis of the plot covers an enormous range, from long-wave radio to high energy gamma rays. The tiny sliver in the middle that is delimited by dashed vertical lines indicates the range of visible light, from red to violet. Notice that the index of refraction increases slightly across this range. This increase is exactly the phenomenon in the OP. You can see that while an increase happens to occur through the visible region, other portions of the spectrum show decreases. The complex nature of the plot stems directly from the specifics of the electronic and atomic arrangements in a water molecule.
One can design materials to have certain desired properties at certain frequencies. This can be as simple as adding impurities (e.g., lead) to glass to increase the refractive index. This can be as complicated as making arrays of metallic panels to perfectly absorb a pesky range of microwaves.
[The bottom plot in the linked image, incidently, shows the absorption coefficient of water over the same range of frequencies. This is a measure of how opaque the material is at each frequency. Notice that the visible region happens to occur right where water has a narrow window of transparency. EM waves with frequencies a factor of 5 or 10 outside of visible won’t even penetrate a thin layer of water, so it would be hard to evolve eyes to work in anything other than the (inevitably named) visible range.]