Why is rhesus incompatibility an issue, while there's no blood group incompatibility?

I’m refering to parents having different rhesus factors. If I’m not mistaken, it’s only an issue if the mother is Rh- and the father Rh+ and normally only for the second and later pregnancies. Correct me if I’m wrong.

I believe I get how it works, but what I don’t get is why, say, an O+ mother having a B+ baby isn’t an issue at all.

The short answer is that it can be an issue. ABO-HDN wiki article should answer most of your questions.

The gist of it is that anti-A and anti-B are usually IgM antibodies, which do not cross the placenta, while anti-D (I guess you could call this anti-Rh factor) is IgG and crosses easily.