I also dislike the “apples and oranges” thing. I’ve usually heard it as “you can’t add apples and oranges”. I know what is meant. If you are asked, how much is 3 thousand plus 4 dozen, the answer is not 7. But it’s a bad way of putting it, because both statements are wrong.
First, you can add apples and oranges. If you have 2 oranges and 3 apples, you have 5 fruits. For that matter, you can add all sorts of things more different than two kinds of fruit. If you have 2 can openers and 3 living room sofas, you can say that that adds up to 5 objects. If you have two chickens and 3 doorknobs, you can say that that adds up to 5 things.
Second, you can add thousands and dozens. The reason that 3 thousand plus 4 dozen does not equal 7 is not that “you can’t add thousands and dozens,” because you can can add the two numbers. You just don’t do it the way the person saying 7 did. It’s not that you actually can’t add thousands and dozens, it’s just that you need to learn the correct method of doing so.