Of course they are sexual organs. Ask anyone, man or woman. Most men and women want them to be stimulated during foreplay, and some women report having orgasms from nothing other than breast stimulation.
They also have that other purpose, feeding infants, which is not sexual. So what? Pretty much all of our sexual organs have dual purposes. We pee through our major sexual organs, the clitoris and penis. Very few people view peeing as a sexual act despite the fact that a sexual organ was used to perform it.