My belief is never be fashion forward unless you’re prepared to put your clothes in the closet for 5, 10 years and wait for them to come back into style or your kid decides they’re cool and takes them.
Fortunately Red, Green, Yellow pants/jackets etc, are very traditional in the ‘preppy’ demographic, so no worries on that front. Also it’s great that the colors are being co-oped into something more dynamic.
That being said, your are correct; (these are chinos, but it’s the same theory), but I think you have to be careful how you put the whole look together.
If you have the confidence to wear it, you should; but keep in mind, there will be haters.