Actually, this does seem to be the case. Here is a compilation of all White Walker deaths. The fight in the Cave of the Three-eyed Raven starts at 4:20. The Children of the Forest are able to kill some wights with what must be dragonglass-tipped weapons. But when one tries to spear a White Walker with one, the armor stops it and he kills her. Then Meera picks up what is the same or a similar weapon and manages to spear him in the throat where the armor doesn’t protect him and he explodes. So the armor protects against normal weapons and dragonglass but not Valyrian steel.