Facial recognition is like language acquisition. Our brains are evolved to do it. As in, we can’t not do it. The exact details of facial features (or verbal phonemes) we distinguish depends on the specific environments we’re exposed to.
A very astute comparison! They both get categorized as “experience expectant development”: hard wired predispositions that develop based on near universal sorts of exposures during sensitive periods to drive development. Babies will be exposed to language, will be exposed to faces, and are wired to attend to them. The exact language learned in “experience dependent” and without sensitive period exposure it is very hard to later even discriminate certain speech sounds. Similarly that ability to more readily tell same race faces apart based on exposure shows up early … albeit has more plasticity longer.
To the point of the OP - humans are very good at appreciating exactly how unique each face is, especially the ones of the subgroup they have most experience with