I read the autobiography of a black (IIR, very black) American dancer who toured with her group in Poland in, maybe, the 1950s. She got the attention and touches, and the interpreter was kept busy.
Different aspect of the subject, in physical anthropology Kroeber(?) made the point that domesticated animals show type variations and humans show similar type variations. Examples: poodles, sheep, a breed of chicken, and sub-Saharan humans all show tight curliness, while it doesn’t appear in non-domesticated animals. And curly hair, the non-kinky variety, also appears in humans, dogs, and cats, as does variations on baldness and on ever-growing hair; this is typically not seen in wild animals.
Humans have color variability (hair blond, brown, red and black) and so do domestic animals (all the colors you see in dogs, cats, horses, cattle, chickens, etc), while any species of wild animal typically is made up of basically one color.
And undomesticated animals rarely go to color extremes, ie white only for protective coloration like with polar bears and winter coats, and black rare in a panther variation.
We understand if a dog is born with an unusual color, people are likely to keep it and try to breed from it. The same with horses, cats, chickens…
Sexual selection is a known factor in other genetic shifts. It is conceivable that in isolated populations, that “fashions” or sexual selection have favored a given skin tone as a standard of beauty, leading to greater reproductive success for the darkest in one population, and the lightest in another(and the fattest in another, and the tallest in another…).
Back to the OP, once you have a population of a settled type, a person of novel appearance is likely to be found attractive, or at least intriguing.