Not “all” but “most”.
He said that any given person who is more attracted to white people can be assumed to be influenced by the culture etc. This only follows from an assumption that it’s unlikely the person might just happen to prefer white people’s appearance.
I agree that it makes sense that at a societal level there’s some influence, but there’s no reason to assume that other factors might not be just as strong or stronger, so that there’s no way to say that it’s more likely than not that a given individual preference is the result of societal preference.
[Same goes for all styles and appearance preferences. Tattoos, hairstyles, etc.]