This is an interesting question. Now, keep in mind that I am one of those people with poor facial recognition skills. I pretty much have to have met someone more than once to be able to recognize them by face alone. More than twice, really. Or even three or four times. I just don’t have a good ‘facial memory’, whatever that’s called. I always said that I would be the absolute worst witness for a crime, 'cuz I would probably be no help at all …
That said, I would have recognized my daughter after first sight. I was helped by the fact that she was five weeks early and tiny (five pounds) as well as having a head of dark hair and distinctive (well, to me anyway) facial features. I could have picked her out of a crowd of babies easily.
Fifteen years later, my son was born. He was four weeks early, not quite so small (six pounds) and the most generic looking baby I have ever seen! He had eye colored eyes, hair colored hair (and not much of it!) and looked pretty much like every other caucasian child ever born.
This did not change as he grew older. Although I did not have to put him into day care until he was three, I found that to me he looked very much like every other little boy in the day care facility. It actually got embarrassing for me. I would walk into the day care room to pick him up (and because he was a very independent and outgoing child, he was always absorbed in some activity and not one to immediately run to mom!) and would hesitate…
I would take a few steps into the room and pause, my eyes darting about frantically trying to remember how I’d dressed him that morning as I noticed at least half a dozen boychicks that could, maybe? be my child? More than once I would start to walk confidently toward a child, only to realize after a few steps…no, not right! wrong kid!
It’s not a happy memory for me. I felt guilty that I could not immediately recognize my own child. And there was certainly no underlying psychological ‘rejection’ factors going on - my son was by far the easier child, the more outgoing, the more fun to be with - if it could be said that I favored one child over the other, he would have been the one in favor.
Well, that was then and this is now. I pretty much can recognize him now. Of course, he’s 22 years old now! I’m pretty sure I could pick him out of a crowd … 
But more than that, once you get past the superficial, Winston Churchill thing, it’s actually not that hard to distinguish, usually.