We remember waking up, we can remember some of our dreams but we can’t remember falling asleep.
Why?
One second you’re awake and the next it’s as if a light was switched off
I agree with the general premise of the OP, but I can recall a few instances where I willed myself back into a dream I had had earlier the same day, or at least the last time I was asleep. In that exercise I became progressively more into the sleep state and less awake. I’m sure I couldn’t identify the moment when I “stepped over the threshold” into full sleep, but it was only a matter of degrees, I will contend.
For me it’s not like a light being switched off; the awareness of surroundings just gradually fades away.
So you remember falling asleep then?
If so you’re the first person I ever knew that did.
For me it’s a case of lying there thinking about the events of the day then zonk I’m out
My wife tells me she can remember falling asleep, but I’m like you. I never remember.
As a general rule, I’ve found that whenever people ask a “never” question here, they always get a million people saying that they “ever.” 
About a hundred years ago in a HS English class, I remember a teacher reading a poem in which it had a line something to the effect of, “Boys never remember falling asleep…”
Is it a gender thing? I never know when it happens.
So well noted. It never pays to say “never” or “always” or “none” or “all” or else all you’ll get are counterexamples of varying degrees of relevance.
All generalizations are wrong.
I sometimes remember falling asleep. It is most likely to happen if a fall into a light dreaming sleep and then wake up.
What happens is that I’ll be thinking general thoughts and then at some point my thoughts, and the images associated with them, become more “real” than my surroundings. Usually at that point I’ll drift back to full consciousness but there have been occasions when I’ve woken up the next day and had some vague recollection of falling asleep.
As to why it is uncommon, I would guess that it is because you are becoming more and more aware when waking up so the last dreaming images in your mind may still be in your short term memory. When you go to sleep you’ve got about eight hours for any half remembered sensations to disappear completely from your memory.