People come in to the developed world from incredibly remote regions all of the time.
Sure, they’d be weirded out for a bit, but chances are within a few months they’d be sitting down to their favorite TV shows and scarfing Doritos. Look at how easily modern technology such as cell phones has been adopted in remote places that are otherwise pretty much medieval. People recognize what is useful pretty quick.
Starting with a solid delousing and a shower? :dubious: People back then didn’t bathe as much as we do now.
I think I’d go for:
Shock - show them some ‘wow’ stuff that is conspicuously and unarguably modern (tall buildings, electricity, mass transport systems, pop-tarts - whatever). I think this would be necessary in order to hammer home the reality of their situation.
Then retreat somewhere fairly placid, to take stock - maybe a quiet holiday resort or something - let them see that some things (food, drink…) are still recognisable and enjoyable.
Give them access to modern dental/medical care, a shave/haircut and a good long soak in the bath, some clean, comfortable clothes and take it from there…
And elec-trickery! And dreams of flying.
Cornelius Ryan’s The Last Battle, about the fall of Berlin at the end of WWII, has some anecdotes about the behavior of soldiers from rural Russia who’d never been in a city before arriving in Berlin. Going by that account, the first place to take your visitor is the bathroom; explain and, if necessary, demonstrate what all the fixtures are for. The Russians didn’t know, and since they couldn’t find outdoor latrines, they left excrement everywhere. Toilets were used for scrubbing and peeling potatoes, and bathtubs, which defied all comprehension, were simply thrown out of windows all over the city.
More amusingly, some of the Russians were seen to unscrew light bulbs and carefully pack them to take home, under the impression they contained light and could work anywhere.
(This is not a knock on Russia, by the way; plenty of rural Americans at the time would have been just as uninformed. But they went through basic training at bases with at least some plumbing, before being sent overseas on ships which had modern heads, so ignorance was fought by the time they got to Europe. Russians recruited off the farm, who’d probably marched all the way to Berlin, had no such learning opportunities.)
OK, let’s start re-training at basic personal sanitation…
I would wonder if he thought my cat was The Devil. or even if it WAS a cat. Marco is a long haired Himalayan and doesn’t quite look like a cat, much less a cat from the Middle Ages…I would think people who were hellraisers in earlier eras would have an easier time adapting to modernity. No fear! How about those wild, headstrong women in historical novels who were always galloping on their wild headstrong horses across the moor? I think they’d get with it in our time and be jumping on wild headstrong motorcycles. 
I also think they might find our air rather bad, especially smog. And the water in lakes and rivers, not to be drunk - keep away from the creek flowing behind the shopping mall! And the sheer noise of modern life. … Yeah, I’ve given this subject a lot of thought - maybe I should write a novel.
At some point you’re going to have to explain to people where Aethelraed The Stranger has come from, though. You know, the guy with the wacky accent who doesn’t have any ID, or a birth certificate, or anything except a passing mention in the Domesday Book…
Steampunk would freak them out though.
But what if your neighbor, Bob Enzyte, dropped by to invite you to a pool party?
In Taiwan, a common story/urban legend/anti-Chinese jibe is that when mainland soldiers first arrived in the late 1940s (having retreated from the mainland ahead of the Communists), they were amazed to see the infrastructure that the Japanese colonial government had put in place. Some soldiers were particularly excited to see faucets attached to walls that would give out water at the turn of a knob. So they asked around where they could get such a thing, and were directed to a local plumbing supply shop. They bought a faucet head, then took it back to their barracks, and nailed it to the wall. Turned the knob, but no water came out. The soldiers, incensed, then went back to the plumbing shop and viciously beat the shop owner for selling them faulty goods.
Yeah, I think about this too, sometimes. With my hypothetical person from the past, I always imagine that America/the colonies have been established, at the very least.
And I’d probably want to give them a makeover of some sort so they could have a hot shower, a modern haircut, and modern clothing. I’d also want to make sure any diseases, etc., that they had were treated, and that they got a good meal. Of course, if they were richer, they might already have some manner of access to these sorts of things, but I’d treat 'em well anyway.
I’d also, as someone mentioned, teach him/her how to use the toilet. In fact, that’d be one of the first things I’d ask: “do you know how to use a flush toilet?”
Any chance this is a two year old from the middle ages? That would certainly make things easier. I would go even younger but the OP did specify a common language – unless maybe you can call “goo goo” a common language?
I’d go find Rufus and get them back into the phone booth.