I went to China in 1981 with a group from high school. The city had been closed to foreigners for decades and we were the only white people that many of them had ever seen. We were followed wherever we went. It was a bit unnerving but you kind of got used to it. I can’t imagine what actual famous people go through every day.
Yeah, my brother mentioned that their height was a big deal, but also the fact that they were white was a thing. Some of the people were farmers who had never been to a city, nor seen a white person up close.
I’m 6’2 and black haired, and I visited China in 2019, and even I would even then often be asked by Chinese people, especially children, for an opportunity to take pictures with me. I can imagine how often this would happen to a less unusual (from a Chinese perspective) looking person at a time when China was less exposed to foreign tourists.
Was it this guy?
Or maybe these guys.
Yes, this. While the median height is indeed significantly lower in China than compared to the US, say, it’s nonetheless not that rare to encounter tall people. There were several colleagues in my Shanghai office who were over 6ft tall. And I had a female Chinese friend who was 5’10. And my office was not very big, and I don’t have many friends 
I would guess that people in the cities are taller as nutrition is still a factor.
As you say, a lot of the attention that foreigners get in the smaller towns is simply that many people there may never have seen a non-Han chinese person outside of TV.