I used to do it a lot, not so often now. For me it was a nervous tic. I don’t know that I would even notice others doing it.
I stroke my beard all the time, and in the most stereotypical manner possible - I start by placing my hand just below my mouth, with my fingers touching my left cheek and my thumb touching my right cheek, then run my hand down, my fingers pressing harder as they go until they end up pinching the tip of my beard about an inch past my chin. Sometimes I do a slight upward motion at the end, so it feels like I’m flicking my beard upward. It’s a very satisfying gesture.
I haven’t heard any complaints yet.