I’m sure that large portion of the reaction is based on the appearance of the visitor. Ulf touched on the perception (at least in fiction, though I doubt the reality was much different) of Native American visitors, but I also suspect any large group of only men would be suspected more than a moving wagon full of an extended family. And of course, I suspect various missionaries/preachers would get a lot more tolerance.
In fact, this question reminded me of the Johnny Appleseed column from the Cecil Columns of 2023.
Which should fit the time frame and settlement period as suggested by @Dinsdale. Perhaps a slightly more established section of the expansionist era - but the refocus on his preaching work establishes a different class of traveler’s and the expected security risks. In a different vein, I also suspect travelling tinkers and merchants that could do critical repairs and staples that might be otherwise out of reach would be very welcome and given much hospitality.