I take your point about the somewhat arbitrary nature of city boundaries, but the fact is that those boundaries do have an impact on the way that people move.
For example, suburbs outside city boundaries often have lower tax rates than the cities, or have different public school systems. While people might not specifically choose to cross the city line just to feel safer, someone who wants to move in order to feel safer is likely to also take into account things like tax rates and school districts. And, to tell you the truth, the evidence does indicate that, particularly during the height of white flight, a lot of people did feel that moving across the city boundary made a big difference to their quality of life. Plenty of good recent works of urban history demonstrate this.
Not only that, but as more people move, the disparities increase. The decline in city population reduces the tax base, leading city officials to raise taxes and cut services in order to offset the decline in tax revenue resulting from flight to the suburbs. The ongoing rise in taxes pushes more people to move, and those who can afford to move to the suburbs tend to be wealthier.
Also, while your point about the distance of the city boundary might be true, i’d be interested to know how many other cities have suffered comparable population decline in the areas around the downtown core of the city. According to the US census, the population of Detroit peaked in the 1950s at around 2 million, and there are now fewer than 900,000 people in the city. Draw the city boundaries wherever you like, but a population decline like that leads to large areas of urban blight, no-go areas, and vacant buildings in which drug dealers and other criminals can carry on their activities. I don’t think it’s any coincidence that another city with big drug and violence problems—Baltimore—has also suffered a large decline in city population since the 1950s, dropping from 950,000 to about 670,000.
Like Shagnasty said, though, the extent to which the really bad violence (homicides, etc.) affects random people is often not as great as the statistics suggests. Every week, the free weekly paper here in Baltimore has a run-down of the previous week’s homicides, and they most frequently occur in particular neighborhoods, and often reflect turf war among drug dealers or pre-existing disputes. That’s not to say that innocent people don’t get caught up in the violence, but it’s extremely unlikely that you’ll be the victim of a random murder just walking down the street.
That said, the lower-level violence like street muggings can really affect how secure and how content people are with their lives in the city. While having someone stick a gun in your face and take your purse or wallet might not leave you with any physical scars, it can leave emotional ones, and can change the way you, and the other people in the city, live from day to day.
In the last three months, there have been three armed muggings within 100 yards of our front door. In each case, the victim was a woman, and was unharmed after she gave up her purse. But the woman involved were pretty badly traumatized by having a gun stuck in their face, and this spate of muggings has made me much more cautious when i walk outside, especially after dark, and much more concerned about my wife’s safety if she’s coming home alone.