States are not “semi-autonomous.” States are sovereign. Each state has the right to order its own government and law enforcement in any way that it likes (so long as it is a “republican” form of government, which has not been explicitly defined).
There is no “U.S. Police.” In many states (such as Ohio), there is no “State Police.” (Ohio has a State Highway Patrol, but its jurisdiction is limited.)
In most states, the main law enforcement agency that investigates most crimes, including serious ones, is the local agency. If the crime occurs within the boundaries of an incorporated city, then its the City Police Department. If the crime occurs outside the boundaries of an incorporated city, then it is the County Sheriff’s Department. (Guys, please note the spelling of sheriff.) The main exceptions will be states like Connecticut, where there are counties in name only. (There is no government at the county level in Connecticut.)
The county/sheriff system is actually more or less derived from Mother England, with its old system of counties/shires and sheriffs/shire reeves. Most state governments (except for Alaska and Connecticut, primarily) devolve much government authority to local governments. Usually, all territory within a state is part of some county and is within the jurisdiction of a county government, whose chief law enforcement officer is the elected sheriff. (In Ohio, the only person who can arrest the sheriff is the county coroner, another elected official.)
If a community incorporates (that is, becoming a city or a village or whatever incorporated designations are created by that state’s law), then it can choose to offer its citizens services separate from the county, including law enforcement. Most cities have a city police department. IN most cases, where there is a city police department, the sheriff will cede patrolling and investigating duties to the police department. In counties where all the territory is part of one or another incorporated entity (such as Cuyahoga County, Ohio), the sheriff might not have any patrollling or investigating duties. However, being a constitutional officer, the sheriff still has supreme jurisdictional authority in the county.
It would be very unusual to find a police force anywhere in America whose patrol officers are unarmed. Sometimes law enforcement officers with more administrative or white collar functions (plainclothes) might be routinely unarmed, but not uniformed officers. Many entities, including private businesses have their own security departments, and many, if not most, of these officers will also be armed, although they do not operate under colour of governmental authority. Most large institutions, such as universities, also have their own security or police departments, and most of the time they will also be armed, again, even if they do not operate under colour of government authority.