The average building requires moderate to major restoration work every 25 years, and periodic repairs as needed. Roofs last 10 to 25 years. Painted wood exteriors last about 5 years; vinyl siding lasts a few decades. Aluminum and masonry can last indefinitely, assuming minor damage is repaired so as to not accumulate.
Most office buildings have firms that regularly clean the exteriors and windows (and incidentally inspect for damage), because otherwise city grime from exhaust and other sources would accumulate. In Hartford, a major office tower was at risk of being mothballed for want of a tenant, and the city would have required it to be wrapped in plastic, lest minor damage not be promptly spotted. The plastic would help prevent loose masonry or broken glass from falling on the public below. In an occupied building, a broken window would likely be promptly spotted and replaced. In economically depressed areas of that same city, the lack of this regular maintenance is very apparent.
Heating and plumbing are the other major points of wear. Most building materials will last indefinitely if dry. Once water is gets inside, mold and mildew can cause materials to deteriorate, and the freeze/thaw cycle can cause cracking. If the heat is turned off, but the pipes are not drained, the building will only last a few years.
About a decade ago, Connecticut vacated a public psychiatric hospital, and turned off the steam lines providing heat, but left the water system to the campus turned on so the fire system would work. The never cut off water to individual buildings, and the plumbing would freeze, burst and leak; they now have to demolish everything, even the stately main building facing a major road that the locals wanted to use as a town hall.
In the Deep South, uncontrolled humidity will eat away plaster/drywall and other inappropriate materials. In tropical countries, the building material of choice is concrete inside and out to resist moisture (this is in areas where electricity is too spotty to allow air conditioning). In the Northern US, humidity is less of an issue, causing mostly cosmetic issues addressed by repainting.
Mostly, occupied structures have people who can catch minor damage before it spreads. A little bit of patching and cleaning goes a long way. Occupied structures are generally heated in the winter as well, heading off the major causes of deterioration.