That’s the first thing I thought of, too, but there isn’t anyplace where it is accessed vertically, so the fact that there is ground above it is not relevant. All entry and exit are at grade.
The US Office of Personnel Management runs a large, active office in an abandoned mine 230 feet below ground.
Although not a building per se, the underground lunchroom at Carlsbad Caverns in New Mexico is quoted as 754 feet below surface. Elevators connect it to the Visitor Center above, so that may be close enough for government work.
The Washington Park MAX Station near the Portland Zoo is 260 ft underground. The Wikipedia article says it’s the deepest station in the US but not the world, so follow the links for deeper places.
Definitely publicly accessible.
Where are you getting the requirement for vertical access??
the Minnesota Department of Natural Resources has a State Park located around the Soudan Underground Mine. Public tours of the mine and the University’s High Energy Physics Lab located deep in the mine are available. The deepest part of the mine is 2,341 feet (713.5 m) below the surface.
Edinburgh has a whole underground city made up of vaults connected by tunnels. I don’t know how many floors down at most, but I remember one pub you had to just spiral down and down endlessly to get to.
As the thread already mentions 10 floors down this probably isn’t a winner, but it should win “most spooky”, and perhaps “oldest”. 
I used to work in T-Building at the DoE’s Mound facility. It’s five stories underground, built with an incredible amount of concrete: the roof is 15 feet thick, the walls are 16 feet thick, and the slab is 8 feet thick.
The gold vault at the New York Federal Reserve is 5 stories below ground level
Publicly accessible?
I once had an office there. We called it "The Basement of the Nation’s Attic.
Yes. “Guided museum tours also include a trip to the New York Fed’s gold vault.”