As far as I can see it was unoccupied for a few hundred years after the Venetian attack.
Touché. 
Why would it not be likely that the answer to the OP is some rock hut that a family line has used for millenia?
Why exactly are the Pyramids excluded? Just because they are an easy answer?
I’d say that “in use” requires that living people come and go for reasons other than to scavange or gawk at the place.
What sqweels said. What about Haiga Sophia?
The Kaaba has been around for some time, according to Gibbon it existed in Roman times as well. But there is no solid date
As far as I can see, my suggestions of the Pantheion and the Maisson Carre are still the winners – they’ve been in constant use as buildings you can go in and out of, and they’re used for something besides being gawked at (they spent almost all of their existence as temples/churches, although the Maison Carre is now a museum mainly). And they’re centuries older than He Hagia Sophia.
The caves at Cappadocia?
Missed Edit
For Cappadocia I remain unsure as to how much building actually went into them.
Okay, here is me winning the thread:
This abri (rock overhang) in the south of France. Called Laugerie-Haute. Located near water and offering shelter exposed to the south, this place has been inhabited/in use for over twenty thousand years. The last three centuries or so as an archeological site, but even then the guard lived there.
The funny thing is, Wikipedia has made it impossible to edit that list, due to “vandalism”. I discovered that when I wanted to replace Nijmegen with Maastricht as the oldest city in the Netherlands. There a kind of feud going on between the two cities about who is the eldest. Nijmegen wins when you look at archeological remains; Maastricht wins if you look at the amount of two millennia old buildings still standing and still in use. Nijmegen has almost none left, due to heavy bombings in World War Two.
So, I bring Maastricht to this thread. It may not have the oldest individual buildings, but the city as a whole is a wonderful mix of very old buildings that have been lovingly put to new uses.
Some examples: a thousand year old church made in to a bookstore that tops the “The 20 most beautiful bookstores in the world” list.
Another church, almost as old, now turned into a hotel;
A 14 century church now made into an historical archive.Notice the gap in the wall; that is the place where the city wall of 1000 AD was. It still runs on across the street, as it forms the back wall to the houses.
Possiobly. Your references talk about it being inhabited at many points along its history, but not continually. I suspect that the recent three hundred year gap may not be the only one.
And this is really an overhang, not an actual building. People have put buildings into this location because it was a protected overhang.
Meh - as far as cities go, Maastricht is a pretty weak offer compared to a lot of places in the Middle East, or even in Greece or Italy. It’s nice and all, but not really a thread winner. Shit, in Damascus they even have a street that is mentioned in Acts that is still called the same!
It was demolished and completely rebuilt in 693 AD, which eliminates it by the OP’s rules:
Fair enough. But our climate is better. 
[quote=“AK84, post:25, topic:610502”]
What sqweels said. What about Haiga Sophia?
[quote]
Speaking of typos, it’s Hagia Sophia. The modern name of it is Ayasofya, which is how it’s pronounced as well as spelled.
According to Islamic lore, it was first built by Abraham (while the site goes all the way back to Adam). But in historical times it’s been destroyed and rebuilt a number of times. Most recently during the Umayyad caliphate. Does it count if the foundations have persisted, even if the superstructure hasn’t?
The Dai Miao Temple on top of Tai Shan in Shandong, China has been there and in continuous use as a temple since the Qin Dynasty (221 BC).
But for how much of that time was it a building? It’s mostly not an artificial construction, is it?
For most of that time, people would have lived there in wooden or tent-like contructions, perhaps with a stone base. I don’t know if that counts as a building. It was certainly the only kind of building cro-magnon people made. But other buildings mentioned in the thread have also been -partly- built over. My point is more that Laugerie Haute is a site where archologists have found continuous traces of human inhabitation (in fairly large groups for the time) of over such an astounding long time. I think of it as the oldest city in the world, and I felt I walked on hallowed ground when I visited.
And to think that only a coulple thousand years later, they built all that splendid stuff in the Middle and Far East…
My house was built in 1930.
Dang.
For N. America that’s pretty old. 
If rock overhangs count then Ubirrhas Laugrie-Hautt soundly beat. “The rock faces at Ubirr have been continuously painted and repainted since 40,000 BCE.” Note that’s right up to the present-day, no awkward gaps to contend with.
If we stick to man-made constructions I think the Maison Caree is the winner thus far (although surely there must be some old Hindu temples out there).