Actual time was set by the astronomical phenomenon as others have mentioned. Noon at the summer or winter solstice, IIRC, is the noon time.
To be able to specifically point to seconds, you have to measure seconds. I’m not sure the technology or instrument accuracy mattered before the 1700’s. (Heck, even in the 1960’s a typical cheap watch would lose a minutes a month. Getting close to seconds a month would require serious fine-tuning.) Galileo among others observed that a swinging weight kept time proportional to it’s length, regardless of weight - the principle of a pendulum clock. You fine-tune it by adjusting the length of the pedulum.
The issues became most relevant in the Age of Discovery; the overarching puzzle was how to detemine longitude. Latitude was easy - measure the angle location of the big dipper or other stars. Longitude meant taking the angle, rise or set of known stars, comparing that to tables that had been painstakingly created. Once portable but reliable clocks were possible,that solved a lot of problems. If star A rose at such and such a time GMT at Greenwich, and 1 hour later further west, then you were 360/24 or 15 degrees west of Greenwich. Knowing details like location meant the difference between a fast voyage and a short one onto the rocks.
If you are less concerned about seconds, then the specific timeline probably goes back to Constantine and the ascendency of Christiantity; or more likely, to Julius Ceasar and his calendar. There are plenty of books and articles on calendars and how they evolved. Before Caesar, IIRC Romans simply adjusted their calendar as needed. If feast days drifted from appointed times, the government decreed a few days - this also fixed the problem that month and year did not sync up properly. He came up with the 12 months, different month lengths, and leap years.
Christians took this because they needed to calculate holydays, especially Easter and Christmas. 1500 years later, it was obvious that 365.25 days was not entirely accurate; Pope Gregory’s modifications were to eliminate the 11 days of drift over 1500 years and make 3 of every 4 centuries non-leap-years. (Centuries not divisible by 4 - ie. 17 for 1700, 18, 19, 21, etc. are not leap years. 1600 and 2000 were.) Dropping the 11 days got Easter back on track.
Part of your discussion of discontinuity is that due to protestantism, many non-catholic countries were not happy adopting a Pope’s decree about calendars, much like our attitude to French metric systems or maybe a decree from a communist country; so IIRC for example England did not drop those 11 days until the 1700’s. Russia did not do so until after the revolution, so the October Revolution started on November 10th our calendar. Orthodox churches still use the old time, so Russian Orthodox Christmas happens 12 days after Dec. 25th.
Other cultures had their own calendars, and it is possible usually to “sync up” those times; but that is not the same as a continuous number system. As far as Chinese or Japanese numbering, or similar, they adpoted the very common calendar of what year of a monarch’s reign it was. I have no idea how reliable the continuity would be, given times of troubles; but it’s a common system for many kingdoms. I have no idea how reliable or continuous the Jewish Calendar is, given that there would be years where one group or another would be too preoccupied to lose track of time; but like Europe in the dark ages, it’s unlikely that the whole of europe could drop a whole year in the AD count.
)