I’ve heard that in the last 2.5 million years (depending on your source) we have had between 5 and 17 ice ages or glacial episodes.
Thinking about it though it would seem as powerful as an ice age glacier is, it would wipe out all of the evidence of previous glacial episodes. So how do they determine how many there have been?
Well, the huge error bar in that estimate range pretty much implies that they don’t know how many there have been. But the real answer is the same as for ogres: layers (sedimentary, in this case). And not all glaciations are equal, the ice isn’t in the same place and doesn’t have the same extent every time.
Even better, there are places on Earth which are never free of glaciation (in the few million year timeframe, anyway), so there you can just take ice core samples (again with the layers).
It’s often not direct evidence of glaciation, but rather proxies that indicate cycles of CO2 in the atmosphere, or the ratio of oxygen isotopes in deep-sea sediments, by which temperature cycles can be inferred.
It never hurts to read.
The Wiki article has a graph, for example, of CO2 and dust in ice cores from Vostok Antarctica ice cores; the level of things like CO2, pollen, and dust are good indicators of overall plant activity and hence the spread of vegetation. The article also shows the extent of several glaciations in Germany, where the giveaway would be the moraine of gravel and sand left by the glaciers when they stop going forward, dropped when the ice melts. Where these end in the soil layers can help date the glaciation time. Glaciers will pick up odd rocks and drop them hundreds of miles from their origin - when rocks, where and when tells us something about the date it happened.
It’s the aggregate of dozens of different methods of gathering evidence that all agree to some extent, that points to specific intervals. As previously mentioned, the real answer is hazier the further back you go because, as you suggest, a lot of this earlier evidence does get “overwritten” by newer events.