Here in the earthquake zone we keep on hand four gallons of water per person, but we renew it every year or so because they say the plastic of the containers can affect the water over time, giving it a bad taste.
Not what the OP asked, but most big drugstores will also have a variety of tablets that can be dropped into water and will sterilize it (or at least make it drinkable). Very small quantities of bleach have the same effect. These treatments are meant for microorganisms; they won’t clean lead or mercury out of water, for example, but it’s the microorganisms you probably care about in an emergency.
I’d stick a few packs of these tablets or a jug of bleach (check to make sure it says it can be used for water purification, some have additives) next to my water stores, and not worry about bacterial growth. Don’t drink the bleach by mistake.
Homebrewing places sell them for sterilizing equipment. 1 tablet is enough to do a gallon of otherwise safe looking water. A pack of 50 is around a dollar.