Most high altitude balloons are “zero pressure” meaning the pressure inside is exactly the same as the outside. There’s a hole in the bottom, just like a hot air balloon. Since the balloon envelope doesn’t need to withstand a pressure difference, this allows you to use an extremely thin material, like the 3.5-micron film I mentioned. The balloon is only partially inflated at launch, but as the external pressure decreases it expands. When it’s fully inflated the gas starts escaping from the bottom, that’s the maximum altitude for that balloon. Temperature shifts also cause them to lose gas, so they can only fly for a couple of day/night cycles. One way to extend flight time is to take it to Antarctica or the Arctic where it can remain in the sun for over a week. This is done routinely.
These balloons are usually equipped with a device for ripping up the balloon. The payload is cut off first, and it lands by parachute. The balloon itself is then destroyed; they try to recover it too, but only because otherwise they’d get complaints about littering. Both the balloon and payload are equipped with radio transmitters so they can be easily found. If you don’t destroy the balloon remotely, I expect it’ll slowly come down within a few days.
A more recent development is the “superpressure” balloon which are strong enough to withstand a moderate pressure difference. If designed correctly, pressure inside will be slightly higher than the outside at neutral buoyancy altitude. At night it will go down slightly, but in the morning it will come back up to the same altitude because there is no loss of gas. Google for “ULDB” (ultra-long duration balloon) for more info.