Is long term weather prediction mathematically impossible ? Are there similar limits to AI and Autonomous systems?

Factual Questions
41

There’s a rigorous way to see that there must be certain questions about the weather that can never be decided by computational means. First, in the long term, dynamic systems settle into a set of trajectories in their phase space, the so-called ‘attractor’. In the case of the weather, some points of that set might correspond to sun, others to rain. Thus, predicting the weather is predicting where in that attractor the system will end up.

This is subject to chaos: for arbitrarily small perturbations of the initial state (variances in the exactness with which the weather right now is known), predicted future states eventually diverge arbitrarily far. But things are actually worse: the attractor of a chaotic system is a fractal set, i.e. has a non-integer dimension. But it can be shown that the question of membership in such a set is undecidable—i.e., there exists no algorithm that takes a putative state of the weather in the future, and is able to decide whether it lies on the attractor—whether it will actually ever occur.

This is equivalent to the famous halting problem, which is the problem of deciding whether an arbitrary computation ever stops and produces an answer. Indeed, the halting problem can be viewed as deciding membership for a certain fractal set.

This sort of problem also pops up for AI and autonomous systems, although it’s practical relevance isn’t clear. in principle, you can never know if any program does what you intend it to do: this is Rice’s theorem. The proof is, essentially, that if you had a procedure to check whether a given bit of code computes the first ten digits of pi (or anything else), then you can solve the halting problem—by creating a program that computes the first ten digits of pi if a given other program halts, and feeding that to your checker.

This seems a bit ridiculous: we create programs where we know what they’re gonna do all the time. But this is not in tension with the above claim: we simply can’t give a procedure that does so in every case, but that’s not in opposition to doing so in a lot of cases of practical relevance.

But still, the basic point, as such, remains: there will always be questions about complex (and in practice, that often means quite simple) systems such that there is no general way to find their answers (as long as you’re limited to computational means).

42

Not really, and that misconception falls into the climate-change-denial trope that we can hardly expect to predict climate change at the end of the century when we can’t even accurately predict the weather for next Thursday. But those are two completely different things.

Weather forecasting tries to predict specific weather conditions at a particular time and place, which is subject to myriad chaotic factors that can throw it way out of whack. Climate modeling, instead, studies the effects of persistent long-term changes in the earth’s energy budget. They may or may not have particular relevance to changes in any given geographic region, depending on their granularity.

In essence, climate models work because they’re not interested in the day-to-day chaotic behaviour that often confounds weather models, since those effects average out over time. To make a simple analogy, it may be difficult to predict the weather right here three days from now because there’s a zillion factors that could affect our prediction, many of which are not known. But I can confidently predict that it will be much warmer here in July – six months in the future – than it is right now, because there is a strong and persistent element of increasing solar energy input that has already begun.

43

Good post.

My mistake was thinking of “long term” weather like I would think about “long term” investments. “Long term” weather isn’t really very far down the road.

44

Well said. Just yesterday I was reading the report our local NWS office puts out every January about the weather for the year just ended.

What struck me was the rainfall table on page 4. Here’s an excerpt from it rearranged in geographical order from south to north:

Location2022 Rainfall (inches)Departure from Normal
Homestead General Airport60.38+0.48
Miami/Tamiami Executive Airport64.48+8.40
Miami International Airport71.55+4.14
NWS Miami – University Park67.20-2.54
Hollywood North Perry Airport66.43+4.45
Ft. Lauderdale/Hollywood Int'l Apt58.32-2.63
Fort Lauderdale Executive Airport66.86+10.36
Pompano Beach Airpark51.96-3.66

These locations are all within a few miles of their nearest neighbor. They’re all within about 3 miles of the coast of pancake flat greater Miami. From the southernmost to the northernmost is about 40 miles total. Whether we look at the departure from normal, or look at the 2022 actuals, or sum the two to see what long term “normal” is, there’s an amazing range of values over just a few miles.

Good luck figuring that out for a single day a month in advance for all-but identical locations just a few miles apart.

45

Pauli’s Principle is the Exclusion Principle, for Uncertainty see Heisenberg, of Breaking Bad fame.

46

One small bit to add -

Chaos Theory is not exclusively about unpredictability; it is also about being to predict that systems will still often be drawn into certain stable results, even if the paths to get there are manifold.

http://www.scholarpedia.org/article/Basin_of_attraction

Easiest way for me to visualize (and maybe too simplistic) is rolling a standard six sided die. It would be well nigh impossible to predict by calculation what precise path the die will take, but one of six outcomes are attractors of high probability.

47

The problem with predicting weather accurately is that it would take an amount of data and a computer to analyze that data that is far beyond what we currently have (and probably will ever have). Splitting up the Earth into cubic kilometers (or cubic meters or cubic millimeters or whatever) and having a piece of data for each of those cubes isn’t sufficient. We would at least have to know about each atom on Earth (what they are and where they are moving and how fast) to make an accurate prediction in any future time of what they will do. There are about 10 to the 50th power atoms in the Earth. If nothing outside the Earth affected what was happening to anything in the Earth, that much data and that big a computer would be sufficient. The problem is that things outside the Earth do affect what’s happening in the Earth. So we really would need enough data for all of the observable universe and a computer big enough to handle that data. There are about 10 to the 82nd power atoms in the Earth. So that much data and that big a computer would be sufficient.

But that doesn’t take care of quantum uncertainty. If knowing everything about every atom in the observable universe is not sufficient to predict what each atom will do in the future because of quantum uncertainty, that would still not be good enough. The problem is not that it is mathematically impossible, but that is apparently physically impossible. Assuming that our mathematical systems are correct, we can say things like there are an infinite number of prime numbers. The problem is that doesn’t apply to the physics of our universe. There aren’t an infinite number of atoms in the observable universe. 10 to the 82nd power isn’t anywhere close to infinite. The question is whether there is quantum uncertainty in our universe.

And all this is irrelevant to artificial intelligence.

48

Not really, even for that stage of your argument; because we’d need to know how all the interactions among those atoms, and among everything made up of those atoms, worked. And in terms of the actual weather many of those things made up of atoms are living beings; and how the interactions among living beings work is a moving target, because those interactions are continually changing.

49

So it’s even more complicated and would require even more data and a computer capable of handling that much data.

50

Indeed. I didn’t mean to contradict your argument, but to add to it.