# Does anyone check to see whether weather predictions were correct?s **URL:** \ **Category:** Factual Questions\ **Created:** [February 20, 2018, 6:14am UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177 "2018-02-20T06:14:44Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Steven\_Estes](https://avatars.discourse-cdn.com/v4/letter/s/94ad74/32.png) [@Steven\_Estes](https://boards.straightdope.com/u/Steven_Estes)\ **Post date:** [February 20, 2018, 6:14am UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/1 "2018-02-20T06:14:44Z")
The weather report often says, say, there’s a fifty percent chance of precipitation tooday. Does anyone ever check to see whether it had, in fact, rained on half the days in the past five years for which the weather report said there was a fifty percent chance? More generally, does anyone keep a box score to find out how the weathermen are doing? Also: Are the weathermen/women on TV climatologists or meteorologists? What’s the difference? ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [February 20, 2018, 7:49am UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/2 "2018-02-20T07:49:27Z")
I’ve done it in the past - I ran a project for a couple of weeks where I tracked the content of the 5 day forecast for my area (south coast of England, where the weather is famously variable). 5 days out, the forecast was almost never correct; in nearly all cases, the forecast for any particular day changed more across the course of the 5 days lead time, than the actual weather changed from day to day. The 5 day forecast is therefore worthless, at least where I live. A prediction is not a prediction if you get 5 chances to amend it (and nearly always do amend it) before it hits. The next-day forecast is usually fairly accurate. ---
**Author:** ![Francis\_Vaughan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/francis_vaughan/32/3093_2.png) [@Francis\_Vaughan](https://boards.straightdope.com/u/Francis_Vaughan)\ **Post date:** [February 20, 2018, 8:05am UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/3 "2018-02-20T08:05:35Z")
The forecasters certainly keep track. That is how they keep improving their systems. There are accuracy reports available for all the major forecasting systems if you dig. Beware, this gets technical very quickly. The forecasts you see in the general media are really only a top of the iceberg of what the professional forecasters have. A lot is driven by a public desire for a single answer, not a set of probabilities. So you get the most likely forecast that the forecaster determines is reasonable given a range of local factors but derived from the detailed NWP (numerical weather prediction) system plus ensemble runs (where the initial conditions are perturbed in meaningful way to induce a reasonable spread of possible outcomes - essentially mimicking the spread of unknowns in the initial conditions and allowing some understanding of the sensitivity of the weather to that spread. A big problem is forecasting can be not so much what, but when something arrives. The ensembles provide a spread here as well. But the public usually doesn’t see this. ---
**Author:** ![Francis\_Vaughan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/francis_vaughan/32/3093_2.png) [@Francis\_Vaughan](https://boards.straightdope.com/u/Francis_Vaughan)\ **Post date:** [February 20, 2018, 8:37am UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/4 "2018-02-20T08:37:43Z")
> [@Steven\_Estes](#): > > Also: Are the weathermen/women on TV climatologists or meteorologists? What’s the difference? If they have a qualification they will be meteorologists. Meteorology is what happens in the next few days or week odd. Climate is what happens in the long term - seasons to years. If someone says tomorrow will be wet they are a meteorologist. If someone says that this summer will be dryer than normal, they are a climatologist. ---
**Author:** ![mcgato](https://avatars.discourse-cdn.com/v4/letter/m/ac8455/32.png) [@mcgato](https://boards.straightdope.com/u/mcgato)\ **Post date:** [February 20, 2018, 1:55pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/5 "2018-02-20T13:55:08Z")
Nate Silver’s book “The Signal and the Noise” has a chapter on this. It looked at forecasts 1 to 10 days out and assessed how good they were. I think most of it focused on predicted temperatures, though rain was also mentioned. Though things have improved over the years, the book said that any forecast beyond 7 days is pretty much guessing. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [February 20, 2018, 1:56pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/6 "2018-02-20T13:56:11Z")
I’ve often pondered the lack of information in weather reports. For instance, suppose that the hourly forecast has a 25% chance of rain at 1:00, 2:00, 3:00, and 4:00, but near-zero before and after that. That could mean that there’s a storm that’ll last about an hour that’s sure to strike sometime today, but they’re not sure exactly when, or it could mean that there’s a storm that’ll last about four hours, but that it’s probably going to miss us. And there’s a big difference between those: If it’s the first scenario, then if it’s dry for the first couple of hours, then you know it’s going to be wet sometime later, but if it’s the second scenario, then if it’s dry for the first couple of hours, you know it’s going to stay so. Or, of course, each hour could be independent, and knowing the outcome of the first hour or two won’t tell you anything about the rest of the afternoon. ---
**Author:** ![DPRK](https://avatars.discourse-cdn.com/v4/letter/d/4491bb/32.png) [@DPRK](https://boards.straightdope.com/u/DPRK)\ **Post date:** [February 20, 2018, 2:05pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/7 "2018-02-20T14:05:31Z")
Is there a recommended web site that (1) has access to a decent supercomputer and (2) presents a short-term forecast clearly (see above post)? Quantitive information like “there is 100% chance of a storm, and it will start raining in 35-65 mins…” would help, although there is the question on how to cleanly describe all the probabilities, especially if you care about more than a couple of hours into the future. ---
**Author:** ![watchwolf49](https://avatars.discourse-cdn.com/v4/letter/w/e9c0ed/32.png) [@watchwolf49](https://boards.straightdope.com/u/watchwolf49)\ **Post date:** [February 20, 2018, 2:06pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/8 "2018-02-20T14:06:29Z")
I’ve done the same thing as **Mangetout** and got the same results … and I went ahead and yarded up the error anailyse from the NHC [“NHC Forecast Verification”](https://www.nhc.noaa.gov/verification/) … so as a general rule the 24-hour and 48-hour forecasts are usually pretty good, 72-hour gets iffy and any 96-hour or longer forecast is mostly guesswork … depending on the current conditions … I’ve seen the 3am forecast be completely wrong by 5am … I’ve seen 120-hour forecasts spot-on correct … The folks we see on TV are most likely journalists and received their college degree in some liberal arts program … that’s not to say they know nothing of the science, indeed some of the TV forecasters are very competent in the field and are very talented at “dumbing down” the technical details for the typical TV audiences … my niece spent some time as a weather-girl so if you want I can inquiry of her how much meteorology she took in college, and don’t worry, this is the nice niece, the one that doesn’t always bitch-slap me whenever I open my mouth … Many many years ago there was an effort to depreciate the term “meteorology” as we know today that meteors don’t cause weather … so sometimes we’ll see the term “atmospheric science” in these contexts … obviously this attempt to change the terminology failed and so whether someone says they’re a meteorologist/climatologist or an atmospheric scientist may depends on which school they graduated from … ---
**Author:** ![Wendell\_Wagner](https://avatars.discourse-cdn.com/v4/letter/w/8491ac/32.png) [@Wendell\_Wagner](https://boards.straightdope.com/u/Wendell_Wagner)\ **Post date:** [February 20, 2018, 2:07pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/9 "2018-02-20T14:07:09Z")
There is a method of testing forecasts for many different sorts of predictions called the Brier score. It was originally invented in 1950 to test the predictions of the U.S. Weather Bureau. It’s much more accurate than typical sloppy methods of testing forecasts. If you wish to know more about how to do good forecasting in general, you should read the book _Superforecasting_: > **[Brier score](https://en.wikipedia.org/wiki/Brier_score)** > > The Brier Score is a strictly proper score function or strictly proper scoring rule that measures the accuracy of probabilistic predictions. For unidimensional predictions, it is strictly equivalent to the mean squared error as applied to predicted probabilities. > The Brier score is applicable to tasks in which predictions must assign probabilities to a set of mutually exclusive discrete outcomes or classes. The set of possible outcomes can be either binary or categorical in nature, and the prob... > **[Superforecasting: The Art and Science of Prediction](https://en.wikipedia.org/wiki/Superforecasting)** > > Superforecasting: The Art and Science of Prediction is a book by Philip E. Tetlock and Dan Gardner released in 2015. It details findings from The Good Judgment Project. > The Economist reports that superforecasters are clever (with a good mental attitude), but not necessarily geniuses. It reports on the treasure trove of data coming from The Good Judgment Project, showing that accurately selected amateur forecasters (and the confidence they had in their forecasts) were often more accurately tun... ---
**Author:** ![watchwolf49](https://avatars.discourse-cdn.com/v4/letter/w/e9c0ed/32.png) [@watchwolf49](https://boards.straightdope.com/u/watchwolf49)\ **Post date:** [February 20, 2018, 2:17pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/10 "2018-02-20T14:17:48Z")
> [@Chronos](#): > > I’ve often pondered the lack of information in weather reports … [snip] The NWS provides a “forecast discussion” with all their forecasts … usually several paragraphs where the forecasters will say why they say what they say … Here’s an example from the St Louis office a couple of hours ago: > [@](#): > > The sfc front currently extends from just west of KUIN to west of > KCOU. A sfc wave is actually expected to move NE along the front, > which will help to move the front west slightly this morning. > > A lot of similarities from yesterday for today. Very strong LLJ > across the region to pump moisture into the area. The only > difference is that winds will be slightly more sly today and > perhaps not quite as strong. The approaching deep trof and back > building ULJ will help provide broad lift across the region. … and this continues: [spoiler]Believe showers will increase in activity, roughly from I-44 in MO and I-55 in IL, through sunrise. An approaching s/w will help precip fill in across much of the CWA north and west of this corridor. The rain will gradually move SE with the front thru today. While showers will continue ahead of the front, believe the more widespread precip will remain just ahead and behind the front. That said, much of the guidance suggests the precip will end as the sfc temps drops to freezing and below. While the front shud be roughly from KVIH to KSTL to K3LF at 00z this evening, temps shud be be reaching the freezing point in far nwrn portions of the CWA as the precip comes to an end. This trend shud generally continue as the cold air continues to move SE into the region. With a warm ground and anticipating only a brief period of freezing rain, have held off issuing any headlines for now. However, some uncertainty still exists regarding this scenario. The 06z NAM just arriving continues to suggest more precip further back into the cold air. Have also held off issuing any flood headlines at this point. While there is expected a prolonged period of rain thru tonight with the area expected to receive 1 to 2 inches by sunrise Wed, believe that it will fall over a long enuf period of time that rivers will be able to handle the water. This rain will help to reduce the ongoing drought across the area and may increase the flooding potential later this week. Tilly .LONG TERM… (Wednesday through Monday) Issued at 331 AM CST Tue Feb 20 2018 The primary forecast issue in this period continues to be the coverage and duration of freezing rain on Wed night and Thu morning. There is little doubt that at least some freezing rain will occur on Wed night and Thu morning (perhaps mixed with sleet at times) but there is a moderate level of uncertainty regarding the duration of FZRA and therefore the amount of additional ice accretion which can be expected on Wed night and Thu. Lingering freezing rain on Wed morning should transition to rain as surface temperatures warm into the mid to upper 30s. Many areas will see a lull in precipitation during the day, especially over parts of central and northeast MO as well as west central IL. Additional precipitation is expected to develop and spread northward on Wed night and Thu due to isentropic ascent, and this is the time period with uncertainty regarding ptype. The NAM remains a moist and cold outlier and would suggest several hours of moderate to heavy FZRA. The 20/00z GFS is still warmer than the NAM (although it is slightly colder than the 19/00z GFS), but it also ends the precipitation much earlier compared to the NAM and would suggest a much shorter period of FZRA. There is as yet no strong signal to suggest that either solution is superior, although the NAM remains quite an outlier compared to other guidance and it is not the preferred solution at this time. Therefore, this forecast package follows a middle ground which is coincidentally similar to the SREF: a broad area of FZRA overnight on Wed night (with a zone of mixed precip along its northern edge) which transitions to rain from south to north on Thu as a stalled frontal boundary begins lifting northward and surface temperatures warm into the mid/upper 30s to 40s. The expectation of at least some FZRA on Wed night is consistent with the expected synoptic pattern where isentropic ascent creates precipitation which then falls into a colder air mass at the surface. This conceptual pattern matches most of the FZRA events in the CWA. The southwest flow pattern aloft will bring additional chances for precipitation to the area between Thu night and Sat night. A brief period of upper ridging is expected on Sunday ahead of yet another developing trough over the western half of the CONUS. Models are in poor agreement regarding the strength, timing, and evolution of this feature, therefore SChc PoPs look reasonable for early next week. The coolest day during the extended period is likely to be Wed when highs are in the 30s. Temperatures will rise each day between Wed and Sat followed by a dip on Sun due to the passage of a cold front. Kanofsky && .AVIATION… (For the 12z TAFs through 12z Wednesday Morning) Issued at 541 AM CST Tue Feb 20 2018 Sfc front is currently just west of UIN and west of COU. Believe the front will sink SE slowly today. Expect cigs to drop behind the front. Shower activity shud increase by mid morning across the area, but especially at COU and UIN. As the cold air moves in tonight, a brief period of FZRA is possible just before the precip comes to an end. There is still some uncertainty regarding how much FZRA is possible. Expect cigs to improve this evening and thru the night. SPECIFICS FOR KSTL/KSUS/KCPS: Expect shower activity to increase by mid afternoon along with low end MVFR cigs as the front approaches. As the cold air moves in tonight, a brief period of FZRA is possible just before the precip comes to an end. There is still some uncertainty regarding how much FZRA is possible. Expect cigs to improve late tonight, but remain MVFR thru 12z Wed. Tilly && .LSX WATCHES/WARNINGS/ADVISORIES… MO…None. IL…None. && WFO LSX [/spoiler] It's a little thick but it's there for anyone who wants to read about it ... ---
**Author:** ![Jasmine](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/jasmine/32/2964_2.png) [@Jasmine](https://boards.straightdope.com/u/Jasmine)\ **Post date:** [February 20, 2018, 2:21pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/11 "2018-02-20T14:21:24Z")
I think our meteorologists do a pretty good job considering the fact that this region is very complex weather wise. Chicago is at a latitude where there are a lot of things going on, and Lake Effect adds even another dimension to what is already a complex situation. I’ve always wanted to be a meteorologist in Phoenix. “Today, it will be hot and sunny.” Okay, now I can go to the beach for the rest of the day. 😃 ---
**Author:** ![Francis\_Vaughan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/francis_vaughan/32/3093_2.png) [@Francis\_Vaughan](https://boards.straightdope.com/u/Francis_Vaughan)\ **Post date:** [February 20, 2018, 2:37pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/12 "2018-02-20T14:37:00Z")
Probably the best current global forecasts come the ECMWF. [Here](https://www.ecmwf.int/en/forecasts/quality-our-forecasts) they provide an overview of their quality. The 2015 report is [here](https://www.ecmwf.int/sites/default/files/elibrary/2015/15275-evaluation-ecmwf-forecasts-including-2014-2015-upgrades.pdf). ---
**Author:** ![Colibri](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/colibri/32/1841_2.png) [@Colibri](https://boards.straightdope.com/u/Colibri)\ **Post date:** [February 20, 2018, 2:55pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/13 "2018-02-20T14:55:22Z")
Steven Estes, I have contacted you by PM to indicate that you should reduce the number of threads you are starting every day. Since you have chosen to ignore this, I am going to start closing your threads. Please start no more than two threads a day, and wait until some of your questions have fallen off the first page before starting new threads. From the Registration Agreement: Quote: Do not post the same or similar messages or threads to multiple forums; multiple threads on the same topic; or an excessive number of threads on any topic within a limited period of time. Colibri General Questions Moderator ---
**Author:** ![Colibri](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/colibri/32/1841_2.png) [@Colibri](https://boards.straightdope.com/u/Colibri)\ **Post date:** [February 22, 2018, 2:09pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/14 "2018-02-22T14:09:39Z")
Since no new threads have been opened and several have now fallen off the front page, I am reopening this one. Colibri General Questions Moderator ---
**Author:** ![watchwolf49](https://avatars.discourse-cdn.com/v4/letter/w/e9c0ed/32.png) [@watchwolf49](https://boards.straightdope.com/u/watchwolf49)\ **Post date:** [February 22, 2018, 2:33pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/15 "2018-02-22T14:33:12Z")
Today’s _Weather Report_: Morning comes, she follows the path to the river shore … ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [February 22, 2018, 3:40pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/16 "2018-02-22T15:40:19Z")
**Jasmine** , just what beach were you planning on going to in Phoenix? ---
**Author:** ![Jasmine](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/jasmine/32/2964_2.png) [@Jasmine](https://boards.straightdope.com/u/Jasmine)\ **Post date:** [February 22, 2018, 3:49pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/17 "2018-02-22T15:49:33Z")
> [@Chronos](#): > > **Jasmine** , just what beach were you planning on going to in Phoenix? Um … A manmade pool with a whole lot of sand around it! 😃 ---
**Author:** ![kenobi\_65](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/kenobi_65/32/13178_2.png) [@kenobi\_65](https://boards.straightdope.com/u/kenobi_65)\ **Post date:** [February 22, 2018, 4:11pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/18 "2018-02-22T16:11:20Z")
> [@watchwolf49](#): > > The NWS provides a “forecast discussion” with all their forecasts … usually several paragraphs where the forecasters will say why they say what they say … I’ve been a weather nerd for my entire life; I fully intended to become a meteorologist until I was 16 or 17, and discovered just how much high-level math and physics you needed for a degree in meteorology. 🙂 The NWS site is always bookmarked on my computers, and I love reading the forecast discussions. Yes, there’s a lot of “inside baseball” in it, and it’ll often be very technical (and, thus, sometimes difficult to read for the casual reader), but it usually provides a lot of insight, particularly when the models aren’t in clear agreement, and the forecaster is having to make a judgement call on the forecast. ---
**Author:** ![markn\_1](https://avatars.discourse-cdn.com/v4/letter/m/f9ae1b/32.png) [@markn\_1](https://boards.straightdope.com/u/markn_1)\ **Post date:** [February 22, 2018, 4:25pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/19 "2018-02-22T16:25:44Z")
> [@watchwolf49](#): > > Today’s _Weather Report_: > > Morning comes, she follows the path to the river shore … Suite! ---
**Author:** ![watchwolf49](https://avatars.discourse-cdn.com/v4/letter/w/e9c0ed/32.png) [@watchwolf49](https://boards.straightdope.com/u/watchwolf49)\ **Post date:** [February 22, 2018, 5:20pm UTC](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177/20 "2018-02-22T17:20:32Z")
> [@kenobi\_65](#): > > I’ve been a weather nerd for my entire life; I fully intended to become a meteorologist until I was 16 or 17, and discovered just how much high-level math and physics you needed for a degree in meteorology. 🙂 … [snip] Hydrogoddamnics … about the longest equations in any scientific field … [Next page](https://boards.straightdope.com/t/does-anyone-check-to-see-whether-weather-predictions-were-correct-s/809177.md?page=2)