The weather report often says, say, there’s a fifty percent chance of precipitation tooday.
Does anyone ever check to see whether it had, in fact, rained on half the days in the past five years for which the weather report said there was a fifty percent chance? More generally, does anyone keep a box score to find out how the weathermen are doing?
Also: Are the weathermen/women on TV climatologists or meteorologists? What’s the difference?
I’ve done it in the past - I ran a project for a couple of weeks where I tracked the content of the 5 day forecast for my area (south coast of England, where the weather is famously variable).
5 days out, the forecast was almost never correct; in nearly all cases, the forecast for any particular day changed more across the course of the 5 days lead time, than the actual weather changed from day to day.
The 5 day forecast is therefore worthless, at least where I live. A prediction is not a prediction if you get 5 chances to amend it (and nearly always do amend it) before it hits.
The next-day forecast is usually fairly accurate.
The forecasters certainly keep track. That is how they keep improving their systems. There are accuracy reports available for all the major forecasting systems if you dig. Beware, this gets technical very quickly.
The forecasts you see in the general media are really only a top of the iceberg of what the professional forecasters have. A lot is driven by a public desire for a single answer, not a set of probabilities. So you get the most likely forecast that the forecaster determines is reasonable given a range of local factors but derived from the detailed NWP (numerical weather prediction) system plus ensemble runs (where the initial conditions are perturbed in meaningful way to induce a reasonable spread of possible outcomes - essentially mimicking the spread of unknowns in the initial conditions and allowing some understanding of the sensitivity of the weather to that spread.
A big problem is forecasting can be not so much what, but when something arrives. The ensembles provide a spread here as well. But the public usually doesn’t see this.
If they have a qualification they will be meteorologists. Meteorology is what happens in the next few days or week odd. Climate is what happens in the long term - seasons to years. If someone says tomorrow will be wet they are a meteorologist. If someone says that this summer will be dryer than normal, they are a climatologist.
Nate Silver’s book “The Signal and the Noise” has a chapter on this. It looked at forecasts 1 to 10 days out and assessed how good they were. I think most of it focused on predicted temperatures, though rain was also mentioned. Though things have improved over the years, the book said that any forecast beyond 7 days is pretty much guessing.
I’ve often pondered the lack of information in weather reports. For instance, suppose that the hourly forecast has a 25% chance of rain at 1:00, 2:00, 3:00, and 4:00, but near-zero before and after that. That could mean that there’s a storm that’ll last about an hour that’s sure to strike sometime today, but they’re not sure exactly when, or it could mean that there’s a storm that’ll last about four hours, but that it’s probably going to miss us. And there’s a big difference between those: If it’s the first scenario, then if it’s dry for the first couple of hours, then you know it’s going to be wet sometime later, but if it’s the second scenario, then if it’s dry for the first couple of hours, you know it’s going to stay so. Or, of course, each hour could be independent, and knowing the outcome of the first hour or two won’t tell you anything about the rest of the afternoon.
Is there a recommended web site that (1) has access to a decent supercomputer and (2) presents a short-term forecast clearly (see above post)?
Quantitive information like “there is 100% chance of a storm, and it will start raining in 35-65 mins…” would help, although there is the question on how to cleanly describe all the probabilities, especially if you care about more than a couple of hours into the future.
I’ve done the same thing as Mangetout and got the same results … and I went ahead and yarded up the error anailyse from the NHC “NHC Forecast Verification” … so as a general rule the 24-hour and 48-hour forecasts are usually pretty good, 72-hour gets iffy and any 96-hour or longer forecast is mostly guesswork … depending on the current conditions … I’ve seen the 3am forecast be completely wrong by 5am … I’ve seen 120-hour forecasts spot-on correct …
The folks we see on TV are most likely journalists and received their college degree in some liberal arts program … that’s not to say they know nothing of the science, indeed some of the TV forecasters are very competent in the field and are very talented at “dumbing down” the technical details for the typical TV audiences … my niece spent some time as a weather-girl so if you want I can inquiry of her how much meteorology she took in college, and don’t worry, this is the nice niece, the one that doesn’t always bitch-slap me whenever I open my mouth …
Many many years ago there was an effort to depreciate the term “meteorology” as we know today that meteors don’t cause weather … so sometimes we’ll see the term “atmospheric science” in these contexts … obviously this attempt to change the terminology failed and so whether someone says they’re a meteorologist/climatologist or an atmospheric scientist may depends on which school they graduated from …
There is a method of testing forecasts for many different sorts of predictions called the Brier score. It was originally invented in 1950 to test the predictions of the U.S. Weather Bureau. It’s much more accurate than typical sloppy methods of testing forecasts. If you wish to know more about how to do good forecasting in general, you should read the book Superforecasting:
The NWS provides a “forecast discussion” with all their forecasts … usually several paragraphs where the forecasters will say why they say what they say …
Here’s an example from the St Louis office a couple of hours ago:
… and this continues:
[spoiler]Believe showers will increase in activity, roughly from I-44 in
MO and I-55 in IL, through sunrise. An approaching s/w will help
precip fill in across much of the CWA north and west of this
corridor. The rain will gradually move SE with the front thru
today. While showers will continue ahead of the front, believe the
more widespread precip will remain just ahead and behind the
front. That said, much of the guidance suggests the precip will
end as the sfc temps drops to freezing and below. While the front
shud be roughly from KVIH to KSTL to K3LF at 00z this evening,
temps shud be be reaching the freezing point in far nwrn portions
of the CWA as the precip comes to an end. This trend shud
generally continue as the cold air continues to move SE into the
region. With a warm ground and anticipating only a brief period
of freezing rain, have held off issuing any headlines for now.
However, some uncertainty still exists regarding this scenario.
The 06z NAM just arriving continues to suggest more precip further
back into the cold air.
Have also held off issuing any flood headlines at this point.
While there is expected a prolonged period of rain thru tonight
with the area expected to receive 1 to 2 inches by sunrise Wed,
believe that it will fall over a long enuf period of time that
rivers will be able to handle the water. This rain will help to
reduce the ongoing drought across the area and may increase the
flooding potential later this week.
Tilly
.LONG TERM… (Wednesday through Monday)
Issued at 331 AM CST Tue Feb 20 2018
The primary forecast issue in this period continues to be the
coverage and duration of freezing rain on Wed night and Thu
morning. There is little doubt that at least some freezing rain
will occur on Wed night and Thu morning (perhaps mixed with sleet
at times) but there is a moderate level of uncertainty regarding
the duration of FZRA and therefore the amount of additional ice
accretion which can be expected on Wed night and Thu.
Lingering freezing rain on Wed morning should transition to rain
as surface temperatures warm into the mid to upper 30s. Many
areas will see a lull in precipitation during the day, especially
over parts of central and northeast MO as well as west central
IL. Additional precipitation is expected to develop and spread
northward on Wed night and Thu due to isentropic ascent, and this
is the time period with uncertainty regarding ptype. The NAM
remains a moist and cold outlier and would suggest several hours
of moderate to heavy FZRA. The 20/00z GFS is still warmer than
the NAM (although it is slightly colder than the 19/00z GFS), but
it also ends the precipitation much earlier compared to the NAM
and would suggest a much shorter period of FZRA. There is as yet
no strong signal to suggest that either solution is superior,
although the NAM remains quite an outlier compared to other
guidance and it is not the preferred solution at this time.
Therefore, this forecast package follows a middle ground which is
coincidentally similar to the SREF: a broad area of FZRA
overnight on Wed night (with a zone of mixed precip along its
northern edge) which transitions to rain from south to north on
Thu as a stalled frontal boundary begins lifting northward and
surface temperatures warm into the mid/upper 30s to 40s. The
expectation of at least some FZRA on Wed night is consistent with
the expected synoptic pattern where isentropic ascent creates
precipitation which then falls into a colder air mass at the
surface. This conceptual pattern matches most of the FZRA events
in the CWA.
The southwest flow pattern aloft will bring additional chances for
precipitation to the area between Thu night and Sat night. A brief
period of upper ridging is expected on Sunday ahead of yet another
developing trough over the western half of the CONUS. Models are
in poor agreement regarding the strength, timing, and evolution
of this feature, therefore SChc PoPs look reasonable for early
next week.
The coolest day during the extended period is likely to be Wed
when highs are in the 30s. Temperatures will rise each day between
Wed and Sat followed by a dip on Sun due to the passage of a cold
front.
Kanofsky
&&
.AVIATION… (For the 12z TAFs through 12z Wednesday Morning)
Issued at 541 AM CST Tue Feb 20 2018
Sfc front is currently just west of UIN and west of COU. Believe
the front will sink SE slowly today. Expect cigs to drop behind
the front. Shower activity shud increase by mid morning across the
area, but especially at COU and UIN. As the cold air moves in
tonight, a brief period of FZRA is possible just before the precip
comes to an end. There is still some uncertainty regarding how
much FZRA is possible. Expect cigs to improve this evening and
thru the night.
SPECIFICS FOR KSTL/KSUS/KCPS: Expect shower activity to increase
by mid afternoon along with low end MVFR cigs as the front
approaches. As the cold air moves in tonight, a brief period of
FZRA is possible just before the precip comes to an end. There is
still some uncertainty regarding how much FZRA is possible. Expect
cigs to improve late tonight, but remain MVFR thru 12z Wed.
Tilly
&&
.LSX WATCHES/WARNINGS/ADVISORIES…
MO…None.
IL…None.
&&
I think our meteorologists do a pretty good job considering the fact that this region is very complex weather wise. Chicago is at a latitude where there are a lot of things going on, and Lake Effect adds even another dimension to what is already a complex situation.
I’ve always wanted to be a meteorologist in Phoenix. “Today, it will be hot and sunny.” Okay, now I can go to the beach for the rest of the day. 
Probably the best current global forecasts come the ECMWF. Here they provide an overview of their quality.
The 2015 report is here.
Steven Estes, I have contacted you by PM to indicate that you should reduce the number of threads you are starting every day. Since you have chosen to ignore this, I am going to start closing your threads. Please start no more than two threads a day, and wait until some of your questions have fallen off the first page before starting new threads.
From the Registration Agreement:
Quote:
Do not post the same or similar messages or threads to multiple forums; multiple threads on the same topic; or an excessive number of threads on any topic within a limited period of time.
Colibri
General Questions Moderator
Since no new threads have been opened and several have now fallen off the front page, I am reopening this one.
Colibri
General Questions Moderator
Today’s Weather Report:
Morning comes, she follows the path to the river shore …
Jasmine, just what beach were you planning on going to in Phoenix?
Um … A manmade pool with a whole lot of sand around it! 
I’ve been a weather nerd for my entire life; I fully intended to become a meteorologist until I was 16 or 17, and discovered just how much high-level math and physics you needed for a degree in meteorology. 
The NWS site is always bookmarked on my computers, and I love reading the forecast discussions. Yes, there’s a lot of “inside baseball” in it, and it’ll often be very technical (and, thus, sometimes difficult to read for the casual reader), but it usually provides a lot of insight, particularly when the models aren’t in clear agreement, and the forecaster is having to make a judgement call on the forecast.
Suite!
Hydrogoddamnics … about the longest equations in any scientific field …