Knock it off. Clearly people are interested in engaging but if you’re not, then rather than snide backhanded remarks, simply don’t.
[/moderating]
I take offense at this moderation. I wasn’t backhanded in my snideness at all! 
Moderation deserved and noted. I’ll contain my snideness to my own thoughts going forward.
I appreciate your bringing in how 538 actually operates in the discussion. Yes, as election eve approaches 538 overweights recency. (Details here.)
It certainly makes sense to do that. That last week is where many voters finally decide and information older than that ages in value quickly.
The current approval rating is not the circumstance that overweights recency over house reliability. Quite the opposite circumstance.
So now that you are wanting to engage with how “Silver does it”, let’s. I’ve pointed out to you a few posts up that C+ rated Rasmussen is one of five houses that report at least weekly, three that are rated higher than it it is.
And there are two new less frequently reporting highly rated house polls that pop in each week on average.
IF, for simplicity and as the edge case, we assumed that highly rated houses all suddenly aged to zero weight after a week (which of course they do not, we know he has them lose weight slower the better rated they are) and all other polls stayed static but C+ rated Rasmussen, how much do you think would Rasmussen need to move to move the tracker by 2, or even 1, given it being one of just those seven, five of which are more highly rated and therefore significantly more heavily weighted?
The answer of course depends on how much a C+ house’s weight is discounted relative to a B or A house. Counting them the absurd all the same weight Rasmussen would need to move 7 to move the needle one and 14 to move it two, and the number goes up dramatically as we apply whatever decreased weight it is actually given. Given that Rasmussen actually moved about 3 over that time period it is certainly the case that it did not meaningfully or significantly impact the tracker’s output in this particular case, and given that its weight is not the same, that older quality house results do have input more than a week out, and it is highly unlikely that Rasmussen has moved double digits over time while the quality houses have all stayed flat, it is almost certainly such as a general rule.
A single C+ house moving 5 or so points differently than all the quality house results move will have no perceptible impact on the curve, and aggregated with a multiple other C+ houses would start to look like a mostly flat line, as their individual noise would cancel each other out. (Which is what happens with your Harris overweighted and no house correction aggregated line, even as individual quality house results as a group moved by 4.5. in that time period.)
Cantankerousness aside we can learn a lot about how the 538 weights polls based on pollster rating and freshness by looking at that Iowa 2016 forecast. There’s nothing special about Iowa here. That just happens to be the one I had open. Also, there are certainly some differences between this model and the Trump approval model, but many of the underlying principles should hold.
Here are some insights that should clear up some misconceptions held by other posters in this thread.
• Month old data from the same pollster, even an A+ pollster, is weighted significantly less than fresh data (e.g. Selzer poll from beginning of October is about 15% the weight of Selzer poll from beginning of November although sample size plays a role here as well).
• A fresh C- poll with a larger sample size can have greater weight than a two week old A- poll (e.g. Survey Monkey versus Quinnipiac).
• A C+ poll in the field at the same time as a B poll can have a higher weight due to a larger sample size (e.g. RABA (2.43) > Emerson (2.27)).
• Even a C- pollster can have higher weight than the average weight of other pollsters (e.g. Survey Monkey).
There’s probably some other things we can draw from that, but these are the ones that leapt to mind. Many of those bullet points, in some form or other, were called absurd earlier in this thread.
I think it’s safe to say that Nate Silver does not consider C- to C+ pollster data to be ‘utter shit’. Two of three highest weight polls fall into this category and a rational analyst does not feed ‘utter shit’ into his model.
Unsupported and likely incorrect.
I am curious - why do you think that it is “likely incorrect” that current approval ratings are more like polling results several months out, when recency is relatively less valued, than they are like polling results as Election Day approaches, the case in which Silver overweights recency?
Do you really not understand why election eve is “special”?
And do you have any response to the math demonstrating the limit to how much a single tracker, let alone one of the lowest weighted ones, can move the output given its being one of seven pollsters reporting at least weekly and on average at least two high quality houses dropping new data each week, even ignoring the rapidity that C+ trackers age compared to the data from the higher rated houses (thus, for simplicity sake, ignoring the impact of higher weighted houses results more than a week old)?
I mean other than to just state “likely incorrect” with no supportive argument or rationale.
The math really isn’t very hard to follow and it really is head-shaking to have someone respond to “2 + 2 = 4” with “likely incorrect.”
Last item is a request - I don’t program but can you run your program also showing a line of the aggregate (as you define it) that includes Rasmussen please?
My suspicion is that even within your universe, with your method’s lack of weighting by rating, and counting each instance of a daily tracker as much as the results by houses that report out every three weeks or less, you will see that your new aggregate line is barely different than your current one, and will not conform to 538’s output any better.
Thank you in advance.
It’s certainly reasonable that 538 might not treat the election as the ultimate end of the approval rating and gauge recency differently than in primary and general election polling.
Not sure I understand that statement Carnal K.
But we know that 538 considers recency as less important early on and as gaining in importance as Election Day approaches. They state that. The week before the election is the extreme case of recency weighting.
And we know that counting only recent (one week old or less) polls (an extremely high value on recency over quality) a single C+ tracker moving 3 to 5 points would have an insignificant input on 538’s outputs.
I’m saying that "as election day approaches " might not be considered for approval ratings because it’s a constantly running number. Nobody is taking general election polls in December 2020 but they will be doing approval polls.
Meant to add, that’s not unreasonable because job approval isn’t hard locked to voting preference. For instance, GOP presidents have often had approval ratings amongst minority groups that they’ve never come close to getting in vote tallies.
Pretty much this.
Polls six month out from an election and polls seven months out from an election have pretty much equal predictive value so it doesn’t really make a ton of sense to drastically devalue the seven month poll in favor of the six month poll just because it’s a month old.
Furthermore, polling far out out from an election is typically sparser than polling with an election imminent. For this reason you may not want to be too aggressive with time decay because then your model would tend to reflect only the most recent poll and be less useful as a forecast for and election six months in the future.
538’s Trump approval model is trying to keep it’s finger on the pulse of the American electorate. For this reason it probably handles data decay much more like a situation where an election is imminent than a forecast of an election six months from now.
Finally, I didn’t respond ‘likely incorrect’ to the statement ‘2 + 2 = 4’. I quoted the exact statement that I thought was likely incorrect.
Carnal K,
AH.
I’d suspect you are right and that recency is not increased in weight for approval rating even as election day approaches, it would not the same sense to … but by then no one is really caring about approval rating for its own sake and it is not factored into any of the elections forecasts.
Obviously it mattered that Obama had crawled back into net positive approval territory by election day (at this point he had, briefly, fallen almost as low as Trump consistently sits, underwater by about 8) but it was the which candidate would you be vote for polls that were the actual polling inputs, as should be.
Actually, not quite. Again, we go to the source. Their goal is less a single moment pulse than having predictive power.
To be precise that is the actual goal. In service of that goal, again, as clearly stated by 538, highly rated houses are weighted more heavily and lose weight more slowly, and poorly rated houses and weighted lightly and lose weight more quickly. The curve is then calculated using fairly aggressive parameters as that turns out to do a good predictive job:
Yeah this is getting into the weeds. But I would still love direct answers to the questions and request I made.
I think you are saying the same thing Lance did about this.
Lance, FWIW, I found your models interesting. I can see the value in peeking at Rasmussen because they are so important to the Republican understanding of the world.
I particularly approve of your NL West model. 
Ravens vs Vikings, Ravens just tried a quantum froth stochastic variation. Didn’t work.
I’m not sure how you are reading this and still coming to the conclusion that month old data could still be weighted as heavily as fresh data.
The month old data only falls within the 30 day bandwidth while the fresh data falls into each of the 10, 20, and 30 day bandwidths.
Thanks.
I think I have figured out your Twitter handle. Thanks for following.
Lance Turbot, you remain consistent in ignoring and failing to respond to the basic math that illustrated the limits that single poll of 7 published weekly or more frequently can have, especially when it is a low rated one and five others are highly rated, and most fixated on my answer to a hypothetical universe in which there were only two pollsters reporting, one A+ reporting monthly, and and one C+.
(And I remain curious as to whether or not you’ll run your script adding a line for your “all else and Rasmussen” to show how much in your model of “all else” adding Rasmussen weighted as strongly as all others and counting every day’s report as a full weight moves that curve’s needle.)
But yes in the case of only two pollsters reporting, one high quality and one poor, I would suspect that 538 would consider the daily tracker as the equal of one report per period of time such as 3 or 4 weeks, aging its weight faster than it ages the highly rated tracker, and then applying the smoothing curve as above for the sparse data circumstance - lengthening the time intervals under consideration in the event of sparse data such that five fully counting polls are included. IOW barring the circumstance being election eve the older highly rated house’s result would still dominate the result, and should.
Meanwhile I had used Iowa as an illustration of my thought process. At the time the most recent poll available was Change Research, a C+ rated pollster, and had Warren way on top above both Biden and Sanders by 11. I stated clearly that its recency did not offset it being only C+ and that I would not believe the race had significantly shifted from the previous A+ rated result showing Biden up 9 over Warren and the even older B YouGov Biden +5 (with Sanders in second and warren in third, down by 12) or Selzer older yet showing Biden +8 until a highly rated firm said the same thing.
Would you have assessed differently? Would you have concluded that the C+ rated pollster was proof of a significant change in Iowa with Warren now on top, worth more because it was more recent?
Of course today a new YouGov is out - Biden +3 over Sanders in second and +12 over Warren, farther below Biden and Sanders than she was in the last YouGov poll.
Looking back it seems that placing the meaning of a C+ firm as of low value by itself, mostly but not completely ignoring it, was more predictive of future polling, was a more rational course, than taking it very seriously would have been.
(Note for Warren fans - she leads as second choice and as one of the ones being considered. She is still positioned well.)
All evidence indicates that one poll could have a noticeable effect with a large swing. Even a C+ polster.
You have a lot nerve assigning me tasks after your (admitted) snideness toward me throughout this thread. Do it yourself. The 538 database can be downloaded as a csv. Fire up Excel and get it done.
Alternatively, you could try asking nicely.
This is wrong in few ways. Even on day 1 the A+ would only have somewhere in the neighborhood of twice the weight of the C+ poll, and you can read right off the page that they count Rasmussen as a new data point every three days. They do this because Rasmussen posts a three day rolling average. Counting it as a new poll every three days doesn’t count any interviews twice, but also doesn’t throw any out. The idea that it is treated as one report every 3 or 4 weeks is completely unsupported.
You really can’t seem to get over the idea that a fresh C+ poll could be weighted more highly than a stale A+. I don’t know what to tell you at this point. It is obvious that this can happen and it is obvious that this should happen. You yourself have posted a couple different quotes from 538 that support the idea that this does indeed happen.
Your thought process is consistent with 538’s and mine here. 150 days from an election or 200 days from an election polls have roughly equal predictive power so it doesn’t improve a forecast to significantly decay polling this far out, particularly when polling is sparse. This was explained in a previous post.
Furthermore, a poll that is a large outlier from all other polling should be regarded with suspicion. 538 handles this by adjusting the weight of such a poll downward algorithmically. I agree with the way they handle this.
Finally, there’s a reason 538 doesn’t actually do a forecast this far out for something like a the Iowa Caucuses. The uncertainty band would be huge were they to try to put something out there now. Maybe even big enough to include that Change Research poll.
Please be circumspect in typing other’s usernames. This seems like a typo to me, but it’s always a good idea to use the appropriate handles when responding.
[/moderating]
Lance Turbo, I did request very politely in post 105 with a thanks in advance explaining that I don’t program (or do Excel). Again please and thank you. Or someone else who knows how.
Bone. An honest typo (and I suspect Freudian typo not autocorrect) but seeing it pointed out did make me laugh. I had caught myself and corrected that typo a few times before as I just have that image, like a fish slapping scene from Monty Python, in my head. Missed it this time. Sorry. No ill intent I promise.