I’ll repost an easier to digest example that I posted in a different thread.
Here’s a slope chart of PredictIt probabilities (debiased for favorite/longshot bias) compared to 538 for races where either gave any party a 10% or greater chance to win. There were 18 such races and no surprises outside these 18.
538 is taken from when they froze their forecast early 11/3. PredictIt data was the average of several observations on up to midnight on 11/2. So roughly the same timeframe.
538 was more Biden leaning than PredictIt pretty much across the board.
Here’s that same info in table form with a column for results.
| Race | EV | PredictIt | 538 | Result |
|---|---|---|---|---|
| SC | 9 | 0.004 | 0.100 | 0 |
| AK | 3 | 0.015 | 0.149 | 0 |
| MT | 3 | 0.027 | 0.180 | 0 |
| IA | 6 | 0.140 | 0.399 | 0 |
| OH | 18 | 0.162 | 0.453 | 0 |
| TX | 38 | 0.238 | 0.382 | 0 |
| GA | 16 | 0.383 | 0.582 | 1 |
| FL | 29 | 0.417 | 0.691 | 0 |
| ME-02 | 1 | 0.520 | 0.573 | 0 |
| NC | 15 | 0.561 | 0.639 | 0 |
| AZ | 11 | 0.645 | 0.680 | 1 |
| PA | 20 | 0.749 | 0.843 | 1 |
| NE-02 | 1 | 0.837 | 0.717 | 1 |
| MI | 16 | 0.849 | 0.947 | 1 |
| NV | 6 | 0.876 | 0.878 | 1 |
| WI | 10 | 0.887 | 0.944 | 1 |
| MN | 10 | 0.897 | 0.958 | 1 |
| NH | 5 | 0.910 | 0.891 | 1 |
If you calculate Mean Squared Error across these 18 races, you get:
PredictIt → 0.0849
538 → 0.1225
Lower is better.