As always, The Washington Post offers, er, interesting political math.
It tells us that in the seven battleground states that would probably determine the electoral vote in the Presidential election, Vice President Kamala Harris leads in three, former President Donald Trump leads in two, and the other two are ties. A tie is defined as a margin of a quarter of a percentage point. No explanation.
But digging into the story, we read that "every state is within a normal-sized polling error of 3.5 points and could go either way." In other words, all seven states are too close to call. Neither Harris nor Trump leads in any of them merely because they lead in the sample. The sample is never a perfect reflection of all voters, and one must consider how imperfect the reflection may be before basing conclusions on the sample.
For example, suppose that in Pennsylvania, Harris led in the sample by one vote. Would we conclude that she is winning the Pennsylvania race? Surely not. A one-vote margin is tiny. It is very likely that one winning vote results from an error, such as voters who misunderstand the question. So we would not put much faith in the conclusion that Harris is truly ahead.
How large must the margin be, then, for us to conclude that it gives us good information? The answer to that question is a statistic called the "margin of error."
The Post tells us that the margin of error is 3.5%. The usual interpretation is that if the margin exceeds 3.5%, then the probability is 95% that the leader in the sample is truly winning.
But here is where the Post math gets really interesting. The explanatory notes say that the 3.5% estimate is based on a calculation that in "the last few presidential cycles...the average modeled polling error in competitive states was 3.5 percentage points." Which presidential cycles? Which competitive states? Were they the same as this year's seven battleground states? Who knows?
Well, OK, 3.5% is the "average" polling error. I presume that this means that chances are 50% that the candidate ahead by 3.5% in the sample is actually winning the race. I presume wrong. Reading on: "...To account for this [3.5% polling error], our averages factor in the 90th percentile possible error (i.e., how bad would the error be in the worst 10% of cases)." In other words, chances are 90% -- not 95%, not 50% -- that the candidate ahead by 3.5% in the sample is truly winning. Feel free to scratch your head.
Freaky fractions
Sports fans, here's the score. Usually, the margin of error is based on the probability distribution. This is the range of probabilities for particular outcomes. For example, a probability distribution for the outcome of one coin toss is 50% no head (that is, a tail) and 50% one head (no tail). The distribution for the outcome of 100 coin tosses can give us the probability of zero heads (or 100 tails), the probability of one head (99 tails), and so forth. All distinct probabilities sum to 100%. For example, on a coin toss, there is a 50% chance of no head and a 50% chance of one head, adding up to 100%.
The distribution of a Harris margin may be the probability of minus 100% (that is, she got no votes), the probability of minus 99% (she got 1% of the vote), etc. We could also look at fractions like minus 99.9%.
The most common distribution used is the normal. This has a bell shape: Small probabilities at the extremes (like minus 100% of the vote for Harris, or plus 100%) and large probabilities in the middle (like a zero margin for Harris, that is, both candidates get the same vote).
The probability distribution is a theory. But it leads to accurate conclusions when correctly handled. For example, if we observe that Harris loses 100% of a well-executed and large poll sample, we may confidently conclude that she is not winning the race. To calculate the precise margin of error, one fits out the probability distribution by using information from the poll sample.
But The Post derives its margin of error not from a probability distribution but from recent actual errors. Its information came not from the current sample but from past performance. How it gets from this estimate based on past empirics to the present theoretical one is beyond me. Perhaps it assumes the same probability distribution for past election cycles as for the present poll samples, but The Post says nothing about this. It looks to me as if it arrived at its estimates essentially by playing pin-the-tail-on-the-donkey.
To recap: Harris is winning in three states! No, wait a minute. We're not sure. It could be an error. No, wait. We're not sure how to calculate the possible error. No, wait....
This matter is serious, and not just for nerds like me. The Post is wrong about how close the race is. It's too close to call not only across the battleground states on average, but in every battleground state. The Post's nonchalance would lead campaigns to understate the need for staff, volunteers, ads, and money in most battleground states.
The Post's FAQ asks: "Are you going to release the code of your model?" The newspaper replies: "We really want to and are working on that." Outstanding. Shouldn't The Post have released the code when it published the results? One delays code publication to clean up confusion and error. Why didn't The Post clean up the code first?
Continuing: "When we release the code, we're also hoping to publish a more technical explanation." In other words, The Post did not think through its assumptions, since one does so by writing out their justification. The Post winged it.
Democracy dies in darkness. And The Post is smashing the lamps. -- Leon Taylor, Seymour, Indiana, tayloralmaty@gmail.com
References
Lenny Bronner, Diane Napolitano, Kati Perry, and Luis Melger. Harris vs. Trump 2024 presidential polls: Who is ahead? - Washington Post September 13, 2024.
