The "Margin of Error" Does Not Mean What Journalists Think It Means

In the Princess Bride, the kidnapper Vizzini keeps using the word "inconceivable" to describe various events until Inigo Montoya states "That word, I do not think it means what you think it does." This is a good metaphor for the continual, repititous mis-use of the political polling "marging of error."

Polling is a branch of statistics which is mathematically sound in every way except when it comes to evaluating the relative positions of candidates vis a vis the voters. This is because to get an accurate sample for your poll, you have to know what the general population is. Gallup or Quinippiac or whomever can accurately poll the American people on anything but politics because the U.S. Census Bureau provides data on the entire U.S. population which allows them to accurately gauge the citizenry's preferences based on a small sample.

However, for political polling the population of voters is undefined until after the election and polling organizations have to make assumptions based on historical and other models of what the voting population will look like in order to draw a sample from which to poll. This is why political polling is as much art as science.

Now we get to the part where journalists continually demonstrate their poor educations. The size of the sample directly correlates with the poll's margin of error, e.g., the larger the sample size the smaller the margin of error. Generally any poll with a margin of error greater than 3.5% is worthless and there's no point really paying attention to it. Political campaigns use polls with larger margins of error (and smaller samples) because they are easier to obtain.

The "margin of error" in actuality is the percentage chance that the sample population is completely wrong on either side of the bell curve. So a margin of error of 3.5% means that the poll has a 1 in 30 chance of being COMPLETELY WRONG because the sample is off.

Reporters frequently report that if Candidate D beats Candidate R by 2% in a poll and the margin of error is 3% that "the candidates are statistically tied." That is in no way true. It means that Candidate D has 2% more support than candidate R, but that there is a 1 in 33 chance that the entire poll is wrong, meaning either candidate could be ahead by 10%, 20% or more. By constantly repeating this defective meme the mainstream media perpetuates a misunderstanding of polling by the general public as well as constantly reminds the educated among us that they are undereducated with no real understanding of what they are reporting on.