After major elections the question reliably arises why the polls were wrong again. What usually follows is either an accusation of manipulation or the conclusion that polls are worthless.

Both miss the mark. The error sources are well studied, largely technical, and the greatest source of frustration for the pollsters themselves.

The sample is the problem

The core of any poll is a small group meant to represent a large one. With a thousand respondents the pure sampling error sits at roughly three percentage points — that's mathematics and can't be argued away.

Which already means a two-point difference between two parties is statistically almost nothing. It still gets reported as though it were a change.

The bigger problem isn't the size of the sample but its composition. And that has deteriorated considerably over the last twenty years.

Who doesn't answer

Response rates for telephone polling have fallen dramatically. Where a substantial share of those called once participated, today it's only a few percent depending on the survey.

When 95 out of 100 people called don't take part, the decisive question is no longer who you selected at random but who was willing to answer.

And that willingness isn't randomly distributed. It correlates with age, education, availability, trust in institutions — and possibly with political preference.

This is probably the single most important error source in modern polling, and it can't be fixed by increasing sample sizes. A larger biased sample is equally biased, just more precisely wrong.

Weighting

Institutes correct for this by weighting. If people with lower formal education are underrepresented, their answers count for more.

That works as long as you know which characteristics need weighting. For age, gender and region there are reliable population figures.

It gets harder for characteristics that correlate with voting behaviour and for which no reference data exists. If people with low institutional trust participate less and vote differently, that's very difficult to correct, because you don't know that group's share of the population.

One additional point often overlooked: when weighting by past voting behaviour, you're relying on recollection. And people misremember systematically — there's a known tendency to report having voted for the winner.

Who actually turns out

A poll measures intentions. An election measures actions. Between them sits a gap that varies in size.

Institutes try to model likely turnout, and that modelling is one of the largest error sources of all. It rests on assumptions, and when an election mobilises or demobilises unusually, those assumptions are wrong.

It's one reason polls perform particularly badly at elections with unusual turnout.

Late deciders

The share of voters who decide only in the final days has risen substantially over the decades. Party attachment has weakened and willingness to switch has grown.

A poll conducted ten days before an election simply cannot capture that movement. It may have been entirely accurate at the moment of fieldwork.

This gets routinely ignored in public discussion. A poll is a snapshot, not a forecast, even though it's read as one.

What happens to the numbers

And then there's the reporting problem, which has nothing to do with methodology.

Media report polls like sports results. A one-point movement becomes a headline even though it sits inside the margin of error and is almost certainly noise.

The institutes publish their margins. In the reporting they usually don't appear, because they ruin the story.

Why we still need them

Worth adding a note on the methods that have partly replaced telephone polling, since they carry their own problems.

Online panels are cheaper, faster and can reach large samples. But participants opt in, which means they are self-selected, and people who volunteer for survey panels differ systematically from those who do not — they tend to be more politically engaged, more online, and in some panels they answer a great many surveys, which brings its own distortions.

Good panel providers correct for this with careful recruitment and weighting, and the better ones perform comparably to telephone work. Weak ones produce numbers that look precise and mean very little.

Which leads to a practical suggestion: the method matters as much as the sample size. A poll of two thousand people from an unvetted online panel is not more reliable than a properly constructed telephone survey of a thousand. That information is usually published in the methodology note, and it is almost never mentioned in the article reporting the result.

For all that: polls are the best instrument we have for finding out what a society thinks. The alternative isn't better information, it's gut feeling and whoever shouts loudest.

And they're roughly right at most elections. The spectacular failures stick in memory; the many serviceable forecasts don't.

My suggestion for handling them: look at the average of several institutes rather than individual surveys. Ignore any change under three points. And treat every poll as a description of the present, not the future.