Political Polling

Did you ever wonder how pollsters can keep coming up with claims that 90+% of Americans want universal background checks preemptively-prove-your-innocence prior restraint on rights when every time it goes to the voters actual results never come within 30 points of the claim?

Poll: Vermonters strongly support new gun-control measures
More than two of every three Vermonters say they support the new gun-control measures signed into law by Gov. Phil Scott this year, according to the results of a poll conducted by Vermont Public Radio and Vermont PBS.

Vermont is pretty liberal, so maybe that shouldn’t be a surprise. But it’s a bit of a strange state when it comes to RKBA. When I lived in New England, I often heard how much Vermonters like their guns; libertarian types would note it with approval, liberals with befuddlement, and conservatives made jokes about “heavily armed hippies.” But everyone knew it.

So how did VPR/VPBS come up with results indicating such heavy approval of gun control? The secret is in the polling methodology, something I like to look at nearly as much as alleged results. In this case, the clue is:

“For the landline sample, interviewers requested to speak with the youngest male member of the household who is at least 18 years of age; if there was no male in the household, interviewers requested the youngest female.”

They actively screened for the youngest voting-eligible demographic. The group which most strongly leans liberal, statistically speaking. That accounts for the liberal skew in the results.

I mentioned this to someone, as an example of the worst built-in survey bias methodology I’d ever seen. And that person told me something of which I was unaware: that most phone poll calls she’s gotten do the same thing. It’s been years since I participated in a phone poll,* so I hadn’t realized this. It’s the first time I spotted that selection criteria in methodology notes (usually I see a pro-urban selection bias, and over-representation of Democrats compared to the general population).

Those young adults also tends to have the lowest voter turnout, which accounts for the fact that polls rarely match voting reality.

That age selection game is particularly problematic for Vermont.

“Vermont faces a demographic challenge. Our population is stagnant and getting older. We have fewer school-age kids, which drives up the per-pupil cost. We have fewer young adults to invigorate the workforce and pay forward the costs of retirement and health care for older Vermonters.”
The Mass Exodux Myth

But VPR and VPBS think that comparatively tiny group speaks for the older folks — gun-toting hippies — who greatly outnumber them.

So when you hear that “95% of Americans want to a$$-rape the Constitution,” remember that it really means “95% of millenials who probably aren’t going to vote, and think the Army carries semi-auto AR-15s, believe bump-fire stocks are machineguns, and expect blackmarket arms dealers to conduct universal background checks, want to bend you over the table and have their way with you. Without vaseline.”

* That’s because I don’t answer calls with blocked caller ID, or unrecognized numbers. If Pew, Gallup, or whoever wants to poll me, their caller ID should say Pew, Gallup, or whoever. And the last few poll calls I did take (years ago) turned out to be push polls, and I hung up on them.

Carl is an unpaid TZP volunteer. If you found this post useful, please consider dropping something in his tip jar. He could use the money, what with truck repairs and recurring bills.



3 thoughts on “Political Polling”

  1. After the many years I spent working with a motorcyclist lobby, I am always suspicious of poll responses. You know, 57% want this, 97% agree with that. In working on the lobbying efforts I found out that it was how you phrased the question, and where you asked it, that allowed to you get the % answer you desired.
    If I asked 100 people, leaving a church in an affluent area; ” Do you think motorcyclists should wear helmets to protect themselves when riding?” I’ll bet the percentage of “yes” would be in the 90’s.
    HEADLINE : 95% of people think there should be a mandatory helmet law.
    If I asked 100 bikers leaving a strip club, any strip club, on a Friday night, ” Do you think bikers should be forced to wear helmets for your own good?”, odds are it would be the opposite.
    HEADLINE : (Oh who am I kidding, there’d never be a headline. Or any other mention.)
    See “when” and “how”.
    So next time someone starts quoting poll percentages, remember they set the the “when” and “how” so they get the answer “they” want.

    1. There are a lot of tricks. Choice of answers is another. I ran across a survey being run by a college student on enviro/conservation issues aound ’99. It had somehow gotten pushed to a libertarian discussion list. But when any of us tried to participate, all of the choices ranged from “more government control amounting to taking” to outright nationalization. “Other” was not an available option.

      I contacted the student to object to the limited selection, noting that a good many solutions that had worked well in the real world weren’t allowed for.

      She explained that her professor told her these were the only right answers, and that nothing else was needed.

Leave a Reply

Your email address will not be published. Required fields are marked *