Yesterday I began to see in my tweet stream messages telling me to head to smh.com.au and vote in a poll on gay marriage. According to my tweeps the poll was mentioned on conservative Christian websites, meaning a slight majority of respondents ‘against’ gay marriage had emerged, and we should all head to smh.com.au to fix this.
As I’m sure you know, these polls are not worth the pixels they are rendered in. They suffer from what researchers call self-selection bias, meaning that rather than reliably representing the views of the readership, it just shows which self-selecting groups were most mobilised to vote. Less a poll, more a clicking contest. Even the SMH itself acknowledges this, tagging the disclaimer: “These polls are not scientific and reflect the opinion only of visitors who have chosen to participate”.
It’s thus pretty easy to game these polls, especially when you have a net-savvy constituency on one side of the debate. Those against the Australian government’s policy of mandatory internet filtering (as I am) routinely use their online networks to game polls on the topic, such that many show 95%+ against the internet filter. This despite independent polls from professional research organisations showing an 80% majority support the policy.
But it’s all just harmless fun, no? I used to think so, but now I’m not so sure. The reason being is that media outlets (and I’m picking on SMH here, but it’s arguably much worse with TV channels and talkback radio) insert these polls seamlessly into their serious journalism and imply a credence they don’t deserve.
So the SMH puts the poll (with tiny disclaimer) at the top of a story about gay marriage by their chief political correspondent. This lends the poll a weight it does not deserve. Often you’ll read within a news story “An SMH poll showed …”.
If a newspaper journalist had a source for a story that they knew was shifty, unreliable, and driven by a huge unspoken bias, they wouldn’t put a quote from them right at the top of their story would they? Because that is exactly what is happening here.
My view is this: unmanaged online polls are unadulterdated horseshit. If media outlets are to do them, they should cover them in disclaimers, and not report on as if they are a reliable source when they know full well they are not.
If they are going to do them, why not manage the polls correctly? A pop-up poll on their site could be done relatively cheaply and easily to an acceptable standard, with checks in place to verify identity and guard against multiple completions. And most importantly, DON’T allow people to share the poll with their (like-minded friends) via Twitter and Facebook. Then they would actually have something worth writing about, and would actually learn something about their readership (for once in their worthless lives).