Online sampling concerns top of mind in quality debate

Ask a room-full of researches to talk about the quality issues the industry faces today, and it’s a safe bet that online sampling will be one of the main topics up for discussion.

That was the case as IJMR editor-in-chief Peter Mouncey, Mesh Planning’s Fiona Blades, Ipsos Mori’s Ben Page, Heineken UK’s Frances Dobson and Real Research’s Adam Phillips gathered for a panel debate.

Dobson kicked things off by acknowledging the benefits of online sampling, but asking whether researchers really know enough about their respondents, and that they are who they say they are.

For Phillips, the concern was about whether online samples were managed properly to be truly representative while Page wondered whether the industry was losing a skill set because researchers are simply “doing what the computer tells them” and don’t understand the basics of things such as sampling and weighting.

“Are we having a go at online just because we can” asked a voice from the floor? No, replied Dobson: “There are concerns about panels. A lot of agencies can’t answer when asked how representative their sample is.”

But Survey Sampling International chairman Simon Chadwick blasted the panel for “having a conversation routed in the past and ignoring the elephant in the room.” The time to be discussing panel quality was between 2002 and 2009, Chadwick said, and the industry should now instead be focusing more on “how we treat respondents online and the garbage we put in front of them”.

Ipsos Mori boss Ben Page acknowledged survey design as an are the industry “needed to do better in”, but he maintained that there will always be concerns and questions asked about the people that take part in online surveys while there is nobody to “go and knock on the door” to find out more about them.