On the right tracks?

Why do you think large trackers are flawed? Where is the market research industry going wrong?

There are a great many problems. First questions that are invalid (in other words, that don’t relate, or relate very poorly to what people actually do). Examples include: aided awareness, customer satisfaction, purchase intention, recommendation, net promoter score, memory based usage questions (e.g. ‘what brands of laundry detergent have you bought in the past three months?). Second questions that are too long and boring — and that therefore lead to poor quality responses — which in turn lead to noisy statistical models. The best examples are long satisfaction rating lists; long brand attribute association grids. And third, questions that are completely irrelevant to the particular respondent and that create statistical noise as a result e.g. questions about brands in which the respondent has no interest. These are just some of the problems. I’ve written a more complete critique in a white paper called ’The Trouble with Tracking’.

Where is the research industry going wrong? I don’t know how to answer that; it’s just plain wrong. What stuns me is the fact that people just continue to do these things even when they know they’re wrong. People are more afraid of discontinuity in their data, than of falsehoods! It’s mind-boggling. I think one of the biggest problem is the inclusion of very poor quality marketing research data in people’s KPI’s. Perhaps that’s the main way we’ve gone wrong.

Is mobile tracking currently over-hyped?

The promise and importance of mobile tracking isn’t over-hyped. What’s over-hyped are some of the methodologically shallow survey results. So we hear about surveys in Africa that involve huge numbers of respondents — but there’s nothing about the quality of the sample. Also, some mobile tracking simply perpetuates the mistakes of conventional tracking.

What are the pitfalls of mobile tracking?

Rather than pitfalls I’d like to talk about difficulties. The main one is representative sample. Far too many companies are taking the easy way out and only offering surveys on smartphones or web-enable phones. That’s in spite of the fact that 30%+ of respondents in developed countries like the USA, Germany, Japan don’t have phones that connect to the web. The situation’s far worse in the developing countries.

There are obvious things that you have to get right, like not having long lists because order effects are much bigger in mobile than online or face to face; or like keeping the survey short. But those are just ordinary matters of avoiding bad practices.

Jan Hofmeyr, chief research officer, TNS

How can the industry learn to manage the data mobile tracking generates?

Great question. Not that long ago researchers and clients were overwhelmed by the data-stream generated by passive listening apps. At TNS we’ve made a lot of progress since then. The main challenge is to organise and clean the data. But once it’s clean one can do great things with it. Two of TNS’s R&D initiatives involve generating valid brand health measures from the data; and a data taxonomy called mobile chapters.

When it comes to handling the data generated by surveys — that’s just a matter of standard data management except with a real-time element. Many companies, including TNS, have, or are, developing the capability to go straight from the phone to a client portal. We should be ready with a sophisticated live tracking offer in the next month or two.

You’re a proponent of ‘in the moment’ mobile tracking – why is that and why is the data from these more valuable/reliable?

The main benefit is that the data are more accurate because you don’t have to rely on a person’s memory. Their answer is based on what they’re actually doing, using, or seeing (or whatever). We should not underestimate the value of that. As a result we get much more accurate models of brand equity, drivers of equity, ‘in the moment’ feedback as to the challenges that a service or brand may be facing in its market.

The second benefit is the ability to turn surveys around really quickly to deliver close to real-time feedback to marketers. Although marketers tell us they want greater speed (and the old days of a 4-6 week cycle just have to end), I’ve been a bit surprised that marketers are also telling us they don’t want information in real-time when we offer it.

How can mobile tracking support predictive analytics to show future behavioural trends?

Again I think this is just a matter of much greater immediacy of feedback. For instance, we may segment someone in a way that predicts they will be one brand rather than another; or that they’ll respond to one kind of offer rather than another. With mobile we can get in the moment feedback that validates our predictions. Self-reported behaviour after the fact in traditional surveys isn’t as accurate.