Measuring social is harder than you think

At first glance social looks like an easy medium to measure. In one sense, social is the most measurable of channels, with every element from views, to shares, to clicks, to plays leaving an electronic trace of what has happened. However, as social media spending grows, advertisers are becoming increasingly concerned that they are struggling to validate the effectiveness of their campaigns and activities.

It turns out that when dealing with social, counting is really easy, but measuring is much harder. There are many reasons why measuring social is hard but some of the key reasons are:

• Social is usually used in conjunction with other channels. The evaluation of social has to measure direct effects and also how it interacts with other elements. For example, to what extent does TV advertising impact Twitter, and to what extent does Twitter impact TV advertising, and how does the combination of the two impact sales?
• Even within social there is a complex interaction between Paid (buying placement), Owned (using the brand’s own pages/sites) and earned (the word of mouth and hoped for viral effect); unpicking cause and effect can be difficult.
• The metrics for social are often used to manage campaigns or activities. While this can improve the performance of the campaign, it can also confound the measurement.
• Social is often less controllable, from a measurement point of view, than traditional media. For example, creating a control sample that will not be exposed to a social message may be impossible.
• Because social is the most measurable of all media, problems tend to arise from having too many choices of data rather than too few. With too many measurements, some of them are going to look positive, even if only by chance.
• Social is often not time bound in the way traditional campaigns are, i.e. there may not be a clear before, during, and after. For example, the Cadbury Gorilla ad uploaded in 2007 to YouTube is still collecting plays, shares, and comments.
• Although the objectives of a social campaign/activity should always be linked to business objectives, they are not always linked directly to them. Social is often used to achieve tactical ends, such as collecting information, encouraging trial, or simply generating engagement. Evaluation needs to be tailored to the objectives – not simply tied to ROI.
• The budget for social activities/campaigns is often smaller than for, say, TV. Also, the number of social campaigns/activities may be larger, which means the measurement approaches need to be appropriate to the circumstances – for example, a full blown media mix modelling approach is not always going to be viable.

To illustrate the difficulty, imagine a campaign based on TV advertising, social media, and point of sale, designed to encourage people to buy a specific coffee and to post a message to their friends about it, in order to encourage both purchase and copying behaviour. If somebody posts or reads a comment, does that make them a) more likely to buy a coffee, or b) does being more likely to buy a coffee make it more likely they post or read something? The answer, from a variety of studies, is sometimes a), sometimes, b) and usually both. One problem for social is that there is often a lack of the counterfactual – i.e. how many coffees would have been bought by these people if the campaign had not run?

Aren’t some people already measuring social?

There are companies measuring some aspects of social, and many of them are doing a good job. However, when we talk to the companies spending the money, they tell us there is no consistency of message, no consistency of method, and no body of knowledge they can turn to when they want to determine the right approach.

There is a growing realisation that the lack of established practice about the evaluation and measurement of social is costly and is holding back innovation. In order to tackle this issue, a cross-industry initiative has been formed. This project, named #IPASOCIALWORKS, has been created to provide advice on measuring and evaluating social.

#IPASOCIALWORKS brings together the Institute of Practitioners in Advertising (IPA), the Market Research Society (MRS) and The Marketing Society, along with the key social platforms (Twitter, Facebook and LinkedIn). This initiative is believed to be the first time such a wide collaboration across the industry has been formed, which reflects the urgent need to establish clarity on how to evaluate the business value of social.

The project comprises the following elements:

• The creation of a collection of peer reviewed case studies that show ROI delivered via social. The key benefit of these case studies is that they highlight the steps required to validate claims about ROI. You can access the case studies online.
• Industry events focused on the need to move from counting to measuring, covering things like the need to ‘bake’ the evaluation process into the design of social campaigns and activities.
• The creation of a Guide To Evaluating Social; I am the lead author along with a team of industry luminaries advising on its content and format. The guide will be launched in the autumn.

The Guide to Evaluating Social is nearing completion, but there is still time to help by reviewing the current content and sending in suggestions and ideas. You can download the current summary and draft of the guide.

The #IPASOCIALWORKS team would love more case studies, so if you have an interesting case where you can show ROI, in a way that can be peer reviewed, visit https://www.mrs.org.uk/mymrs/socialmedia.

More information on IPA Social Works can be found here.

Ray Poynter is the author of the Handbook of Online and Social Media Research and the soon to be published Handbook of Mobile Market Research. He is the founder of NewMR.org, director of Vision Critical’s knowledge sharing centre, and managing director of the Future Place.