Why you measure Facebook engagement inaccurately

Let’s be clear: nobody is measuring their social activity effectively. This may seem like a sweeping statement but understand that this claim is based on a five fundamentals:  

1. ‘Engagement’ is subjective.

2. Shares, comments and likes are not equally valuable.

3. People can engage with your content without liking, commenting on or sharing it.

4. Consistently using the same measurement metric is redundant if that data is meaningless, which it often is.

5. Just because you’re meeting a KPI, doesn’t make it an accurate measure of engagement.

To further clarify, you may review your social activity and be thrilled that your posts receive hundreds or thousands of likes, comments and shares; you’ve noticed that your page likes have continued to grow at a healthy rate; you’re pleased that the sentiment appears to be pretty positive and your response rate is second to none.

This is great, right? Of course it is. But HOW great is it? Harder to say. Maybe impossible to say.   What’s Already Out There? There are measurement methods that are more sophisticated than others, ones that seek to apply mathematical formulae and logic to the concept of engagement. Take ‘Engagement Rate’ for example. Social Bakers defines its Engagement Rate metric as:

One would assume that the purpose of this metric is is to assess what percentage of your audience has engaged with your content. However, while this formula will produce a figure that is someway representative of how well a page is performing, it certainly doesn’t do what it purports. So should we trust it and consider using it?

The Problems with Engagement Rate  

There are a number of well documented problems, including the formula’s inability to distinguish between interactions by fans and non-fans, because you don’t have to be a ‘fan’ to interact with Facebook content.   And what if a ‘fan’ has watched a video or clicked on the link you posted on Facebook but didn’t like, comment on or share it?Does that not count?

Thirdly, this formula only looks at the total number of fans the page has, which isn’t representative of how many people have seen content. Facebook provides data on the number of people reached, inclusive of organic and viral reach, and it seems this figure is more meaningful than the number of people liking the page who won’t necessarily have seen your content.

Finally, this formula assumes that each like, comment or share is from a unique user. If a Facebook page has posted three times in a day and one user has liked, commented and shared all three, they are responsible for six interactions which this formula translates to six unique user interactions.

The Formula, Evolved

This gives rise to a slightly evolved formula, one that, admittedly, still doesn’t account for one user making multiple interactions:               This formula has the added benefit of producing a higher Engagement Rate figure as the number of fans reached will always been lower than the total number of fans.

Why This Still Isn’t Good Enough

As previously mentioned, likes, comments and shares are not of equal value, despite this formula quantifying them as such. It would be far more meaningful to implement a weighting system to likes, comments and shares. A study by Edgerank Checkershowed that a comment was about four times more valuable than a like – this was measured by comparing the average number of clicks a link posted to Facebook received per like vs per comment. So in short, a link that receives 10 likes will receive approximately 30 clicks whereas a link receives 10 comments will receive approximately 120 clicks.

To my knowledge there hasn’t been any research done into clicks vs shares, however it would be sensible to assume a person sharing content will invariably have clicked and consumed the content they’re sharing before they choose to share it with their friends, suggesting a share could be more valuable than a comment.   Without proper analytics and research into a target audiences’ existing behaviour, these figures may seem a bit arbitrary.

The value of a like, comment or share will vary from Facebook page to Facebook page. It would certainly be worth doing some research, looking at the analytics from previous Facebook activity and seeing how click-throughs to content correlate with the number of likes, comments and shares that it receives. Of course, this is only applicable to Facebook posts that have a direct call-to-action such as a link or a video. It would be invaluable to know how such weighting varies from sector to sector, if at all.

The Formula, Evolved (Redux)

For argument’s sake – and for the purpose of evolving this formula – let’s suggest that a like should be assigned a value of one, a comment the value of four and a share the value of 10.   These tables assume that there were 750 likes, 200 comments and 50 shares across three Facebook posts, reaching a combined total of 5,000 people that day.

Thus, Weighted Engagement Rate is born. As with the previous evolution, this produces a higher number, one that will not only look more impressive to clients but will also be more significant in terms of measuring engagement – the very thing we want to determine. It encourages genuine engagement by weighting comments and shares more heavily. So, ultimately, if you produce great content that people want to share and talk about (comment on), you’ll be rewarded with a higher engagement rate, if you use this formula:

It is important to note that this formula is dependent on specific research into a Facebook page’s existing audience and how they respond to content. It could be possible to produce industry standard weighting that takes an average across all sectors. This would only be beneficial if the numbers were fairly similar, however.

What do you think? Should brands look to Weighted Engagement Rate to measure their performance on Facebook? What’s the formula missing? I’d be keen on hearing your thoughts.

Toby Margetts is an Engagement Executive at Beyond.” His twitter handle is @tobymargetts
  • Phil@Aberfield

    Finding a consistent and accurate measurement of social activity is the Holy Grail of digital engagement. Weighting comments and shares makes sense, but the formula has to assume that all comments and shares are positive – which they won’t always be. And, as you say, any formula would need to be developed to take account of the type of page it is, and/or the brand’s industry sector. I’m all for a formula that works for Facebook engagement, but I just have my doubts we’ll ever find one that includes all the variables. Obviously if there’s a missing variable then the formula doesn’t work.

  • Toby

    Totally agree Phil. This does assume that each comment and share is for a positive reason which, of course, it isn’t always. Very difficult to find a non-time consuming way around this one.

  • Pingback: Facebook Engagement: We’re Still Doing It Wrong - AllFacebook()

  • Pingback: Facebook Engagement: We’re Still Doing It Wrong | Everything About Facebook!()

  • Pingback: Facebook Engagement: We’re Still Doing It Wrong | Facebook Opportunity()

  • Pingback: Facebook Engagement: We’re Still Doing It Wrong | iBusinessPage.com()

  • Pingback: Facebook Engagement: We’re Still Doing It Wrong | Breaking Internet Marketing News()

  • Pingback: Facebook Engagement: We’re Still Doing It Wrong | fbaffiliatex.com/blog()

  • http://twitter.com/page23 Tara Davanzati

    In regards to Toby and Phil’s comments, I’d agree, but that’s when you represent sentiment alongside this information so you can get an idea of if the reach your page is getting, the engagement of that community, and the overall ‘feeling’ of that community. Speaking from experience, Community Managers do their best to diffuse negative comments and turn them neutral or positive, so I can see the above formula working very well as long as page sentiment is also tracked on the side. Rarely would we see an engagement score alone anyway. It’s almost always shown beside page likes, impressions, sentiment, and response rate.

  • http://twitter.com/sjgriffs Sophie Griffiths

    xxxx

  • http://twitter.com/sjgriffs Sophie Griffiths

    I am not sure whether agonising over likes, comments and shares really makes an awful lot of difference. After all that’s pretty shallow engagement in the grand scheme of things. Of course, positive interactions on a page are good and you need to make sure you reduce the negative but overall they do little to enhance brand affinity/loyalty etc – after all we’re only measuring this of existing fans and their friends anyway which is a stagnant base that you just have to keep happy. These metrics should not be the core of any social media strategy, nor should they be the benchmark for success or failure. If anything we need to investigate WHY people share and do more around that, and not worry too much about the rest. The most successful brands in social are excellent content publishers and they do not use social media in isolation. Get that right and the rest will follow.

  • http://www.thoughtlabs.com/ John Maver

    With regard to Toby’s comment, I don’t think that it matters whether the comments are positive or negative. If you are a car manufacturer and your post gets people talking about your car, even the negative comments can be very useful to understand consumer thinking. And, regardless of the commenter’s sentiment, their action still has the potential to drive additional actions from their friends who may agree or disagree. In then end, people are still interacting with your brand.

  • http://www.thoughtlabs.com/ John Maver

    I think the idea of a weighted metric is useful, but not all posts have the same purpose for gathering user actions. Many posts may be designed to garner a particular type of action – you see Coke’s “Like this if you are thinking about summer”, while others may be topical conversations needing quality comments to be useful.

  • Pingback: Facebook Engagement: We’re Still Doing It Wrong - Small Biz Growth with BizChris()

  • Pingback: PR News: Weekly comms round up 14/03/13 - Cision UK()

  • Pingback: PR News: Weekly comms round up 14/03/13 - Cision UK()

  • Pingback: Facebook Engagement Measurement: We’re Still Doing It Wrong - Engage Social()

  • John_Bartlett

    This is an excellent piece. As soon as I saw the Social Baker ‘s engagement rating formula I knew something was extremely inherently wrong with it. Namely, it’s biased towards pages with less fans. The reason being, the more fans you have the harder it is to lift your engagement score both practically and mathematically. Ultimately this creates serious issues when companies or brands decides to use the engagement score literally, to compare themselves against their competitors. In fact the Social Baker’s business model is based on these types of comparisons.