Research measurement needs to think outside the box

Billboard ooh by Christiano Betta, FlickrThinkbox’s latest Pathways To Profit research, carried out in conjunction with Ebiquity, proclaims TV advertising creates the most profit for advertisers. The study also suggests out-of-home (OOH) advertising delivers a negative return on investment.

Can this be true? The world’s longest-running ad medium has been delivering negative ROI for the globe’s most successful brands all along? No.


The issue here is how ad media are measured against each other. The Thinkbox research measures profitability in the short to medium term – think months, not years. This is a simplistic metric to choose, especially considering the recent work of Peter Field and Les Binet from the IPA and Thinkbox that outlines the need for measuring and understanding the long term effects of advertising – think years, not months. Could it be that clients more predisposed towards TV advertising (for example, supermarkets) display more profit in the short term?

Many of Kinetic’s clients use OOH to reach an upscale and urban audience so it’s short-sighted to measure the medium at a national spend level. A financial client may utilise OOH to reach an upscale audience in Canary Wharf. Potentially their campaign will deliver a negative national ROI in short-term profit. That’s not the point of the campaign.

We need to remember what OOH is good at – scale, ubiquity, frequency, and stature. Contrary to what other ad sectors may have to say, advertising doesn’t need proof of instant interaction to measure success.

As a researcher it is important to think about what you are measuring. As a brand manager and media planner it is important to know how you are expecting your advertising to work. Put Mo Farrah in a sprint against Usain Bolt and we all know who will win. Is that the best way of measuring who is the fastest runner alive today?

This is about trust and accuracy. We can’t see all of the inner workings of the research model, so we have to trust that the ROI data in the model is fair. As it stands, it seems as though this analysis uses investment data that is not matched to the return. Without that, it isn’t a fair analysis.

We’ll leave you with Professor Byron Sharp’s well-timed tweet in response to the research:

Jennie Sallows is head of insight at Kinetic

  • Neil Mortensen

    Hi Jennie,

    We’ve spoken to Ebiquity for some clarity on this. On your point about not aligning local OOH investment to national sales performance, Ebiquity rarely depends on national level models solely. To enhance the robustness of their modelling outputs, they model across a wide range of sales hierarchies (e.g. TV region, postal sector, store, retailer) and so they can model the relationship between media and sales from many angles. This helps them achieve multiple observations per ad medium to ensure they are observing uplift consistency. A key approach is to exploit the differences in media weight at a local level and to measure how this translates into sales performance.

    As you say, how is it possible to measure the impact of OOH posters in Canary Wharf at a national level? Exactly; there is very little chance of aligning the true effect in a higher level model. Therefore, they would construct a series of models at a much lower level of granularity to detect the true level of increment.

    You say that OOH is much more about the long term than the short, and we fully support the IPA’s work into this that you cite. Ebiquity’s ROI analysis is not just about the week of exposure or the campaign but also significantly beyond the campaign. Their study measured incremental sales generated in the months post campaign – and we can be talking about up to 12 months post exposure. So their modelling approaches give ALL media lines a fair opportunity to demonstrate their true value. Anything beyond 12 months is notoriously difficult to measure via statistical techniques. And yes, if the impact of OOH does miraculously kick in across years 2 and 3 then it is possible that traditional ROI techniques may be doing OOH a disservice. However, this is doubtful; if an ad medium is showing limited evidence of commercial return across 12 months then it is very doubtful then it will generate value in the longer term.