Engineers Find Facebook Mobile Data Up To 80% Inaccurate
- by Laurie Sullivan @lauriesullivan, November 9, 2016
Rakuten Marketing engineers believe they have uncovered a measurement flaw in Omniture, Google Analytics, Coremetrics and other analytics packages that measure the click-through rates (CTRs) and cost per clicks (CPCs) for Facebook mobile campaigns.
In Rakuten's Facebook Measurement Divide report released Wednesday, containing the analysis of client performance data, the company reveals discrepancies between Facebook conversion tracking and Web analytics costing advertisers insight into 192% more attributable revenue and higher return on ad spend.
The cross-device campaigns analyzed reveal that attributable revenue only comprised on average 5.6% of the total revenue generated across mobile-only, desktop-only and cross-device campaigns -- and as little as 2.4% for one retailer in the study.
Bob Buch, SVP of social at Rakuten -- which supports attribution, affiliate, search, mobile, lead generation and more -- said when the company began digging into clients' campaigns it found that the CTR metrics were significantly higher and the CPCs quite a bit lower. "Omniture was missing 80% of the revenue, which explains why marketers are not investing more in mobile," he said. "I'm not saying there's something inherently wrong with the platform, but I do know it is not measuring mobile accurately for nearly every client we work with."
Buch believes the tracking is inconsistent with what advertisers see in their Web analytics for several reasons. For starters, there are technological challenges that prevent conversion tracking on Facebook from functioning correctly, he said. In other words, there are additional conversions happening that are simply not recorded anywhere.
The report goes into more detail, outlining how post-click conversions are tracked differently by Facebook conversion tracking than in Web analytics. It also suggests that Web analytics platforms that rely on cookies cannot accurately track cross-device conversions because of the inherent challenges with identifying consumers across devices, and that some discrepancies are attributable to certain mobile operating systems and Internet browser combinations blocking third-party cookies.
Buch sees some of Rakuten's bigger clients apply what he calls a "mobile multiplier." He also says that it will be interesting to see what Adobe, Google and other platforms do to correct this discrepancy. When asked whether Rakuten sees this discrepancy with other social sites, he says, "I suspect this type of discrepancy would happen on any walled garden where there's a mobile app linking to a mobile Web site, but truthfully the other social sites are not advanced enough to see the data at scale. We just don't have the data."
OR...as I've said before...Facebook is inflating its metrics. I would like to see the industry do a balanced investigation of this issue. Facebook certainly has motivation to show higher metrics to support its advertising value. When I look at our FB stats, they don't really match up at all to what I've seen as normal human behaviors. I've been looking at online metrics for 20 years, and while I'm sure the truth is somewhere in between the platforms, I am hard pressed to defend Facebook's reporting. I am also hard pressed to understand why Rakuten would take this position in general, without at least exploring alternate explanations.
Those of you who advertised in the early days of Google will recall the consistent impression inflations and click fraud that went on with its ad partner network, and how that drove their revenues up at a critical time in their company. Why aren't you taking a critical eye at this new development?
The headline of this article is misleading. It's suggesting that Facebook's reporting is incorrect and further fuelling lack of confidence in Facebook's data, which isn't what the report is showing. The Rakuten Marketing Insights report is showing that web analytics and 3rd party tracking are UNDER-attributing revenue to Facebook. This is not about Facebook's incorrect reporting, in fact Facebook's reporting is MORE accurate because 3rd party tracking relies on cookies and Facebook does not. Facebook is able to track cross-device conversions & revenue, which 3rd parties struggle with because of limitations with cookie tracking on some operating systems.
Comparing very flawed against very flawed is futile. Facebook's version of the truth is consistently incredulous. We measure holistically attributed, cross-device, click attribution from FB and its contribution to our client's businesses is usually very modest. Even with a healthy multiplier for "view" credit (which FB won't let us properly tag with third party tools to measure) FB is small.
Using GA/Adobe for media measurement is also incredibly flawed, for reasons not just cross-device, and isn't what the tools were intended for originally - surprising Rakuten did this "study".
"Marketing engineers" ??? Here in Canada (and most US states) any person calling themselves an engineer who is not a registered professional engineer will be fined thousands of dollars and spend time in jail. Its like calling yourself a doctor when you aren't. Except much more serious.
Just a side-note.