The study, dubbed a "Survey of Industry Concerns," is based on the responses of 507 people who attended the forum in January, and gives poor grades to the methods that are the basis of advertising planning and buying for TV, radio, print media, and the Internet.
"For each of the media studied, the user comments were similar. Respondents clearly believe that while the world of media is changing, media measurement is not keeping pace," concluded a summary released last week by the ARF.
The key inadequacies ranged by medium, but three were consistent across all media surveyed:
1 - The inability of current audience research to meet the changing needs of media planners.
2 - The quality of audience measurement samples.
- The size and "representativeness" of audience samples.
TV audience ratings, the subject of a number of controversies including a Congressional investigation, possible legislation and legal suits, is especially poor, according to survey respondents.
The ARF, which has been loath to criticize TV ratings supplier Nielsen throughout the debate, said that when it comes to TV audience measurement issues, the "syndicated service moves too slowly to address urgent needs." Respondents cited "slow reaction time" to rapid changes, especially digital video recorders and video-on-demand, which Nielsen is only now beginning to address.
Nielsen was also criticized for failing to deploy an out-of-home measurement system, as well as a TV commercial ratings system.
"There is no true measurement on how our commercials perform. We want to measure a true ROI and to date we can only give program performance data as an input to our models, not commercial-specific data," said one respondent. "Measurement beyond the simple concept of exposure must be developed--we have to understand the impact of an ad on inclination to purchase--or profile of the brand," said another.
Radio's current research standard, paper diaries, were described as "relics of the past," and respondents encouraged the industry to move forward on the deployment of the portable people meter system being developed by Arbitron, and to come up with more refined means of measuring multicultural audiences.
Print media audience measurement was criticized for its lack of scale--especially for consumer magazines, in which too few titles are actually measured by research suppliers, as well as for the lack of electronic measurement methods. The long lag times between audience measurement and reporting was also deemed a key issue.
Online, the newest of the major media examined, was found to lack two "basic requisites" for suitable audience measurement:
1 - Demographically defined audiences.
2 - The ability to reliably accumulate those over standard time periods. "The lack of adequate integration of the audience metrics provided by third-party ad servers and publishers/sites with the Web-user panels led respondents to judge the metrics as only 'vehicle-centric rather than user-session related,'" concluded the report.
Most significantly, the ARF said online audience measurement currently is incompatible with that of other major media, making it difficult if not impossible for online to be compared on a like-for-like basis.
Multimedia audience comparisons, in general, were cited as another crucial area for development.
The ARF plans to continue the discussion Sept. 29 with a forum on its so-called "Audience Measurement Initiative" during Advertising Week in New York.