• by Erynne , Edward T. Rincon , February 25, 2005

Uncertainty with media ratings is not a pleasant state of mind. It causes advertisers to question their media expenditures and, if serious enough, limit these expenditures. This feeling of uncertainty has been fueled by inconsistent ratings in the television industry as well as recent revelations of inflated circulation figures by major newspapers.

It is no wonder that media and advertising organizations are calling for increased accountability of the organizations that are entrusted to produce these ratings for print and electronic media. The expectation that the advertising and media industry would "regulate itself" without federal oversight is becoming less tenable with a recent announcement by Senator Conrad Burns that such oversight may indeed be necessary.

In response to this industry anxiety, Nielsen Media Research has been the most visible among the media ratings companies to address concerns about inconsistent ratings and the absence of independent audits. For example, Nielsen Media Research recently announced that they would provide $2.5 million for "independent" research to evaluate its television ratings, although Nielsen researchers will remain part of this "independent" team.

Nielsen Media Research also funded a national advertising campaign to improve its image among ethnic consumers in the United States -- considered a needed activity to reduce the higher fault rates among African-American and Hispanic households in their television panels.

Nielsen also engaged two key Hispanic organizations to defuse its critics and obtain a stamp of approval for its Hispanic methodology: Tomás Rivera Policy Institute and the Willie Velasquez Research Institute. Finally, endorsements by the NAACP and the Rev. Jesse Jackson for its local people meter served as another stamp of approval for its methodology with African-American consumers. While Nielsen Media Research appears to be the target of a disproportionate share of the increased demands for accountability, other media ratings services are waiting with intense anticipation.

Undoubtedly, the explosive growth of Hispanics in major urban markets has forced media ratings companies to rethink their traditional methodologies. Why? It's because Hispanic audiences are very difficult to measure. For example, stable measurements over time become more challenging when 60 to 70 percent of Hispanics are renters who tend to be highly transient. Efforts to quantify Hispanic households by language usage are highly unreliable since individual household members vary greatly in their usage of language.

When two-thirds of Hispanics have not graduated from high school, it makes the reading and recording of diaries a dubious exercise in either language. And a general distrust of institutions, especially among recent immigrants, complicates the recruitment and retention of Hispanics in research panels.

For these reasons, media ratings services have a great potential to overestimate or underestimate Hispanic audiences, especially in the absence of independent audits by experts in Hispanic measurement. Standard guidelines for the measurement of Hispanic audiences are greatly overdue.

The lack of industry attention on measurement guidelines is not surprising given the tremendous amount of news hype associated with Hispanic population growth, its spending power, and the virtues of the Spanish-language media. All hype aside, however, it is important to recognize the potential sources of measurement bias that are inherent in all studies of Hispanic audiences:

Sampling Bias: Studies frequently select households located in ZIP codes with high concentrations of Hispanics, a practice that leads to higher representation of foreign-born immigrants who are primarily Spanish-language dependent, blue collar, and less educated.

Instrumentation Bias: Although seemingly obvious, studies do not consistently provide measurement instruments in English and Spanish-language options, sometimes leaving interviewers to translate "on-the-fly." Significant bias is also introduced when under-educated Hispanics are required to record their media behavior on paper diaries, a practice that also tends to favor the recall of Spanish-language programs. Local people meters, while offering more precision, require more active participation by all members of a household. Hispanic households, who tend to have more members per household and younger ones as well, are more likely than non-Hispanics to experience fault rates.

Interviewing Bias: Bias is further introduced by the use of monolingual interviewers rather than those that are bilingual. English-dominant interviewers tend to interview English-dominant respondents more often, while Spanish-dominant interviewers are more likely to interview Spanish-dominant respondents. The preponderance of women as interviewers, coupled with the higher cooperation rate of female respondents, also tends to result in more completed interviews with Hispanic females. It is not uncommon to find Hispanic-targeted studies that are over-represented with foreign-born, Spanish-dominant female respondents because the interviewing team had few bilinguals and enjoyed completing interviews with the more cooperative female respondents.

Analytic Bias: A perfectly executed study can still fall prey to the analyst who transforms the raw survey data into ratings using improper weighting procedures. Post-stratification weighting is recommended when the data collection process leads to imbalances in the expected sample distribution. For example, if the proportion of foreign-born Hispanics in the study sample is 80 percent but the Census Bureau shows that it should be 60 percent, then a post-stratification weight is necessary to ensure that the ratings do not over-represent the foreign-born or under-represent native-born Hispanics. An imbalance by gender would require a similar adjustment. The failure to use weights when needed, or the deliberate use of incorrect weights, can lead to significant errors in ratings produced for any medium.

To improve industry confidence in the measurement of media behavior, especially those involving African-American and Hispanic audiences, bold steps need to be taken by the media ratings companies as well as organizations that use these ratings. Media ratings services will need to become more transparent in disclosing the details of their ratings methodology so that independent experts can evaluate their soundness.

Media ratings services should also resist efforts to seek endorsements from ethnic organizations, launch advertising campaigns, or prematurely disparage its critics - tactics that unnecessarily delay the discovery of viable methodological solutions. Organizations that buy or use ratings services, on the other hand, should demand more accountability from the ratings industry. Some important first steps might include the following:

· Develop standards of measurement for Hispanic or multicultural audiences that apply to all print and electronic media.

· Develop an audit team of measurement professionals who are experts in audience measurement techniques for multicultural audiences, and who work regularly with the analysis of related data.

· Encourage all major print and electronic media ratings services to obtain accreditation from the audit team of multicultural measurement experts.

· Communicate the complete results of these audits to the public.

Who knows -- the media ratings companies just might become more profitable when the cloak of secrecy surrounding their measurement methodology is lifted. Multicultural audiences might be measured more accurately. Federal oversight might take a back seat. And advertisers might be willing to spend more money. It could happen.