• by Erik Sass  @eriksass1, April 26, 2006

Magazines must take steps to protect their content from scraping and aggregation by online search engines, even as they harness the power of search for their own purposes. That's the main conclusion of The Boston Consulting Group, which today is expected to release a study about search engines at a conference hosted by the Magazine Publishers of America.

"Online search presents real opportunities and real potential threats for the magazine industry," states the study, "A Perspective on Online Search for the Magazine Industry."

"Search providers, (in particular the large three search providers: Google, Yahoo and MSN) are beginning to co-opt the consumer relationship online," continues the report, which laments that the major search engines are becoming "gatekeepers" to other sites, while also attempting to keep visitors on their own properties as long as possible.

Other threats found by the Boston Consulting Group include the prospect that search engines open magazines "to a wide variety of new competition," including blogs and other user-generated content, and that search engines disintermediate traditional media companies by selling ads around other companies' content.

The report calls upon the magazine industry to "discuss and debate alternative approaches to search that allow each company to act independently while preserving strategic flexibility across the entire landscape."

The group also proposes that one solution might be for magazines to work on developing a "hybrid" model that leaves their content "searchable but not scrapable."

The consulting group issued its recommendations in response to the hard lessons learned by newspapers as they moved online, when a few newspapers "allowed their content to be scraped and aggregated by search engines." "The individual actions of these papers affected the strategic flexibility of the rest of the industry," states the report, which urged the magazine industry to avoid a similar fate.

The chief danger of opening content to scraping and aggregation is the value that would be lost as fewer readers visit magazines' proprietary Web sites or pick up print copies, the report added. But at the same time, the industry must strike a careful balance--as search is also a valuable means of drawing readers to proprietary Web sites; on this subject the consultancy also warns against the other extreme--creating "walled-in" consortia of online content that are totally closed to search--noting that similar efforts by the newspaper industry, like the New Century Network, have already failed.

Syndicating content and opening it to aggregation might seem like a reasonable compromise, but this approach also has its pitfalls, the consultancy warns. For one thing, "an aggregator unbundles content from brands, displaying only headlines and a few lines of text from a sea of providers." As a result, "content publishers, regardless of the depth of coverage or analysis, lose control of how content is delivered to and consumed by the reader"--and with it, much of the value proposition for their advertisers.