Commentary

Planning to Fail

One of the earliest computer bugs was a real bug. A moth. That moth is in the Smithsonian's Museum of American History still taped to a page in the 1947 log book of the team working on Harvard's Mark II Computer where the moth was found stuck between two relays inside the computer And so the term "debugging" was born. In the 66 years since, bugs have transformed from flying insects to mistakes made by humans in lines of computer code written during the creation of software. The growing complexity of the media systems we use for planning, buying and measurement means that bugs also have potentially severe consequences.

Recently, MediaPost reported a "glitch" in Nielsen's software used in television advertising campaign planning and analysis. My company sells television advertising, and Nielsen's ratings are the measurement currency of television. As a Nielsen client, we have a keen interest in the accuracy of TV ratings. We're also unique, however, because we are builders of the software we use to manage and analyze large sets of anonymous TV viewing data used to create advertising packages for our agency and marketer clients. My first response on reading the news was: "Of course there are bugs." I've been making advertising technology for the last 12 years and know firsthand that there will be glitches even with code generated by the bestsoftware architects, coders and quality assurance engineers.

When using small, sample-based measurements projected to large populations -- as is the case with existing TV currency -- a single small error in calculations caused by a software bug can lead to giant errors in projections of audience reach. If you travel 10 feet using a compass that is wrong by 1 degree, you'll end roughly where you expected and may not notice the error. However, if you fly from New York to Atlanta using a compass off by 1 degree, the error is compounded dramatically and you could end up somewhere in Tennessee. Advertising companies now have big data sets and new methods to apply to that data to spot mistakes and minimize the downside impact of unintended consequences. In the case of my company, we validate our data against Nielsen's data and use very large samples of directly measured television viewing to help minimize the multiplication of errors.

Because we assume that there will be errors in data or software bugs, we automate the slicing of our data, predict what the results should be and use multiple methods to calculate the actual results, which are then verified against each other. On the surface it may seem a bit extreme for a seller of advertising to go to this length. But experience has taught us not to trust the compass alone. We check our compass against multiple GPS systems, which we check against each other. We also bring back up batteries, a map, a flashlight and a granola bar.

As sellers and buyers of media increasingly rely on software to make decisions, they need to be able to better judge how their organizations and the makers of their business-critical media systems anticipate consequences when bugs happen. Agencies and advertisers should also think about their own organizational Plans B, C and D. The impact of bugs should be measured across the organization and mission-critical planning and buying tools should be evaluated both individually and collectively.

Buyers of media should also expect transparency from vendors about how their systems cope with failure. The bugs will keep coming. Servers will continue to fail. Instead of thinking "what would we do if this happens?" think "how will we act when it happens?"

In the case of media software, expect to see other glitches that have gone undetected simply because there has historically been little data and other systems for comparison. As new, secondary sources of information become available, we should expect to find bugs and errors discovered more often. Media companies that rely on technology should plan to fail. Because technology does fail. It's time that the media industry began to expect the unexpected.

Jack Smith is chief product officer at Simulmedia.

Next story loading loading..