Commentary
Perverting Your Metrics
- by Pat LaPointe , Op-Ed Contributor, September 29, 2009
To: John Bulleader
From: Barbara Researcher
Subject: How to improve our Net Promoter scores
In response to your question of last week, I have considered several options for how we can improve our recently flagging Net Promoter scores and thereby increase that portion of our year-end bonus linked to that specific metric.
1. We could restrict our sample of surveyed respondents to only those who have recently purchased from us, and ignore those who either didn't like us enough to buy from us -- as well as those who bought from us a while ago, but may be having second thoughts due to our poor reliability and service.
2. We could change our sampling approach to only solicit surveys from those who buy online, since our Web site is so slick and efficient. This has the added benefit of reducing our research expenses so we can still afford those front-row football tickets.
advertisement
advertisement
3. We could change the way we calculate Net Promoter to take the percentage of customers who score us as 7s through 10s, and subtract those who score us as 1s or 2s since we know that 3s to 6' are really the "marginal" middle group -- and thereby take some credit for producing partial satisfaction.
4. We can offer customers a $10 bonus coupon to allow our sales associates to "help" them complete the survey before they leave the store, thus providing both convenience and value to our customers.
5. We can reduce the frequency of surveying from monthly to annually, to make it virtually impossible to link our marketing or sales actions back to increases or decreases in the scores. This will create some much confusion over interpretation and causality that bonuses will have long been paid by the time anyone actually agrees on what to do next.
6. We can have our sales reps do the surveying themselves. This will allow us to capture notations about body language of the respondents, too (side benefit: see football reference above).
Any or all of these strategies could essentially ensure success. Provided our market share doesn't fall too fast, we're unlikely to draw any undue attention.
Please let me know how you would like to proceed. We can also adjust any of our other metrics in similar ways.
I comment often on inherent biases in studies and research and even reporting. How you craft a survey's questions, whom fills out the survey, ,whom reads the survey and whom commissioned the survey all add bias.
I saw the worst of it during the Presidential Campaign since I received surveys from both the RNC and DNC. Often there was limited choices to questions and I was forced to use a sharpie to add my choice.
Great examples of bias: When George Bush decided that global warming doesn't exist in his early years, the research he used was paid for by Government Money and done by the American Petroleum Institute (vs say the EPA). Remember Tobacco didn't used to cause cancer.
Just whom do you think you are to question George Bush?
Howiem you are killing people's ears even though your comments are welcomed reading. It is..who fills out the survey. Who is the subjective case. Whom is the objective case and always takes a preposition of to, by, for, from, of, etc. Just in case anyone is reading this, the words I of the subjective case and me of the objective case works the same way.
Mike, George Bush and $10 coupons just don't add up ! :)
"Whom," said the owl, thereby showing he was educated.
For the life of me I have been unable to figure out why anyone believes the Net Promoter score provides useful information a company can use going to market.
I've spent the last 10 years telling people all the reason surveys suck - and that the only reliable way to build loyalty and revenue is to base predictions on actual customer behavior. It's easy. It's fun. And it's affordable.
So I applaud the post, Pat - Nail it to a door someplace.
To whom, Jonathan. Again, who is subjection as Who is at the door?, not whom is at the door? Who told you to give that envelope to me? Subject-verb-object. Jr. high grammar.
Thanks for the tips, I learned a couple I hadn't thought of! You may want to consider this: if you are really worried that your product or service stinks, try inverting the scale (1 = Excellent, 10 = Rotten) most respondents are so ingrained the other way, you'll generate good scores and can improve your football seats location.