Yet, every one of them expressed frustration that even well produced tests generate results that are difficult to read and measure. For example, Arti Parikh of Orbitz had to add in a second measure in order to break a tie. Christine Scott of General Mills found that the version that had the higher clicks (in one test 95% increase) broke even on interactions.
Another great lesson came from Carolyn Ude of ESPN.com, who found that those who received three messages actually responded less than those who received two. This was a real learning around lifecycle, and ensuring that all the messages reach subscribers when they are truly "in market," not just when we hope they are in market.