• by Cory Treffiletti , Featured Contributor, December 27, 2023

Most of my career in advertising has been about testing and learning.  I love some good data, and testing is the most important way to get good data.  However, most people forget a simple fact: You have to isolate variables to get a good test. Each test you run must be focused on proving or disproving a hypothesis.

There are many poorly orchestrated tests in the advertising world. Too often people set up a “test” to see one creative over another.  That is not a test. That is an optimization.  Two different types of creative is not a test. There are too many variables.  You can’t know what hypothesis you were proving by simply pitting two concepts against one another.  All you find out is which one works better, but you are not likely to know how to scale to the next round.  You need to be clear upfront about what you are looking to learn, so you can build off that for the next round. 

Testing is all about learning, and you have to build knowledge over time.  That means knowing that you learned something by isolating a variable that you now carry through to the next round.  The you isolate a new variable and test from there.  This is how you orchestrate a series of tests that build knowledge rather than simply optimize a campaign. 

That’s it. Pure and simple.  In the new year, let’s make sure that testing is done with discipline or else, call it what it is: optimization. 

Happy New Year!!

advertisement

advertisement