• by James Byerly , Op-Ed Contributor, March 13, 2017

Many brands are intrigued by the promise of dynamic creative optimization (DCO), since they can run singular campaigns that serve several permutations of an ad to different audiences, and then watch as the campaign optimizes itself. But DCO’s allure brings about questions from marketers who look to integrate the technology into their toolkit for the first time.

A common question is, “How do I know when I’ve run enough impressions to have a statistically significant result, before I adjust the rotations?” Brands are curious about the optimal number of dynamic creative variations needed before they can identify which performs best.

Most marketers understand there isn’t a single number that works for every campaign. Determining the optimal number actually just takes some easy math or data analysis. It’s a worthwhile exercise for any brand interested in DCO, and helps define strategies and tactics.

Before getting started, we should highlight a few important factors:

The number of dynamic variables in the creative. This includes different calls-to-action, images, products displayed in the ad, and other elements.
The number of variations to be served. To find this, we need to know the values used for each dynamic variable.
The given metric for success. If we want to determine success, we need to know how the campaign is viewed in the end, and which metrics, such as interactions, clicks, or conversions, mean the most to the brand.

From here, marketers can solve for the number of impressions needed to identify a winning ad permutation.

Let’s take a hypothetical ad and assume there’s a total of six dynamic variables in the creative: CTA (call to action), product name, product image, product description, product price-point, and background image. Assume that product name, image, description, and price point will be matched together and won’t vary. We can actually bundle the product elements together, calling it “product served.” That leaves us with three true dynamic variables: CTA, background image, and product served.

Next, we need to determine how many values exist for each dynamic variable. Assume there are five CTA variations, 10 product variations and three background image variations -- we can determine that 150 different versions (5 x 10 x 3) can be served. This is the optimal number of dynamic creative variations to understand performance.

Finally, to find the minimum number of impressions needed to achieve a statistically significant result, we need to identify metrics that determine success. Consumer ad interactions are easy to achieve, followed by clicks and conversions.

It’s important to understand that the more difficult the result (lower expected rate) the higher the number of required impressions. For example, if our hypothetical campaign is measured on interactions, they’d likely run 10,000 impressions per ad variation. With clicks, we may want to raise that to 100,000 impressions per variation. For conversions, it may easily require 1 million impressions per variation.

If we want to find the best ad version that drive clicks, we take the 150 different ad versions and multiply that by 100,000 impressions per version, arriving at a minimum of 15 million impressions.

It’s worth noting that some ad servers might have specific data analysis protocols in place for determining the threshold. In many cases, a certain metric rate must be achieved before adjusting the campaign. Analyzing data in the background is beneficial, and provides marketers with an understanding of the needed impression volumes for a fair version test.

This mathematics lesson is intended to serve as a lens through which marketers can form expectations for DCO. Many marketers are eager to adjust campaigns too early, so it’s important to remember certain thresholds must be met before making changes. Marketers who see the big picture and practice patience with DCO will likely reap the rewards of optimization strategies.