Listening To The Engine
When dynamic ad technology company Teracent started helping HP retarget users off site, a funny thing happened. The engine behind the dynamic ad creation system started telling the marketers that some of the rules it made for the system were holding back performance. In a branding campaign for a retailer, the Teracent engine used real-time multivariate techniques to form and custom-serve on-the-fly audience segments that performed at least as well as pre-existing BT segments. According to Senior Vice President of Sales and Marketing Chip Hall, sometime a real-time engine can read the most relevant behavior better than historical behavior. Learn to listen to the engine, he argues to us today.
Behavioral Insider: How is this technology different from other dynamic ad creation systems?
Hall: We create relevancy by bringing together all available advertiser-centric information: insights from a Web site, geo or demo data that a publisher might provide, third-party data, or even the creative. Then we optimize the correct marketing message or product mix against it. In real time, we're performing data-driven ad optimization. A self-learning engine powers it. We are seeing a 300% to 700% click-through rate improvement over traditional static or simple rich media, and between a 200% and 400% conversion rate increase.
BI: Is much of this retargeting, then?
Hall: The low-hanging fruit is the purchase intent data that you get from their own Web site. From a behavioral targeting standpoint, when we use the customers' Web site data we can do real-time BT segmentation. We also have the ability to layer in publisher data. So when we run campaigns with Yahoo for example, and that BT data is sent to us during the ad call we're able to leverage that for further targeting. The data usage is prescribed for the advertiser in that publisher channel. We don't comingle any data. We don't claim any data ownership.
BI: How did the creative variables work in HP's case?
Hall: They want to understand on a weekly basis what are the trends going on against certain geos and demos. In real time we are creating these little mini BT segments. So we go back out onto the Internet and see where we can find these people. With Yahoo, we will layer their information on top of the retargeting data to create an even finer grained sense not only of what products they want but really who they are on a geo, demo and BT basis. Then in real time we make every impression better than the last. We see what the cookies are looking [like]. We use that information to inform all optimization models. We have a closer sense on the next cookie that comes through what success looks like. We create our own optimization model but starting with the BT inputs to work with.
BI: In this case, how many creative variations might you serve dynamically?
Hall: There is no set number. The creative is driven from the template approach. Much like an old postal form letter, you start with a template form that has some holes in it and the holes are filled in real time by the appropriate data pieces. These can be creative pieces like background color or calls to action, tag lines, as well as product=specific information. All of the logic and the data is held on the server. The template is just filled in depending on the information we get about the audience at run time. The ad is reacting in real time to the audience as the audience hits the page.
BI: Are you just retargeting the user with the product they just visited at HP or is it more predictive than that?
Hall: We create micro models as each wave of impressions come in. We do predictive modeling in that. It is very rapid A/B multivariate testing of each successive wave of impressions. The purchase intent data is an input that helps inform the optimization, but it is not a one-to-one match. It is more complex than that. We know this person looked at this specific printer. But now it is 12 noon, and we're being told by Yahoo data that this is a woman of this age. Since we've got some knowledge about what at noon women of this age in Boston should be buying, we will layer that in. So it might not show the exact printer she looked at before but something in the printer class that would make the most sense given what the model says will be the most successful.
BI: What is this teaching you about creative that does and doesn't work?
Hall: There are no sweeping generalizations. We report very specific creative and product feedback on an audience basis. For HP we had third-party data to targeting against people we knew were looking at non-HP products. For some reason, when we showed a certain printer with a certain background, it performed worse than a static ad. It was weird. Intuitively, the human in us said it didn't make sense. But over a statistically relevant number of impressions the engine said to stop showing this combination.
In our system you can do rules or have the engine do all the learning all itself. We had placed a rule against it and the rule was just wrong. This happened with a bunch of clients. It proves that historical knowledge is inherently flawed because it is about what happened yesterday, not what is happening right now.
The reality is that the customer is always right. You just have to be smart enough to listen and react to it. Lo and behold, the engine learned not to show that blue background on this printer anymore and it chose a different combination that did start to work. It was a combination of product and coloring that we thought didn't' make any sense, but clearly it did because it was delivering the conversions we were looking for.
BI: Can the technology work apart from retargeting, with branding?
Hall: The question is what data are you using to inform the ad? Retargeting data is very powerful, but it aims at the bottom of the funnel. With awareness or brand campaigns there sometimes is very little data to work with: basic geo and all the creative elements. At the top of the funnel, we can take very minimal data, but a billion impressions will start creating custom BT segments for an advertiser. People in this part of the country will click this and these people will click this. The optimization engine is learning and growing. People still will use pre-existing customer segmentations and BT segmentation because it is what they own, know and trust. But as you use the engine you start figuring out how much self-learning can I use vs. pre-existing rules and segmentations. There is a mix. You are not dependent on any one piece of data.
BI: Aren't the creative variables more constrained in a brand awareness case?
Hall: In this case, it was top of the funnel awareness but for a digital circular campaign. Our data set was multivariate. The seed we were using was that week's circular data. And then through a publisher we had a set of BT data. We started with this stew of things. So without any retargeting data as to what these people wanted.
Through creative optimization we were able to blow away the standard they were using precisely because we would model in real time what success looked like. They had been using some pre-existing BT from one publisher. They started stripping away some of the BT segmentation and rules because they wanted to experiment and let the engine say what works. The performance increased.
By being so heavy in their preexisting BT, they were actually limiting the engine's ability to merchandise. By letting the engine have more free reign in certain pockets of the inventory, we were showing products that didn't make sense to human [marketers], but the audience was clearly reacting well to it. A self-learning engine driven by algorithms can manage that, and you will learn a whole lot.