Having been around since forever, A/B testing may not be fading away, but thanks to advances in artificial intelligence, it may be moving more to the back of the line.
At the annual eTail East conference and exhibition in Boston this week, the exhibition hall was filled with companies promoting countless versions of data analytics, customer targeting and, most prominently, various promises of AI-driven results.
One of the standouts, with some solid AI conversion case studies, was San Francisco based Sentient, a 10-year-old AI company that uses a branch of artificial intelligence called evolutionary algorithms.
I sat down with Jeremy Miller, director of marketing at Sentient, to kick around the current status and potential future of the old world of A/B testing in relation to the new world of AI.
“In traditional A/B testing formats, you have your control vs. an experiment,” said Miller. “You run that experiment against your traffic and whichever design performs better is the one you deploy, which is pretty much the tried and true practice.
“But people have found that six out of seven experiments don’t result in a positive outcome, so you actually have to put a lot of energy and resources to try to determine how you can actually increase conversions using A/B testing.”
Sentient’s conversion-focused product called Ascend lets marketers test all their ideas simultaneously instead of in a linear, sequential way.
“With evolutionary algorithms, it mimics the process of evolution. It takes all of those ideas and does continuous optimization,” Miller said.
“The marketers can give the AI all of their ideas, and then based on the traffic that comes to their site, it starts to understand which ideas increase conversions better. The ones that increase conversions better get to live on and the ones that do not convert better get pruned away and do not get to live on.”
Conversion could be any metric determined by the brand or marketer, such as getting a lead, making a sale or a meeting a revenue goal. It’s basically an end goal determined that’s handed off to the AI engine, which then determines how to meet that goal.
In one case for Cosabella, the global luxury lingerie brand, the Sentient AI engine tested 160 page designs and the AI evolved over seven weeks ending in a 38% improvement in conversions.
“In two or three years, everyone is going to take advantage of AI,” Miller said.
Despite results like these, A/B testing is not likely going away any time soon.
“A/B testing is confirmatory,” Miller said. “If you’re looking to confirm a hypothesis, an A/B test can still provide some value.
“You may have a strong hypothesis and you want to see if it is correct, then A/B is a method of doing that. It’s not exploration. It’s already predetermined what you think is correct and you’re just validating or invalidating.
“When you’re exploring, you’re really navigating toward the highest possible conversion. You’re exploring all the way up the mountain to get to the highest peak vs. just this local peak, which an A/B test really gets you.
“You might get lucky with A/B testing but you didn’t allow for the possibility that something else could have allowed it to be better.”
Sentient’s AI just eliminates the issue of hoping to be lucky.