Having been around since forever, A/B testing may not be fading away, but thanks to advances in artificial intelligence, it may be moving more to the back of the line.
At the annual eTail
East conference and exhibition in Boston this week, the exhibition hall was filled with companies promoting countless versions of data analytics, customer targeting and, most prominently, various
promises of AI-driven results.
One of the standouts, with some solid AI conversion case studies, was San Francisco based Sentient, a 10-year-old AI company that uses a branch of artificial
intelligence called evolutionary algorithms.
I sat down with Jeremy Miller, director of marketing at Sentient, to kick around the current status and potential future of the old world of A/B
testing in relation to the new world of AI.
“In traditional A/B testing formats, you have your control vs. an experiment,” said Miller. “You run that experiment against your
traffic and whichever design performs better is the one you deploy, which is pretty much the tried and true practice.
“But people have found that six out of seven experiments don’t
result in a positive outcome, so you actually have to put a lot of energy and resources to try to determine how you can actually increase conversions using A/B testing.”
Sentient’s
conversion-focused product called Ascend lets marketers test all their ideas simultaneously instead of in a linear, sequential way.
“With evolutionary algorithms, it mimics the process
of evolution. It takes all of those ideas and does continuous optimization,” Miller said.
“The marketers can give the AI all of their ideas, and then based on the traffic that
comes to their site, it starts to understand which ideas increase conversions better. The ones that increase conversions better get to live on and the ones that do not convert better get pruned away
and do not get to live on.”
Conversion could be any metric determined by the brand or marketer, such as getting a lead, making a sale or a meeting a revenue goal. It’s basically an
end goal determined that’s handed off to the AI engine, which then determines how to meet that goal.
In one case for Cosabella, the global luxury lingerie brand, the Sentient AI engine
tested 160 page designs and the AI evolved over seven weeks ending in a 38% improvement in conversions.
“In two or three years, everyone is going to take advantage of AI,” Miller
said.
Despite results like these, A/B testing is not likely going away any time soon.
“A/B testing is confirmatory,” Miller said. “If you’re looking to confirm
a hypothesis, an A/B test can still provide some value.
“You may have a strong hypothesis and you want to see if it is correct, then A/B is a method of doing that. It’s not
exploration. It’s already predetermined what you think is correct and you’re just validating or invalidating.
“When you’re exploring, you’re really navigating
toward the highest possible conversion. You’re exploring all the way up the mountain to get to the highest peak vs. just this local peak, which an A/B test really gets you.
“You
might get lucky with A/B testing but you didn’t allow for the possibility that something else could have allowed it to be better.”
Sentient’s AI just eliminates the issue of
hoping to be lucky.