• by Ray Schultz , Columnist, October 25, 2023

Human beings beat artificial intelligence in an email creative test. But don’t start celebrating just yet.  

First, live copywriters won by a bare margin. Second, this wasn’t normal email marketing — it was phishing. Still there are lessons here for email teams of all types.

“With only five simple prompts we were able to trick a generative AI model to develop highly convincing phishing emails in just five minutes — the same time it takes me to brew a cup of coffee,” writes Stephanie Carruthers, chief people hacker for IBM X-Force Red, a group of autonomous hackers within IBM Security. 

Carruthers adds: “It generally takes my team about 16 hours to build a phishing email, and that’s without factoring in the infrastructure set-up.”

The initial task was to determine what might prompt an employee to click on a link in a phishing email. 

Carruthers writes: “We asked ChatGPT to detail the primary areas of concern for employees within those industries. After prioritizing the industry and employee concerns as the primary focus, we prompted ChatGPT to make strategic selections on the use of both social engineering and marketing techniques within the email.”

On the one side was the ChatGPT-created email. In the opposite corner Carruthers’ team had “seasoned X-Force Red social engineers.”  

Here are the results. 

The AI Phishing click rate was 11% and the human rate was 14% -- not bad.

What did human writers bring to the party? They brought:

• Emotional Intelligence — As Carruthers writes, humans “understand emotions in ways that AI can only dream of"
• Personalization — People provided “a reference to a legitimate organization, delivering tangible advantages to their workforce 
• Short and succinct subject line (“Employee Wellness Survey,” versus “Unlock your Future: Limited Advancements at Company X”)

Carruthers concludes: “Humans may have narrowly won this match, but AI is constantly improving. As technology advances, we can only expect AI to become more sophisticated and potentially even outperform humans one day.”