• by Ray Schultz , Columnist, November 24, 2023

Publishers who want to jump headlong into AI should remember this: it doesn’t necessarily work when creating content. 

Human beings beat artificial intelligence in a recent email creative test. But it was a narrow win.  

Human writers won by a bare margin when it comes to emotional intelligence. But AI was more efficient.  

“With only five simple prompts we were able to trick a generative AI model to develop highly convincing phishing emails in just five minutes — the same time it takes me to brew a cup of coffee,” writes Stephanie Carruthers, chief people hacker for IBM X-Force Red, a group of autonomous hackers within IBM Security.  

Carruthers adds, “It generally takes my team about 16 hours to build a phishing email, and that’s without factoring in the infrastructure set-up.”

The initial task was to determine what might prompt an employee to click on a link in a phishing email.  

Carruthers writes, “We asked ChatGPT to detail the primary areas of concern for employees within those industries. After prioritizing the industry and employee concerns as the primary focus, we prompted ChatGPT to make strategic selections on the use of both social engineering and marketing techniques within the email.” 

On the one side was the ChatGPT-created email. In the opposite corner Carruthers’ team had “seasoned X-Force Red social engineers.”  

The results: the AI Phishing click rate was 11%, the human rate 14% — not bad at that. 

What did human writers bring to the party? 

Emotional Intelligence — as Carruthers writes, humans “understand emotions in ways that AI can only dream of."

Personalization — people provided “a reference to a legitimate organization, delivering tangible advantages to their workforce." 

Short and succinct subject line — from the people, “Employee Wellness Survey,” versus AI's “Unlock your Future: Limited Advancements at Company X”).

Carruthers concludes, “Humans may have narrowly won this match, but AI is constantly improving. As technology advances, we can only expect AI to become more sophisticated and potentially even outperform humans one day.”