Many publishers wonder if AI will help — or hurt — their credibility. The BBC decided to find out for itself. 

It conducted research into “four prominent, publicly available AI assistants – OpenAI's ChatGPT; Microsoft's Copilot; Google's Gemini; and Perplexity,” the BBC reports. “We wanted to know whether they provided accurate responses to questions about the news; and if their answers faithfully represented BBC news stories used as sources.” 

The BBC concluded that “AI will bring real value when it’s used responsibly. But AI also brings significant challenges for audiences, and the UK's information eco-system.”

Specifically, the BBC found that:

• 51% of all AI answers to questions about the news had significant issues in some form 
• 19% of AI answers that cited BBC content introduced factual errors, whether statements, numbers or dates
• 13% of the quotes sourced from BBC articles were altered from the original source or were not there at all

Now, the latter two are not massive percentages. But they are bad enough. And the mistakes are serious ones. 

For instance, Google’s Gemini stated, “The NHS advises people not to start vaping, and recommends that smokers who want to quit should use other methods." On the contrary, “the NHS does recommend vaping as a method to quit smoking,” the BBC reports. 

Then there was Microsoft's Copilot. It stated that “Gisele Pelicot uncovered the crimes against her when she began having blackouts and memory loss.”

Wrong. Pilicot “found out about the crimes when the police showed her videos they had found when they confiscated her husband’s electronic devices,” BBC writes. 

Meanwhile, Perplexity mis-reported the date of Michael Mosley’s death. And ChatGPT stated last December that Ismail Haniyeh was part of Hamas leadership. Actually, Haniyeh had been assassinated in Iran that July. 

The takeaway from all this? 

“Ensuring people can find trusted information in the age of AI will require AI and media sectors to work together, and the BBC is ready and willing to work closely with others,” it says, adding, “We are also planning a series of AI literacy activities to help audiences navigate their use of AI.”