• by Cory Treffiletti , Featured Contributor, Yesterday

I use AI tools for ideation, content creation, and editing in the same ways each of you do.  At first, I was shocked at the time savings and the value they provide.  They are a boon for productivity. 

But the more I dive in and use these tools, the more I realize they suffer from a simple problem.  If I’m being formal, I say the issue is the underlying data that informs the models.  If I’m being less formal, it’s “garbage in, garbage out.” That’s something you should worry about when trying to protect your brand, both personal and corporate.

AI models are not always precise. Sometimes it will change a number it uncovers in a stat it finds online.  Not wildly different, but just enough that it can no longer be cited accurately. Sometimes the model will invent a source that sounded plausible but wasn’t accurate, conflating something it did find from a verifiable source. It will confuse two similar topics and blend them into an answer that’s technically coherent but factually incorrect -- even though, to an uninitiated person, it is logical and clear.

When someone asks an AI platform a question about your company, similar issues arise.  The model is not simply reciting your website copy back to users. It synthesizes what it sees from multiple sources, whether you gave it those parameters or not. It pulls from reviews, forums, news coverage, competitor comparisons, old press releases, and anything else it can find, and it merges all of that into a single, confident-sounding answer.

Confident is the operative word. The model does not flag uncertainty the way a good analyst would. It does not necessarily evaluate the source to determine accuracy over plausibility.  It states things plainly, whether they are true or not.

A UC San Diego study this year found that AI-generated summaries were wrong roughly 60% of the time, and that those summaries were still influencing purchase decisions. Separate research surveying marketers found more than four in 10 said hallucinated or false information had slipped past review and gone public before anyone caught it.

That is a brand’s reputation being written by a system that, by its own nature, is going to occasionally make things up and say them with total conviction.  If you compound that with the fact that most of the internet is personal opinion rather than fact, you start to understand why AI does what it does -- not that it makes it any easier to accept as a brand manager. 

This is why GEO and AEO, as important as they are for visibility, are the smaller half of the problem.  While you're focused on the idea of being mentioned, you also need to be fact-checking what the mentions are about. Being mentioned by an AI platform used to feel like a win, but it is only a win if the mention is accurate. If it’s not, you have all the reach of the citation and none of the control you used to have over your own message.

For a marketing or brand team, that changes the job, shifting to reputation management and not just engine optimization. Visibility monitoring is already table stakes for your brand. Accuracy monitoring has to join it, and may be even more important.

Someone on your team needs to own the question of what AI platforms are saying about you, not just whether they are saying anything at all. Treat it the way you would treat a wire story or personal op-ed about your company that you did not write and cannot edit. You cannot stop it from running without a legal cease and desist, and those are expensive and time-intensive. You can track it, and you can absolutely correct the record when it is wrong, even if that takes time.

You need to know what is being said, and you need to engage to fix it, or else the underlying information fueling the AI model will be inaccurate, which fulfills the “garbage in, garbage out model.  If those mistakes are perpetuated, they can have a detrimental impact on your brand.