• by Gavin O'Malley , June 26, 2015

Despite deep assurances from vendors, sentiment analysis remains a highly imperfect practice.

That’s according to new research from Forrester, which explores the popular use of natural language processing, text analysis and computational linguistics to interpret the opinion and tone of consumer comments, posts and other social media activity.

“Most vendors boast high sentiment accuracy, but they’re full of BS,” Allison Smith, an analyst at Forrester, writes in the new report.

The problem with sentiment models is that people so often disagree on the meaning of language. “Ask a few people to decide if the statement ‘That concert was sick’ is positive or negative, and you’ll see that slang and jargon confuse the issue,” Smith notes.

The solution? Smith suggests looking for listening technologies with pre-built integrations that compliment what your colleagues already use. “When you’re ready, you can, for example, emulate Discover and marry qualitative social feedback with replayed Web sessions to understand how to fix consumer pain points with the Web experience,” she explains.

Also, when customer references target the same demographic, or have regulatory concerns like yours, or have a brand name with multiple meanings, Smith says it’s a good idea to get their perspective on the tool.

Working with a vendor that already knows how to filter the right mentions -- or where to find the most relevant conversations -- prevents you from being anyone’s guinea pig.

Separately, vendors too often tout metrics that amount to counting consumer interactions on social, but which have little value to individual clients.

“The truth is that most organizations don’t know what to measure,” according to Smith.

Rather than measuring the success of social as a channel, Smith recommends using social tactics like listening to support the rest of the marketing “RaDaR” -- or reach, depth, and relationship marketing.

Despite the current shortcomings of vendors, Smith is bullish about the future of social listening. In fact, when interpreted correctly, she believes that social data can answer just about any question that a company has about a customer.