Does Sentiment Matter?

The Paine of Measurement 

Does sentiment matter? I raised this question to a room full of sentiment analysis providers at Seth Grime’s Sentiment Analysis Symposium (SAS11) in New York earlier this month. Needless to say I was not the most popular person in the room.

My point was that there are many many caveats to using sentiment analysis and the biggest one is that, unless you have proof that it makes a difference to your business, it’s all a tremendous waste of time.

The SAS11 conference was full of examples of great technology. Many of the problems that I raised concerns about earlier have been, if not solved, at least addressed with more sophisticated NLP and text mining software. But from my perspective, the bottom line is still the bottom line.

Unless you can tie an increase or decrease in sentiment to some customer action, does it really matter?

Sure if Apple or Southwest or Dell see a spike in positive tweets there may well be a correlation with online sales. But if they spot a big uptick in negative Tweets does that translate into fewer sales? And what about the millions of businesses that can’t make a direct connection between customer sentiment and customer action because they aren’t doing business online, they sell through distributors, or they don’t have enough volume of conversation to have a statistically valid sample?

My point is not that sentiment analysis should go away. Clearly it isn’t going to. But the significance of your results needs to be calibrated to the impact those results have on your business. And we should also consider that the notion of just defining and tracking positive vs. negative really is not sufficient in a world when people express their feelings about a brand or a company in so many other ways. Never mind the entire new version of the English language that Twitter has introduced.

Automated positive vs. negative sentiment tracking ignores what I would argue is the most important concept you can glean from social media – the relationship that your customers have with your brand. Nowhere in any of these systems did I see the concepts of trust, loyalty, or commitment addressed.

We at KDPaine & Partners are currently using human coding to tease these concepts out of social media conversations. We envision a time when we can pull at least some of these concepts out automatically, but it’s not exactly here yet.

Happy Measuring,

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements.

Comments

[Rpaulsingh]

You make some interesting points and we agree that sentiments have to relate to business. However, saying you derive it manually and so it is better is probably a fallacy too. What is needed is a hybrid system wherein you start with a humanly curated system, create an automated system and then continue to enhance it with human curation. So one has to look at both share of voice and sentiments and we have created some better matrix like net perception score and share of voice and socialnuggets index to measure them.

So for example we noticed that HTC Thunderbolt smartphone was rising is both sentiments and share of voice compared to Apple iPhone. That doesn't mean Apple's iPhone sales were declining but it certainly indicated some issues which did show that their sales were not growing as fast while HTC did score a huge sales increase in last month. Also when you look deeper we found out that some of its features like lack of 4G and others were hurting Apple iPhone 4. Is this useful? Could you really derive this manually when there are over a 1M conversations every month. Perhaps not? You can't create interpretation automatically and thank god we as humans are still needed.

[Katie Delahaye Paine]

Thanks for you comment, but I have to suggest that even before you even get to human interpretation of the data, you need to make sure that what you are saying is "positive" actually is. I also think that you have to look not just at the number of positives but at the ratio between the negatives and the positives. If I were the marketing manager for the HTC or the iPhone, before I made any decisions, I'd insist on human coding of a random sample of at least 20% to ensure that the data you were provding was at least 85% accurate. THEN I would do my interpretation.