...when Coca-Cola put buzz sentiment data into the same analytical framework it uses to evaluate other digital media, Mr. Schmidt [senior manager-marketing strategy and insights at Coca-Cola] said, "We didn't see any statistically significant relationship between our buzz and our short-term sales."
Note that this study only concerned “buzz,” which the company defines as conversations taking place on social networks and measures by “counting the raw publicly available comments” on Facebook, YouTube, etc. See the article in AdAge.
Reading between the lines, it's clear that Coke's social media measurement techniques need refinement. Isn't the raw number of comments an awfully crude measure? The Coke study covers only "buzz," not sharing, video views, or other aspects of social media. And when it comes to buzz:
One problem Coca-Cola has is determining whether buzz is actually positive or negative in the first place. In one 2010 study where Coke pulled out more than 1,000 social-media messages randomly and had human raters compare them to automated sentiment analysis by one vendor, there were widespread differences.
"When we say it's positive, the machine about 21% of the time says it's negative," [Schmidt] said. "That can cause some problems in our understanding" of how buzz impacts sales.
Most people claim social media does not influence what they buy, anyway.
And, the most interesting wrinkle is that, according to PRNewser:
...the Yahoo! Finance page features a poll asking the question: “...Does social media (Facebook, Twitter etc.) influence what you buy?” The options are “yes, quite often”, “from time to time” and “never”, which currently leads with a decisive 85% of the vote after more than 50,000 responses.
So, expect to see some social media measurement scrambling in the near future. --Bill Paarlberg, Editor, The Measurement Standard (Thanks to Frank Walton for the tip.)