What's wrong with this picture

Applying various old-world metrics to social media is nothing new. Ever since the first blog was launched, people have been trying to find automatic measurement systems that will deliver the holy grail of social media measurement. If you were part of last night's #BlogChat  you'll know how impossible that is. There are too many different perspectives, needs, markets and audience to wrap it up into nice little package tied with a tape measure tied ino a bow.
The latest is an attempt by Razorfish & Ogilvy to do a Net Promoter-like Score for Social Media.
In general I applaud the concept. There's certainly nothing wrong with looking at the number of positive comments vs the number of negative comments and coming up with a single number. In theory. The devil, as they say is in the details. So if someone proposes this to you, you might want to ask them to define a few of those details such as:

1.  How are they defining positive & negative? In my experience, not every organization sees it the same way. To some, a positive is staying OUT of a story. Our standard definition is positive = leaves you more likely to do business, negative = less likely, balanced = a little of both, neutral = no sentiment implied.
2. Who is deciding if a post/item is positive or negative. Is it a member of your target audience. Does he/she reflect the biases and assumptions of the target audience. If you're using computer-based analysis, think carefully if a computer can do this accurately? We find that computers get it right about 3 out of 5 times. If 40% of the data is wrong, is it still a valid number? If just 25% of the data is wrong, is that okay?
3. Is the system only looking at your brand, or is it looking at the entire marketplace? If there are 500 postings, of which your brand sees 100  positive  and 50 negative (the rest are neutral or balanced), you get a net promoter score of 50. But if the competition is also mentioned in those postings, but scores 200 positive to  50 negative, their score is three times better. A much more important number to track.
4. Are you looking at the right numbers? An even more relavent score is to look at all the positive discussion and see how much is about you. Then look at all the negative coverage and see how much is about you. In this case if there was a total of 300 positive postings and 100 negatives, you'd have a 33% share of the positives, and 50% of the negatives. That's a far more accurate and useful datapoint than just saying you got an NPS of 50.
5. What impact is it having on the business? Any single score by itself is just a data point to the C-suite. Unless you put it into the context of the business, it's irrelevant. Is the increase in your score actually resulting in more leads, more sales? In other words, you have to ask: So What? before you show it to the C-Suite.

Comments

[steve dodd]

Yes, the devil is in the details. And you certainly "do" detail well. Personally, I'm getting frustrated with all the "statistical" reports being published lately with no validation detail, just numbers that somehow get interpreted based on what the person interpreting them wants them to mean (ie: recent twitter stats etc.)