by Katie Delahaye Paine
If someone decided to do a 2013 remake of Mike Nichols' film The Graduate, Mr. McGuire would be whispering to Benjamin, “Big data,” instead of “Plastics.” Google the phrase and you get 1.6 billion results. Data is, in general, a great thing, but the notion that Big Data will solve all our problems is perhaps the most over-hyped idea since Y2K.
Yes, there is lots of data out there, but all of it is worthless without insight and human interpretation.
For those of us in PR and social media measurement this is a godsend. Our job is to use data to point the way to the future, and never before have our jobs been so important or in demand. The more data we have, and the easier it is to use, the more effective we can be.
Bring On the Data Geeks
And if the data and stats are complex enough to demand truly specialized training, then bring on the data geeks. They are the superheroes of the new data landscape, able to leap megabytes of data in a single bound. The mild-mannered eggheads of yesterday are coming into their own and taking seats at the proverbial table.
I’ve done half a dozen workshops with clients lately and every one has had a data geek in attendance. The advantage of having these wizards in the room is that things like correlations and data analytics are cake to them. To a data geek, a huge pile of data is not an obstacle, it is an exciting playground of opportunites and adventures.
How to Avoid Big Bad Data
Big data is a lot like a drug. In the hands of the right people, with the right formulae and the right applications, it can work miracles. But if the quality of the drug is bad, or it gets in the hands of the wrong people, it can be deadly. So too with data. Even the best data is useless if you're trying to use it to tell the wrong story. And even the most talented data geek can't make sense of bad data.
So here are 7 tips to avoid Big Bad Data.
1. Check the quality.
The biggest problem with analyzing social media is the quality of the data. The majority of searches will be far too broad to yield high quality data; you need to carefully construct your search string to exclude irrelevant mentions. This is particularly true if you have a brand name like SAS, which will turn up in all sorts of mentions unrelated to what you are interested in. At the very least you need to make sure that you are screening for misspellings, irrelevant subjects, and irrelevant sites. Content farms and spammers are big problems. Research has shown that up to 40% of data coming into your measurement system could be irrelevant.
2. Check your formulas.
Even experts make simple mistakes in Excel. The now infamous flawed Reinhart & Rogoff study on the impact of debt on national economies is illustrative of two problems. The first is that by simply missing a row or two they affected all future interpretatons of the data and possibly caused recessions around the world.
3. Throw out your assumptions.
The second lesson to learn from Reinhart & Rogoff is to avoid the inclination to interpret causation in a way that suits your premise. Experts analyzing the Reinhart & Rogoff data suggest that they got the causality backwards. (Bonus: Here's a recent letter from R & R to Paul Krugman which defends their work and provides some interesting background.)
The problem is that if you are using data to prove a particular point, you will naturally view the data in a way that supports your argument. Read Nate Silver’s excellent book "The Signal and the Noise" for many more examples. Silver’s premise is that we need to be constantly tossing out our assumptions and continuing to add data to our analysis if we want to improve the quality of our predictions.
4. Don’t confuse causation and correlation.
The majority of marketing mix modeling and other formulas used in PR and social media measurement show correlation, not causation. Sometimes they show both, but be careful with your explanations. There are many, many instances when correlation is all you need to show. If you are claiming cause and effect, you need to have very good data and explanations.
5. Ask "So What?"
Too many people confuse numbers with insight. Insight comes from humans, not a spreadsheet. So when you look at numbers, make sure you apply some common sense. When someone tells you, “We made 1 billion impressions last quarter!” do not be impressed. Ask them, "So what?" What good is 1 billion impressions? The population of the U.S. is only 314 million, so the question to ask is: "So what if you reached everyone in the U.S. three times?"
I recently read a report by an agency that claimed that they generated 600 million impressions, and as a result generated sales. Which sounds like a terrific achievement, until you realize that in the end they only sold 18,000 units of a $50 product. An alternative interpretation was that only .0057% of the population were persuaded enough by their story to actually buy something. So, was reaching a half a billion people really the goal? What if you reached twice that and didn’t sell any more product? What if every single person bought it and you couldn’t deliver the product or service? Is reaching a billion people really a useful definition of success?
6. Remember your audience.
When you are reporting on data, make sure you understand the needs and perceptions of your audience. They usually want to hear about what the data means, not how you got it to reveal its secrets:
- If you’re presenting data to the Board of Directors or Senior Leadership Team, remember that they have the attention span of a gnat. You have about ten seconds to make a point they'll remember, so if you going to use algorithms or anything complicated, be sure they have all the necessary explanations ahead of time.
- If you’re using or presenting data to employees, bear in mind that they'll view everything from the perspective of its impact on their work, their familes, and their job security. So explain the data in those terms.
- The media want a headline, so give them one. Lead with your best conclusions, not a detailed explanation of how you got there. You can fill in the details later.
7. Keep business goals top of mind.
No matter how good the data is or how solid your analysis, the results must reflect the goals of the business. So when you are looking at data, make sure it reveals what someone needs to know to run the business more efficiently or more profitability.
### (Thanks to Bianculli's Blog for the image.)
Katie Delahaye Paine is Chairman, KDPaine & Partners, (a Salience Insight company), and Chief Marketing Officer of News Group International. KDP&P delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, Chairman of KDPaine & Partners, will be glad to talk with you about measurement for your organization.