How to Deal with Big Bad Data: Become A Measurement Fox

The Paine of Measurement
by Katie Delahaye Paine

People seem to have the impression that the rivers of data now overflowing their hard drives will solve all their problems. That Big Data is valuable just because there is a lot of it. Such is not the case: Big Data is an opportunity, but Big Bad Data is a problem.

There are several aspects to this issue...

The Wrong Data

The first and most critical one is that much of our deluge of data is the wrong data. Sure, you can count hits, likes, and followers, or what it might have cost to buy the equivalent number of eyeballs. But that is not going to tell you whether your blogger outreach strategy is helping or hurting your bottom line. 

The Wrong Variables

The second problem is chronic myopia as exhibited by a lack of correct variables. If you are trying to figure out why your results went down, but you don’t know what the competition is doing at the same time, you don’t really know anything. I can analyze a 100,000-line spread sheet in Excel a dozen different ways, but if I haven’t taken into account the right variables when I imported the data, my analysis will be worthless. 

The Wrong Approach

A third problem is the lack of a Bayesian approach, i.e., not learning from prior research. For instance, my company's research demonstrates that 40% of social media data is generated by robots or computer algorithms. Which suggests that people ought to use a multiplier of .4 when looking at their social media results. Instead, many agencies and suppliers are still using multipliers (2, 3, 6.317, whatever) that date back to the Mad Men era. The ridiculous numbers that are generated result in a severe lack of credibility. As in -- true story -- one agency reported to the CMO that 300 billion people had seen his messages in the previous year. That number means that everyone on the planet would have seen his messages almost 23 times. Even the Kardashians don’t get that much attention.

If You Only Measure Yourself, You Don't Know Where You Stand

In the world of measurement I’m constantly running into people and organizations that only want to look at their own data. They want to know how they are positioned, how their messages are coming across, if they did well. The problem with this approach is you don’t learn much that you don’t already know. 

A decade or so ago, we were working with a disk drive company that was very keen on measuring its message communications across Europe. Initially, their communications appeared to be highly successful. By clearly articulating its messages in various countries, and by informing its agencies that message communication was what it would be measured on, the company lifted message communication by 100% overall.

The problem was, however, that their market share was dropping precipitously. One conclusion might have been that the messages weren’t working; they had been communicated, but the market wasn’t responding. In fact, their problem had nothing whatsoever to do with PR, communications or messaging at all.

The company's unfortunate situation was that they had no message (or product!) that would address the difficult topic that everyone in the market was talking about – portable storage. On that topic the competition was eating the company’s lunch. But because the company was only measuring its own efforts, it had no clue that the media was talking about the competition twice as often as themselves.

A simple share of ink over time could have alerted them to the problem. But that wasn’t what they had asked for; they had confined their research to a very narrow window on the world.   

The Fox and the Hedgehog

Nate Silver addresses this point with the analogy of the fox and the hedgehog. A hedgehog adheres to a view of the world that is self-defined. Picture your typical old-school CEO who dictates policy, messages, and strategy based on input from senior leadership that is motivated and compensated by preserving the status quo. 

But the fox sees opportunities and goes for them. Foxes are willing to consider data that may or may not support their premises, and are willing to change their conclusions based on what the data reveals. Most importantly, they don’t force fit the data into the view of the world they’ve been hired to support.

As examples, think of hedgehogs TWA and Pan Am, which saw themselves as deliverers of transportation. Whereas foxy Southwest realized that it was in the business of giving people the freedom to fly. Or Blockbuster, which saw itself as a place to rent movies, while Hulu and Netflix understood they were in the business of providing entertainment.

And yes, I've been a hedgehog myself once or twice. For UNICEF I spent a week trying to prove my personal assumption that photos of baby girls were more compelling than baby boys. Turns out neither are particularly compelling when compared to David Beckham’s naked chest! But, even more interesting, Mia Farrow (a far less visible celebrity with a fraction of the followers) turned out to be far more effective at helping achieve UNICEF’s goals.

The Value in Big Data is Quality, Not Quantity

The answer to the big bad data issue is to double down on your quality assurance and checking. Computers can make mistakes even more often than humans can. Expand your view of the world, look at the marketplace, the topic, or the entire conversation around the brand. Get out of your own cubicle, think about what is happening in other disciples, other parts of the world, other similar industries. Broaden your view of the world and become a measurement fox.

Happy measuring,

Katie Delahaye Paine is Chairman, KDPaine & Partners, (a Salience Insight company), and Chief Marketing Officer of News Group International. KDP&P delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.

The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, Chairman of KDPaine & Partners, will be glad to talk with you about measurement for your organization.

 KDPaine & Partners' Facebook Page.