The Myth of the “Data-Driven” Business: Eric T. Peterson

Do we try too hard to sell measurement?

Eric T. Peterson at WebAnalyticsDemystified posted an article yesterday called “The Myth of the “Data-Driven” Business.” He argues that although we measurement types often preach the goal of data-driven decision making (in his case it's web analytics, over here in PR we often talk of “the data-driven communicator”) what we should really say is data-informed decision making. That we should not become too much a slave to data, and that we should leave room for other business knowledge and experience:

...being “driven” implies imbalance and over-correction — going out of your way to devalue experience, ignore process, and eschew established governance in favor of a new, entirely metrics-powered approach towards decision making.

...I simply have not seen nearly enough evidence that... a “by the numbers” approach creates the type of result that the “data-driven” school seems to be evangelizing for.

He asks the provocative question of whether or not measurement evangelists (that would be us here at The Measurement Standard) sabotage our own efforts by making measurement seem too easy, and too much like it will fix anything. Whereas in reality it is a complex and confusing world out there, and the data that measurement provides doesn't always have the answers.

What do you think? You’ve read a lot about measurement. Do you think we oversimplify the world to make measurement seem more valuable than it really is? Do we damage the reputation of measurement by claiming it is more effective or useful than it really is? --WTP

(I think that image is from Monty Python and the Holy Grail, but I'm not sure.)

--Bill Paarlberg is editor of  The Measurement Standard blog and newsletter, and of Katie Paine's new book Measure What Matters. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

Comments

[Frank Walton]

The disappointing thing about measurement is that it always oversimplifies (this coming from a measurement evangelist, remember). A doctor evaluates your health by measuring weight, blood pressure, cholesterol, etc. No one measure tells the whole story -- and how many measures does the doc need till she gets a "complete" measure? Science works by chopping up our lived experience into pieces, and then assigning attributes and quantities to those chunks. And that's what we do in PR measurement. We just need to keep our scientific-method-modesty string tied prominently around our finger.

The argument between data and something else (gut feel, tradition, belief) is centuries old; maybe it's always been with us. Remember a few weeks ago all that gush in the media when Steve Jobs announced his stepping down as CEO -- he ignored the data; the customer doesn't know what he wants; the creator/dictator decided what we would buy and use. Goofiness factor aside, there is some "truth" here -- history shows lots of successful decisions ("lucky"?) made without or in despite of data.

All that said, I certainly prefer a science-based pilot and air traffic control system when I fly. I expect my doctor to measure the heck out of my vitals, and I'll go get a second (data-informed) opinion to maximize my confidence in a diagnosis. No question about the value of data. The question is: what kind of company (or person) do you want to be? How do you want to make decisions?

[Bill Paarlberg]

“The disappointing thing about measurement is that it always oversimplifies.” I really like that Frank, thanks, and will use it someplace, soon. It seems to be both obvious yet fairly profound all in one. In our effort to understand the complexity of the world, we must err in the opposite direction.