In my first 25 years of measuring communications, I’ve learned a thing or two ’bout birthin’ measurement programs (to fracture a quote from Thelma McQueen in “Gone with the Wind”). By nature, I’m probably closer to being Mr. Spock rather than Doctor Spock. I tend to focus more on what a program will do, rather than on the pain of getting it started.
But, having recently endured my own personal measurement start-up ordeal (see “How I Slept My Way to Measurement Success”), it occurred to me that there are a number of pitfalls and opportunities for disappointment in the process. So I thought back over many, many beginning measurement programs, and wrote down the most common unrealistic expectations. Here they are.
The Top 5 Unrealistic Expectations For Starting A Measurement Program
You expect to have a “measurement number” that will justify your budget and answer all questions.
First of all, measurement isn’t a single number. Your CFO doesn’t go into the Board of Directors meeting with one number. So why would Communications — made up of such a myriad of different components — think it can demonstrate results with one number?
Secondly, measurement is a long-term process. It will take you time to identify changes and trends. Your manufacturing VP doesn’t report on the productivity for today, he or she reports on changes in productivity month-to-month, quarter-to-quarter, and year-to-year. That’s what you’ll be doing, too.
You’ve just gone through an incredibly complicated screening process to find the right vendor, and now you expect everything will be simple and easy.
Like any research project, there’s a lot of data involved. And, as we know, data is never simple, no matter what your program or what tool you choose. You expect to be able to sign the purchase order and have your metrics magically appear. It doesn’t work that way. You are going to have to put in lots more time if you want to get useful data.
You’ve spent all this money, so you expect your data will be 100% perfect from day one.
Expect glitches in the data, especially to start. Computers, like children, take a while to learn the rules. For the first few months you are going to be sorting out search terms and media outlet specifics, and you and your computer(s) are going to make some mistakes.
At first you will probably define search terms that are far too broad, and you will be inundated with mentions that have nothing to do with your organization(s). This is what is called a learning opportunity. For instance, in the world of social media, you’ll be amazed at what people think your name is. Just ask the folks at CarMax (the car-selling company) and CarMex (the lip balm.) Or SAS, the analytic software company, (not the airline or the Semester at Sea folks). Then you narrow your search and the next thing you know the product management types will be screaming because you missed something.
It’s probably going to take at least three months to get it right. You can shorten this cycle if you chose to work with a very clearly defined list of media outlets, defined by specific URLs. Which means not, for instance, asking for “Business Week blog.” In which case, you will discover that there are actually something like 170 of them. Which one, exactly did you have in mind?
You expect results right away.
Unless you know how to do a Vulcan mind meld, you will need to spend time with your measurement vendor’s staff to very clearly define the parameters of your program. You will need to define subjects, messages, campaigns, and any other details you will be tracking. If you give your vendor the wrong messages, incorrect competitors, or out-of-date product names, then they will not know it until you discover it and tell them. If it’s on your website, they will assume it to be current. And if it’s not on your website and you haven’t told them about it, how will they know it exists?
You expect your first report to explain everything.
Your first report will be confusing. No matter how clear the program seems to be, your first report is going to throw you for a loop. There will be charts that use language you may not understand. There will be charts that look weird. Until you see it visually up there on the screen, you probably won’t understand what you’ve ordered. That is the nature of data -- and remember, some of your early data will probably not be what you expected.
People will question your data and the methodology. Like any expectant mother you will need to push (back) hard. It will hurt. Odds are, you are going to have work at it some to get what you want from your measurement. Change is painful but in the end, you’ll have a beautiful product. Congratulations.
Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements.