The Measurement Standard: Blog Edition

Measuring Engagement on the Web by How Often Visitors Cut and Paste

This is really cool. Zachary M. Seward at Nieman Journalism Lab reports on Tracer.

How Can We Take Office Politics Into Account When Doing Public Relations Measurement?

This is the third of three daily posts on the general topic of politics and public relations measurement. The crux of the problem is: If office politics is important enough to bury your measurement report, then shouldn't you take it into account from the start? 

For instance, suppose you know that a manager will hide your report if it shows results that are not flattering to him. And suppose you actually give him such a report, and then he does bury it and no action is taken as a result of your work. So where are you? Did you measure the wrong thing? Or did you just do your job?

A real-life story: Almost twenty years ago now, I was a bright-eyed, idealistic, and naive youngster, with some experience in publishing, and with a graduate degree earned by doing experiments on human beings. I applied for a job writing research reports for one of the major public relations research vendors. During the interview, the Director of Research explained the job: "Every measurement project has a story behind it. Every report we write means someone's ego or budget or job or career is on the line. I have to know what situation we are getting into, and we have to take that into account when we present our data."

Well, I was pretty much appalled, and refused the job. Didn't want to distort what I saw as my objective integrity. But today, of course, I see The Director's point and would love to be able to go back and ask some questions: How to take the backstory into account? What kind of research or snooping is necessary to find it out? How is it possible to present the research data and at the same time allow for the bruised egos or budgets that might result? --Bill Paarlberg, Editor, The Measurement Standard

Office Politics and Public Relations Measurement: Are we measuring the wrong thing?

 

Yesterday I wrote about public relations measurement and office politics, and brought up the notion that measurement, no matter how solid the data, will not be effective if people have a strong incentive to misrepresent the results. We have all heard of or experienced measurement reports that have been ignored or buried because the results were not as good or as flattering or as useable as someone hoped they would be. I would very much like to hear from anyone doing research on this: How often do public relations measurement reports get ignored because of politics? For what reasons do the reports get ignored?

Some measurement vendors will say, "Hey, that's not our problem. They hired us to tell them what results their program was getting, and if they don't like what we tell them, well, that's just office politics that we can't control. So what if our report got buried somewhere? We still did our job."

But here's the crux of the problem: Maybe we are measuring the wrong thing. What good does it do to present your gold-plated research results when they don't fit into the politics or office culture that is going to use them? If your measurement report was doomed to be ignored, then not only was the measurement not effective, but you measured something that was of no value in that situation. You measured the wrong thing.

Wouldn't it have been better to somehow take the politics into account, and actually produce a report that had a chance of being used, of being effective? People must do this already to some extend. Scope out the political lay of the land before they dive into a project. But no one ever talks about it as a vital component of the measurement process. So, how do you do that? What's the technique or procedure or system to plan or deal with this? -- Bill Paarlberg, Editor, The Measurement Standard

Concerning "The Wire" and Measurement: How does politics affect public relations measurement?

So I just finished watching the fourth season of The Wire. (The Wire is an HBO TV series about Baltimore. I watch it on DVD, and it's got to be about the best TV ever. Right up there with The Sopranos and Deadwood.)

One of the themes of the fourth season was "juking the stats," the way in which the police and education departments distort their results to make them appear more effective than they actually are. For instance, the cops would purposely not investigate homicides, in order to keep the official city homicide rate low. And the schools would classify as "proficient readers" those kids who were reading two years or less behind their actual grade level. 

The big point here is that measurement, no matter how solid the data, will not be effective if the people who are doing it have some strong incentive to misrepresent the results. There must be a lot of public relations measurement programs that never get reported, or get distorted in the way they are reported, because the results are inconvenient or embarrassing or otherwise unwanted. So, what good is it to have the best measurement tools and the most accurate measurement program, when the results are destined by office politics to be ignored or distorted?

Here at The Measurement Standard we spend a lot of time and effort trying to convince people that measurement can improve the effectiveness of their public relations, and showing them better and more accurate ways to collect and analyze data. But we never talk about the politics behind measurement as an important aspect of the process. (Well, Katie and I do mention "getting buy-in from everyone involved" in her book, but only very briefly. As if all you have to do to get around awkward politics is to hand people a signoff sheet or something.)

So my purpose here is to propose that maybe we have been ignoring a very important part of the process. Maybe the office politics behind measurement can be just as important as actually getting the right data. (I notice I am not the first to bring this up. See Victor Tan Chen at In The Fray.)

So how can we get a handle on this? Who's doing research on this? How can we get an idea of how many measurement programs have been scuttled, or languish on a shelf somewhere, because the results were not quite what someone, or everyone, had hoped?  -- Bill Paarlberg, Editor, The Measurement Standard

The All-Purpose, One-Size-Fits-All Public Relations Measurement Program Checklist

Measurement Tools 101

 

Everything you need to know before you start any public relations measurement program. Don't leave home without it.

This basic information checklist could be the handiest, most convenient, and most valuable tool in your public relations measurement quiver. No matter what project or media you are working with, these are all the things you should know -- all the ducks to get in a row -- before you jump in the water.

Some of these are easy, some are hard, and for some you've got to think, maybe make some tough decisions. But, by the time you're done your brain -- and your project -- will be up to speed. Plus -- Bonus! -- it will be obvious to your client or your boss that you've not only got a clue, you've got the measurement bull by the horns. And, yes, you can download the Checklist here as a pdf.

The Measurement Program Checklist

Step 1: What are your objectives?

1. What are your organization's key goals for this year?

2. What are your department's key goals for this year?

3. What do you hope to accomplish with your measurement report?

• Get budget approved
• Increase budget
• Increase head count
• Get more internal support
• Get more external support
• Justify my existence
• Get a raise
• Get promoted
• Change the world

Step 2: What audiences are you targeting?

List all that apply, not just the following common ones. (If in doubt, put all the communications people in your organization in a room and ask them.)

• media
• prospects
• customers
• partners
• employees
• governments
• NGOs
• communities
• investors
• thought leaders
• international community
• other

How does a good relationship with your various target audiences benefit your organization?

1. Increases sales

2. Increases attendance

3. Increases donations

4. Increases likelihood of desirable legislation passing

5. Increase preference

6. Gets messages out

7. Improves employee retention

8. Improves employee loyalty

9. Improves customer retention

10. Improves customer loyalty

11. Improves likelihood of purchase

12. Attracts new customers

13. Attracts new prospects

14. Attracts new donors/potential donors

15. Increases amount of purchase

16. Increases frequency of purchase

17. Boosts stock price

18. Increases profitability

19. Reduces turnover

20. Decreases time to market

21. Decreases number of complaints

22. Decreases absenteeism

Step 3: Set priorities

Prioritize your audiences with this exercise: You have a total of 100 points to allocate. Award those points to the audiences that you have identified in order of their importance to your organization, based on your answers to questions in Step 2.

Step 4: Determine a benchmark

Who or what keeps your boss/client up at night? In other words, what are the competitive threats or perceived competitive threats to your organization? Select from the following lists to determine what you will be comparing your results to:

• Competitor(s) / Peer companies
• Industry benchmarks
• Yourself over time
• Last year's results

Step 5: Select the right measurement tool

If your objectives (Step 1 above) include increase awareness, attitude change or education, you will need to conduct a survey. Do you have email addresses for all those you want to survey? Do you have telephone numbers for all those you want to survey?

If you are seeking to measure sales and leads, you should be tracking web site traffic. If you are measuring media relations you will need to consider the following criteria in your measurement efforts:

• Tone of coverage
• Share of voice vs. the competition
• Share of discussion vs. the competition
• Share of spokesperson visibility
• Degree to which you have communicated key messages
• Degree to which you are favorably or unfavorably positioned on key issues
• Audience reached
• Prominence of coverage – the visibility of your brand within the story
• Dominance of coverage – the extent to which your brand dominates the coverage or is subservient to another brand
• Proactive vs. reactive media
• Key analysts quoted
• Type of analyst quoted
• Key reporters
• Key publications
• Key topics or subjects
• Type of article (check all that apply)
  • ____Letter to the editor
  • ____Editorial
  • ____Product review
  • ____Product brief
  • ____Industry round up
  • ____Application article
  • ____Article about the competition
  • Other _____________________________________
• Key messages, list below:

  1._________________________________________________________

  2. _________________________________________________________

  3. _________________________________________________________

  4.__________________________________________________________

5. __________________________________________________________

• Industry issues:

• __Employer of choice
• __Investment of choice (financial strength)
• __Vendor of choice (good/best value for the money)
• __Good corporate citizen
• __Environmentally responsible
• __Global leadership
• __Other: _______________________________________________________
• Prominence
  • Organization is mentioned in the headline, photograph or caption
  • Organization is mentioned in the top 20% of the article
  • Organization is mentioned in the bottom 80% of the article
• Dominance
• Organization is the focal point
• Organization is one of several mentioned
• Organization is only mentioned in passing
• Other _________________________________________
• Subjects (note all that apply)
  • Financials
  • Management changes
  • New product launch (specify product names)____________________________
  • Coverage of crisis or scandal (specify) ________________________________
  • Merger or acquisition
  • Customer
  • Corporate philanthropy
  • Industry
  • Other _________________________________________
• Influencers and key spokespeople
  • Company spokespeople (provide names)

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

_________________________________________________________________

• Competitors' spokespeople (provide names if available)

_________________________________________________________________

_________________________________________________________________

_________________________________________________________________

_________________________________________________________________

• Industry analysts
• Financial analysts
• Government officials
• Educators
• Other

Other considerations:

• Approximately how many clips a month appear about you and the competition? (If you are currently receiving more than 250 articles per month you may want to consider computer-aided content analysis.)
• How frequently do others in your company report results?
• How often would you/your boss/your client like to see a PR measurement report?

###

Easy Strategies to Get Started Measuring Your Public Relations

 

Quick and simple techniques to start measuring your public relations programs and impress your boss.

by Katie Delahaye Paine and Bill Paarlberg

If you're reading this and you're not actually measuring your public relations programs yet, then the only New Year's resolution that really counts is for you to get started measuring. Which, as it turns out, is going to be a whole lot easier than those other resolutions you made a couple weeks ago and probably flunked out of already.

Measurement doesn't have to be a big deal. We've rounded up plenty of ways you can get started without spending much time or money at all. In fact, for several of our tips below, the service is free and the data gets delivered right to your in-box. Of course, if you want to get beyond the newbie stuff you might have to click around the Web some, do some reading, maybe break a sweat thinking about what your data means. (Relax -- that's a different article.)

Here is a baker's dozen of quick and painless techniques to get something started.

1. Get with Google:

• Sign up for Google Alerts for your organization or brand, and for at least three of your competitors or peers. Then tell your boss you have installed a crisis alert monitoring system.
• Put Google Analytics on every website that you control. (Yeah, the instructions need a serious usability makeover, but once you get the code, you just paste it into all your pages. Cake.) Then tell your boss you don't need no stinking WebTrends. (Actually, you probably do need WebTrends if you have to do advanced analyses -- it's much more robust and customizable -- but we're talking quick and easy here.)

2. Time for Twitter:

• Go ahead and get yourself on Twitter. See what it's all about, in 140 characters or less.
• And even if you don't want to start Tweeting yourself, you can still set up an RSS feed for a Twitter Search on your brand/s, your competition, and anything else you want to keep track of. Tell your boss you've implemented a Twitter monitoring system.
• And for you heavy Twitter users, see how influential you -- or your boss -- is on Twitter: Go to www.twinfluence.com.

3. Educate yourself:

• Stay on top of social media every day, with a click or two: socialmediabiz.com will send you daily email updates.
• Get to know the Institute for Public Relations website, especially the research and papers on measurement.
• Check out any of the numerous YouTube videos that talk about how to measure public relations and Social Media, (especially, in all modesty, the ones that feature our Ms. Paine).
• Read Katie Delahaye Paine's book How to Measure Public Relationships. OK, it's not free, but measurement gurus got to make a living. And it's the best $29.95 you'll ever spend on measurement. Click here to buy it now.

4. New media meets old school:

• Set up a free Survey Monkey account and start surveying your constituencies.
• Back to basics: Use Survey Monkey to measure your relationships. Send out a survey to assess the level of trust, commitment and satisfaction they feel about your brand.

5. More easy online stats:

• Get yourself a Facebook account and, at least once a week, conduct searches on your brand and your competition. Then tell your boss you are monitoring Facebook. (For more on measuring Facebook, see "Measuring Community on Social Networking Sites: A detailed how-to guide for measuring MySpace and Facebook.")
• Go to www.xinureturns.com and see how your URLs stack up.
• Go to www.compete.com and see how you stack up against the competition.
• Go to kineda.com and use their widget to see how your "bloglebrity status" ranks.

6. All together now:

• Now put together your own dashboard. Whatever metrics from above you actually have, put them in an Excel spread sheet and set up Key Performance Indicators. Repeat monthly. Progress confidently toward success.

Katie Delahaye Paine's Top Ten Public Relations Measurement Tool Wishes for 2009

Public Relations Measurement Tools We'd Like to See

1. An operant conditioning device
...attached to a sensitive body part of all corporate spokespeople that gives them increasingly painful sensations when they stray off message, and increasingly pleasurable sensations the more on message they are.

2. One source...
...that accurately, and completely gathers items from newspapers, magazines, TV, radio, blogs, Twitter, online publications, websites, YouTube, Facebook, Flickr, etc. I know I've been asking for this for five years, but Santa, I've been really really good and really, really patient and I want it now! I can't be the only one: Who in their right minds wants to filter through eNR, Cision, Meltwater, Burrelles, Vocus, VMS, Radian6, Google News, TVEyes, Critical Mention, and god knows what else just to make sure you're tracking the content of all your key publications?

3. A charting program that gives results like this...
...for PR measurement that will automatically take the data from a media content analysis and create a chartogram to indicate where our clients have the most visible presence, the most favorable presence, the most negative presence, and where the most messages are appearing.

4. 100% accuracy...
...from whatever source we use for content collection.

5. SPSS style statistical analysis...
...for Google Analytics. I want the ability to integrate Google Analytics with things like Twinfulence, my blog stats and other analytic tools so I can actually figure out whether my obsession with Twitter really is driving business to my website. It looks that way, but I want some factor analysis to make it real.

6. Google predictive analysis...
...I want to be able to use Google Analytics to tell me what I should be doing more of. Google Analytics is great at doing stuff like that if you're selling widgets, but I want the same capability for services.

7. Another charting program...
...that automatically puts little bubbles on the chart to signify major events that are driving a change in the results. It's a little thing but it makes me nuts to have to manually add explanations to our client's charts.

8. An interactive application for iPhones and Android phones...
...that allows people to instantly give feedback on what they are seeing, reading, watching or hearing directly into our database.

9. A better way than just tags...
...to search for brand mentions on YouTube and Flickr.

10. A required course on how to measure results correctly...
...for everyone who graduates with a degree in Public Relations or Mass Communications. (And, of course, it would be nice if my book, Measuring Public Relationships, were required reading.)

Fascinating Correlation Research on SAT Scores vs. GPA in Predicting College Performance

In public relations measurement, we are often trying to investigate correlations between variables in an effort to determine what effect public relations efforts have on something else: 

• Does greater use of social media result in more votes in an election? 
• Does more effort on building relationships with stakeholders result in lower future legal costs?
• What predicts sales better: The number of positive impressions or advertising sales equivalencies? 

As we all know, it is often a very tricky business to get clear answers in this sort of correlation research. In particular, it is often difficult to isolate the effects of one variable from another.

Well, here's an example of how this type of research can provide very useful results. In today's NYTimes, this article, "The Test Passes, the Colleges Fail," tells the story of how the impacts of SAT scores and high school GPAs were separated to determine their ability to predict college success at SUNY. Thanks to a serendipitous combination of admission policy changes and the structure of the university system, the effects of the two variables were teased apart to yield some strong and very interesting results.

It's not often, of course, that events drop a ready-made, real-life experiment into one's lap. But in complicated social research like predicting student success, or public relations measurement, these situations can tell a marvelous story.  --Bill Paarlberg

Counting Words (Hey, isn't that what PR measurement does?)

In today's NYTimes there is an article about word counting ("He Counts Your Words (Even Those Pronouns)"), and what the frequency of use of different types of words can imply about their author.

Maybe this analysis could be applied to PR measurement. If increased use of causal words indicates health improvements in individual people, then perhaps companies could tailor the subtle subtext of their press releases by using more or less of them.

The article includes a word use analysis of the political candidates that has some counterintuitive results, to my thinking: The Democrats' use is "concrete and restrained," whereas the Repuplicans' is "cheerfully unrestrained, future-focused." Very interesting. --Bill Paarlberg

KDPaine & Partners Debuts MarketFramer

New Measurement Tools

A new social media-based tool to help marketers better frame messages for their markets

KDPaine & Partners launched a new public relations measurement tool last month to help organizations quickly get a handle on what their marketplace is saying and thinking, enabling them to craft their messages accordingly. Based on KDPaine & Partners' 22 years of media analysis and research experience, the new tool, MarketFramer, takes fundamental principles of sociology, public relations measurement and survey research and applies them to social media, blogs and other consumer generated media, to order to produce a report that answers questions like:

• What are the hot issues and battles being discussed in the market?
• How are people responding to significant developments in your market?
• What are the key phrase and search terms that people are using to describe your business space?
• What's the best way to position your organization or products in a new marketplace?
• What's the best way to respond to a developing issue or crisis?
• What does the market think about your most recent product release or announcement?

"Ever since we started measuring consumer generated media in 1995, people have been asking us if there's a way to get answers to questions like these without doing a large-scale survey research project," explained Katie Delahaye Paine, CEO of KDPaine & Partners. "In part they have budget concerns, and they also question the results of phone surveys when many consumers only have mobile phones. But, they're mostly concerned about the time it takes and the complexity of the programs. People don't have time to wait for survey results, they need answers right away," Paine added.

Conversations are driving choices in the marketplace

"At KDPaine & Partners, we know that conversations are driving choices in the marketplace. Because they're based on what people have experienced, we believe that those conversations are every bit as accurate a predictor as traditional survey research. We thought why not analyze and aggregate those conversations to answer the questions clients are asking?"

Paine explains MarketFramer's simple 3-step process. "It starts with a conversation about which markets are important, which brands are important, and who or what might be influencing the conversation," Paine explains. "Armed with that information, we use a variety of social media analysis tools to search the online environment for conversations that relate to the client's issues, market or questions. We then gather and analyze that data to determine trends and issues and deliver to the client a simple, clear, easy-to-understand report that answers the specific questions the client is asking. Additionally, our clients can access their data online 365/24/7."

KDPP has already used MarketFramer to help a networking company figure out a positioning strategy, and to help a non-profit figure out its messaging. Pricing for MarketFramer starts at $10,000 per marketplace and typical turnaround time is about two weeks.