The Measurement Standard: Blog Edition

How Public Relations Measurement Can Win the War on Terror

It's not a new thing to talk about the war on terror as a war in the media for mindshare. But I've never seen it stated as explicitly as in the article "Marketing Osama" which appeared in a recent issue of The Week, and which was reprinted from a story by John Cook originally published in Radar. The point there is that Brand USA and Brand Osama are fighting it out in the media, and this marketing/PR battle is the important front in the war.

Public relations measurement evaluates exactly the sort of metrics that define the strategy that al Qaida has been using to enlarge its support: media impressions, media content, key messages, contributions and new members. Public relations should be not only measuring the war on terror, but, by virtue of tracking the most important metrics, should be defining the war on terror too. (See "Knowing the Enemy: Can social scientists redefine the "war on terror"?")

One of the more interesting aspects of the war on terror is that there is, (so far, that I know of) no way to know if we are winning or losing, or how we will ever know if we do win or lose. And that's where public relations measurement comes in: If we can use public relations metrics to define progress in the war, we can use them to define success and failure. Public relations measurement can win the war on terror by defining when the war will be over.

And here's how we do it: Let's say that al Qaida's media and marketing efforts now generate X impressions of their key messages each year, which in turn result in Y dollars of contributions and Z new members. We gather and track this data for the last ten or twenty years, and correlate it against terrorist activity of all sorts. (Yeah, it's a big job.) We then will understand what level of exposure to key messages (or Osama bin Laden videos) is required to generate what levels of recruits. Or to generate one new suicide bomber.

Then we define success in terms of the metrics. So, we could define "winning the war on terror" as when al-Qaida's impressions, contributions and recruits are dropping. And then we define victory as the point at which these metric fall low enough so that al Qaida can no longer function effectively. If Osama bin Laden doesn't get his message out, or if no one believes it enough to do anything, then we've won. --Bill Paarlberg

NYTimes Article on Web Stats: Is Web Measurement Technology Stymied by Human Beings Just Being Human?

Don't miss this NYTimes article in today's paper that nicely sums up the on-going Web visitor counting controversy (at least from the public's view). Makes me wonder about the reputation of public relations and of public relations measurement: Here we are congratulating ourselves on being able to measure relationships, and being able to measure trust and transparency and various other esoteric things, but we can't nail down something simple like measurement for Web visits.

What's the problem here? It's not lack of data, there's piles of that. And you'd think that there would be plenty of motivation, what with all the money hanging in the balance. Is it as basic as a battle for business dominance; human beings just being human? "My number's better than your's!"

Maybe the real measurement problem lies someplace other than in the Web traffic reports--like between the ears of the people who use them. Geeze, this relationship measurement stuff might come in handy. (And, on a completely different note, how about those Red Sox, eh?) --Bill Paarlberg

When Is An Impression Not An Impression: How Does Public Relations Measurement Take News Timing Into Account?

The other day I was re-reading Ed Moed's blog post about how much media coverage depends on what other events happen to be happening at the same time. His point, in part, was that events get pushed out of the media by other events, and what we think of as news actually depends greatly on what other news is also being covered.

And this got me to thinking about how public relations measurement typically measures media outputs with media content analysis, but -- to my knowledge -- does not typically take the news environment into account.

My point is that Article X might result in ten zillion impressions, but if it was published on a day with really big other news (an extreme example would be 9/11/01) then those impressions just aren't the same as they'd be on a slow news day.

We all recognize that the news hole shrinks when something else big is hogging all the coverage. But what we're talking about here is not whether or not there's room for an article to get published, but whether or not there is room in the reader's head for another article. Or conversely, if there is no other news in a reader's head, does an article somehow gain extra importance?

I asked Katie Paine about this and she said she recognized the phenomenon from experience. As to how to deal with it, measurement-wise, she emphasized the importance of human experience behind data analysis: "That's actually why [KDPaine & Partners'] human intervention and hybrid approach is so important. The automated systems just give you numbers, we give you an explanation."

Does anyone out there adjust or modify their media analysis results to take into account the news environment? (I mean formally, with some kind of multiplier or variable that corrects raw impressions data for the impact of other news activity on the reader.)

Some might argue that correcting for the news environment doesn't really belong in media analysis, that it is part of some later stage of public relations' impact on the public's mind. If we put that in terms of outputs, outtakes, and outcomes, then what we are talking about here happens somewhere between outputs and outtakes: Between the outputs and outtakes lies the news environment, which may enhance or inhibit the results of impressions.

To my knowledge, measurement does not formally take this "news environment" phenomenon into account. How does the news environment affect PR's effectiveness? How does PR affect a person differently according to what other news they are experiencing?

In "Measurement's Empty Head: Measurement ignores the most complex part of PR" I wrote something that's pertinent here:

Contemporary PR measurement treats the mind of the media consumer like it's some sort of an empty box to be filled: The media dumps in impressions and we measure the outtakes and outcomes. But human beings aren't just empty heads, and they don't consume media in a simple and rational fashion... Measuring impressions is a whole lot easier than measuring what happens to those impressions once they're inside a person's head... What happens there is probably the most complex part of how PR happens, yet state-of-the-art PR measurement doesn't take it into account.

--Bill Paarlberg

A Public Relations Measurement Index Number Based on Ideal Coverage

The Paine of Measurement

Optimum Content Score: An Index Number to Love?
Here's a quick and easy way to design your customized article content score.

For years I've ranted about why a single Index number or a standard PR metric won't work (see "The Problem of PR Indexes: Magic Number or Big Headache?"). My argument has been that each PR program is different, with different goals, different audiences, and different measures of success. So how can you possibly compare them all with one number?

Embrace the Differences

However, I am beginning to believe that there might be a solution to this problem, and that is to embrace the differences--to build the differences into the measure. Here at KDPaine & Partners we now offer our clients a measure that is tailored to match their individual program goals: the Optimum Content Score (OCS). It gauges the success of articles, based on what the client considers to be the ideal article. With the OCS, we distill a potentially large number of variables into one number, based on the relative concept of “ideal" coverage. That way, we can tell you whether your coverage was more ideal than last year (comparing to benchmark), or less ideal than you wanted it to be (comparing to objectives).

What Is the Ideal Article?

For any particular client, we define an ideal article based on their coverage goals at the present time. So, for instance, an ideal article for Client X might be one that:

• contains several key messages,
• leaves a reader more likely to purchase,
• mentions the brand in the headline, and
• appears in the Wall Street Journal.

If an article includes all these attributes, we give it the maximum score, ten out of ten. To determine the score for any given article, points are deducted from the maximum based on how far an article is from ideal, taking points off, for instance, for negative positioning and minor mentions.

The beauty of this system is that the company that is looking to keep out of the headlines can gauge its effectiveness as well as a company that is trying to get into the headlines. For instance, in the middle of a crisis, ideal press might be a minor mention that contains a key message and is balanced.

I realize we're still measuring outputs here (and our long term goal is to measure outcomes), but this sort of customized-to-goals score is a big improvement over imposing one score across a wide variety of organizations and industries that may not have a lot in common. See Ed Moed's "Measuring Up" blog post on the subject here.

OCS allows you to look at those things that are important to your own business, insuring that you are measuring the most relevant elements of what you do. For instance, Sabrina Steele at Raytheon has used it to make better decisions on supporting trade and air shows, using spokespeople, and comparing her effectiveness with her peers.

Another advantage of OCS is that it can be compared to other marketing data to determine what does, in fact, drive outcomes like Web traffic, product preference and consideration. Factor analysis using OCS can tell you exactly which program elements and media efforts are having the most impact on whatever are your ultimate marketing measures of success (typically sales, sales leads or market share).

Design Your Own Article Content Score

You can easily get started with your own version of an optimum content score. Below is a sample list of questions--and you may wish to remove some or add your own--that you can use to determine the characteristics of the ideal article for your own program. Then rate your articles on how closely they come to your ideal.

Wishing you large measures of success,

Sample questions to determine ideal article content:

1. What is the goal of the program? (pick only one):

a. Increase exposure

b. Keep bad news to a minimum

c. Disseminate key messages

d. Increase preference

e. Generate awareness for spokespeople

2. How important is brand or sub-brand visibility?

a. Extremely important

b. Somewhat important

c. Not important at all

3. How important is it that the story be exclusively about your organization or brand?

a. Extremely important

b. Somewhat important

c. Not important at all

4. How visible do you want your spokespeople to be?

a. Highly visible

b. Somewhat visible

c. Doesn't matter

d. Invisible

5. How important is competitive positioning?

a. Extremely important

b. Somewhat important

c. Not important at all

6. How important is the tone of the coverage?

a. Extremely important

b. Somewhat important

c. Not important at all

 

 

Public Relations Measurement for Travel and Tourism

Measurement Strategy

How To Measure PR's Impact on Travel and Tourism
Your six-step passport to success.

by Katie Delahaye Paine

The good news about travel and tourism PR is that it is highly measurable. The bad news is that almost everyone does it wrong. The good news again is that we're going to show you how to do it the right way.

The Wrong Turn: AVEs Are a Dead End

The traditional way of measuring tourism PR has been to use Ad Value Equivalency: Count up the number of column inches generated by earned media and figure out what it would cost to advertise in the same publication. What you come up with is a dollar figure that tells you just about nothing useful, other than that you got some amount of ink somewhere.

(Regular Measurement Standard readers know that we frown on the use of AVEs. The very serious limitations of this technique are well documented. See this section of The Measurement Standard Blog Edition for a discussion and further references.)

On The Right Track: Measure Something That Will Get You Where You Want To Go

What you really need to know is how your travel and tourism ink impacts tourists and their spending: How did your PR efforts change sales and meals tax revenue? Butts in busses? Ticket sales? Heads in beds? Once you know how your efforts affect tourists' behavior, you can go on to decide how to adjust your efforts to further the overall goals of your program or department.

As the great guru of measurement Dr. James Grunig of the University of Maryland reminds us: "The main reason to measure objectives is not so much to reward or punish individual communications managers for success or failure as it is to learn from the research whether a program should be continued as is, revised, or dropped in favor of another approach."

The reality is that there are lots of very accurate ways to measure travel and tourism PR:

• Perhaps the most famous case is Southwest Airlines. By embedding a unique URL that takes visitors to a mirror landing page in each of their Search-Engine-Optimized press releases, they can tell exactly how many tickets they sell as a result of each press release they send out. See this Measurement Standard article for more.
• Elisa Camahort, co-founder of Blogger, uses a similar system to measure the impact of blogging on ticket sales for her clients in the theater world. She gets actors to blog about upcoming performances and then tracks the number of tickets sold by tracking the number of unique visits from the blog to the ticket sales page of their web site.
• The State of New Hampshire's Travel and Tourism Department has seen its budget grow every year, in a traditionally tight-fisted state, in part because it can show exactly how much impact its efforts have on the state's revenue. By looking at historic data on the total reach of its advertising efforts in a given quarter, and dividing that number by the number of visitors in the same quarter, it has established that for every 100 opportunities to see a positive message about the state, three people will visit. It also knows that each visitor, on average, spends $81.76 every day they are in the state. (A complete description of how the New Hampshire Department of Travel and Tourism calculates its ROI for marketing efforts will download from this link.)

What's It Worth To Be First In The Nation?

When the people at the New Hampshire Political Library wanted to calculate the value of New Hampshire's First-in-the-Nation Presidential Primary to the state, they started by adding up the direct spending of the various political campaigns and media outlets.

But that didn't account for the long term impact of the press that 4,000 visiting journalists generated for the state. A media analysis of some 5,000 articles revealed that approximately 10% of them were positive – leaving someone more likely to either visit the state or do business in the state. This translated into 22 million positive impressions.

Using the above NH Travel and Tourism formula, and using only those articles that left an out-of-state reader more likely to visit the state, they calculated that the press coverage was likely to generate an additional 660,000 visits and thus generate some $540 million in new tourism revenue.

Additionally, Ross Gittell, James R. Carter Professor and Professor of Management at the Whittemore School of Business at the University of New Hampshire, calculated that approximately 2% of all those visitors would be business owners or executives looking to start or expand a business in the state. That translated into some 13,200 executive visits. Assuming conservatively that just 1% of those executives started or expanded a business in the state, the media exposure would have yielded 132 new businesses. Since the average business in New Hampshire employs 20 people, media coverage would have generated 2,640 new jobs.

Although New Hampshire doesn't subsidize new business, on average, states spend about $10,000 per job created – meaning that the media exposure from one primary had a business development potential worth $26.4 million.

(A complete description of how the New Hampshire Department of Travel and Tourism calculates its ROI for marketing efforts will download from this link.)

In addition to these financial calculations, the NH Travel and Tourism people also keep a close eye on website visits and click-throughs, calculating an average click-through rate for all promotions as well as a cost-per-click-through calculation that enables the department to calculate which promotional efforts are most efficient at generating click throughs. This allows them to better plan future promotional efforts.

Once again, numbers like cost-per-click-through and revenue should not be used to simply justify the existence of a media relations department. The real value of all this measurement is to better understand how specific tactics and strategies impact the overall goals of your tourism efforts.

PR Measurement Must Move Beyond Measuring Readers and Start Measuring Friends: The New Measure of Success is Popularity

The Paine of Measurement

Why Measure Eyeballs When You Can Measure Friends?

The latest news is the work the IAB is doing to make the numbers agree between the two leading eyeball counters comScore and Nielsen//NetRatings. (We gave these two firms our Measurement Menace of the Month Award this month.) By the time they get those guys to agree, the concept of counting eyeballs will be obsolete.

Google, with its revamp of Google Analytics, is doing a much better job keeping up with the latest demands in measurement. If you're not using it, you should be. It tracks not just traffic but sources of that traffic and conversion rates. It's data you can use to decide which tactics and strategies are paying off and which are a waste of time. Not based on eyeballs, but based on the response from your audience. Now that's useful stuff.

And besides, why measure eyeballs when you can measure friends? Whether it's on LinkedIn, MyRagan, Twitter or MySpace, the new measure of success is popularity. Actually, hasn't that been the measure of success since before the ruler was invented? Among political candidates, the business of comparing the number of friends is rapidly replacing the business of counting the number of column inches and campaign contributions raised. And, I have to say, it's a bummer if I start my day on Twitter and it tells me I have no friends.

The reality is, we all have friends. I'm talking about real friends in real life. When you're going through traumatic times those friends come out in droves to help you. Which is what people should really be measuring. Not how many friends you have on MySpace or Twitter, but how many you have that will help you celebrate during the good times and help you survive through the bad times.

Businesses need to look at these new measurement schemes through the same lens. Do they help the bottom line? Do they defend the reputation in tough times? Do they help spread the word when the news is good?

All the MySpace pals in the world may not boost your stock price, but they just might help prop it up a little longer during a crisis. And that's the big point here: Measure the impact of social media against the true bottom line, not an artificial benchmark created by the media.

Wishing you large measures of success,

AVE Gets A Makeover, But It's Still Just Tarted-Up Output Data

We were thrilled to see Erica Iacono's article on AVEs in PRWeek. Ms. Iacono has done her homework carefully and provides a nicely balanced and up-to-date summary of the AVE controversy. And now some of the other blogs are weighing in on the matter, including PRomulgator (who provides a copy of the article).

(As regular Measurement Standard readers know (see our newsletter here), we have not looked favorably on AVEs. Katie Delahaye Paine, TMS's publisher and probably the most prolific writer in the world of PR measurement, has written plenty about their evils, as have others, including Jim Macnamara, and Bruce Jeffries-Fox.)

The gist of new developments is that if you tidy up your media analysis data, "taking into account a media placement's reach, type and prominence of the story, and positioning," then use a conservative ad rate that reflects rates actually paid by advertisers, then what you come out with is a number that is somewhat more accurate and sports a sheen of respectability.

Our take:

1.) AVEs are popular because they appear to be an easy and cheap way to measure outcomes.
A number with a dollar sign in front of it allows somebody somewhere to believe that they can compare PR's results with those of some other marketing effort. They think they are comparing outcomes, but they are only looking at tarted-up output data. There are ways to make this sort of comparison, but for now they are usually much more complicated and expensive. That won't be the case forever. When tools come along that allow quick and easy direct measurement of the effects of any communications or marketing effort on sales (or whatever your preferred outcome), who's gonna care about AVEs?

2.) On the other hand, you can't argue with success.
Angela Jeffrey is one person who has found some darn good results correlating a type of AVE with business outcomes. Hey, if it works, it works. We don't care if your regression equation includes eye of newt and toe of frog: If it works, consistently, then it could be a valuable measurement tool. Here's the text of an article by Paul-Mark Rendon from Marketing Magazine with more on how AVEs can be useful.

3.) Why should PR waste its time using AVEs to mud-wrestle with advertising, when it could be sitting at the boardroom table doing some clean and serious strategizing?
Jim Grunig is the most prominent proponent of the view that PR should set its sights higher than just being a tool to produce sales. PR can provide extremely valuable services that are qualitatively different than those of other marketing tools, but unless we focus on measuring those, we'll never achieve our true potential. From this point of view, AVEs might very well be a costly distraction.

--Bill Paarlberg, editor