The Measurement Standard: Blog Edition

The Geography of Buzz: Mapping Cultural Excitement

 

There's an intriguing story with some sexy density maps in today's NYTimes, "Mapping the Cultural Buzz: How Cool Is That?" The article presents research that locates cultural hot spots based on the frequency and draw of events like concerts, fashion shows and theater openings. The authors of the research used photographs of parties and events over the course of a year to qualify the events as "buzzworthy," then mapped the location of the events in Manhattan and Los Angeles. The results are maps of the density of different types of cultural events in those cities.

And this got me to thinking that we do much the same sort of thing when we do media analysis in public relations research. We map the density of mentions of companies or products in the media. Then, instead of comparing fashion shows to concerts, we compare products or companies. Would be nice to have some cool density maps, though. 

-- Bill Paarlberg, Editor, The Measurement Standard

Coverage Decline in '09?

The Paine of Measurement

 

 

As media move online, PR and public relations measurement must adapt.

I'm working on a number of year-end wrap up reports for our public relations measurement clients, and I'm starting to notice the impact of the changing face of media on their results. Almost across the board, traditional media coverage (unless you are Obama, a bank in crisis, or a car company) is down. More and more of our clients' coverage is coming from online sources and blogs.

Not entirely surprising, of course. We've been talking about this for months, but it's interesting when you see it begin to play out in your own charts and graphs.

For decades, we've written about the shrinking news hole that impacts results when major events dominate the headlines -- economic meltdown, political elections, 9/11. But in an online digital world, the concept of a news hole is irrelevant. Online, it doesn't cost any more, essentially, to run a 1000-word story than it does to run a 500-word story, and publications can triple the number of stories they write on any one topic with no impact to the bottom line.

Many of my clients -- and yours, as well -- may see a decline in their coverage in 2008 and on into 2009. My hunch is that it's a result of the downsizing of the media organizations themselves. With fewer reporters on staff, media outlets just can't produce the volume of coverage we're used to.

I'm not for an instant implying that there's overall less coverage out there, just that the sources of coverage are changing. Non-traditional media like The Huffington Post are still adding staff and increasing coverage. A new blogger is born every minute. Former print journalists, now unemployed, don't suddenly stop being journalists. They just do it on their personal blogs instead.

All of which has interesting implications for public relations. We're already seeing the more sophisticated organizations rethinking their top tier media lists. We predict that this will become a regular monthly exercise in 2009.

But, in today's rapidly changing environment, even the notion of having a top tier list might be an anachronism. Things may be shifting too quickly to make it doable, and the big opportunities may come from places you least expect them. Read this story ("Twitter Comes Through in the Real World") from my friend Chris Nahil and you'll see what I mean.

It may just be time to rethink the entire PR process. Now there's a fitting resolution for the new year.

Wishing you every measure of success,

Counting Words (Hey, isn't that what PR measurement does?)

In today's NYTimes there is an article about word counting ("He Counts Your Words (Even Those Pronouns)"), and what the frequency of use of different types of words can imply about their author.

Maybe this analysis could be applied to PR measurement. If increased use of causal words indicates health improvements in individual people, then perhaps companies could tailor the subtle subtext of their press releases by using more or less of them.

The article includes a word use analysis of the political candidates that has some counterintuitive results, to my thinking: The Democrats' use is "concrete and restrained," whereas the Repuplicans' is "cheerfully unrestrained, future-focused." Very interesting. --Bill Paarlberg

Measuring the Transparency of Environmental Sustainability Reporting

Here is a link to the pdf of Measuring the Transparency of Environmental Sustainability Reporting by:

Brad Rawlins, Brigham Young University
Katie Paine, KDPaine & Partners, and
Peter Kowalski, KDPaine & Partners

Summary:

The websites of Fortune 50 companies were evaluated to determine the depth and detail of information presented on environmental sustainability efforts of the company. A summary measure, called the Environmental Reporting Transparency Scorecard, was used to rank the companies. Findings:

· Environmental sustainability reporting is still being used as a public relations tool to position companies favorably on environmental issues, rather than to hold companies accountable to stakeholders for environmental performance.

· The majority of the Fortune 50 are providing some kind of environmental information online, but there is room for improvement in the adoption of the interactive and integrated technology that is available for reporting and stakeholder participation within the Web medium. These technologies would enhance the transparency of the reporting process.

· There is little evidence of stakeholder involvement in the environmental sustainability reporting process. If companies are involving stakeholders in this process, the disclosure of such participation would improve the appearance of transparency.

· Involving third party groups that provide reporting standards helped organizations to be more transparent in their reporting process, as determined by the ERTS scorecard.

Measuring Reach on Radio

by Peter Kowalski

-- Editor's Note: The following was provided in response to a client's query, and it makes a good blog post, too.

Radio broadcast content remains among the most difficult to record, archive and retrieve, and of the traditional news media, is the least frequently (and accurately) monitored. Radio content is tracked using speech to text systems, which rely on voice recognition software that is often error prone. The sounds involved in the key phrase "Red Cross" are fairly unambiguous, and I would anticipate that your organization would have less trouble than something initial-based like the ASPCA, or something based on a proper name like Susan G. Komen for the Cure. As opposed to print or Internet monitoring, though, we would put very few restrictions on a radio broadcast search, instead relying on human readers to filter through segments for information that may or may not be pertinent to a particular investigation.

There are four major providers for radio broadcast content:

-- Cision tracks major markets and national news shows, but in my experience with using Cision in a limited capacity for a higher education client of ours, I would not recommend using their services for radio monitoring. Their accuracy is extremely low.

-- VMS tracks national networks (e.g. NPR, PRI, ABC Radio, Dow Jones Radio) and stations with news, talk, business and financial formats in the top 20 metros. They admit to the challenges inherent in radio monitoring, but an independent study found them to be 40% more accurate than Cision in retrieval.

-- National Aircheck is a niche company, and has monitoring specialists who record and monitor news, talk, business & financial radio across 2,500 stations (all major FM + almost all AM in the top 50 metros). As they specialize in radio, they have a large client roster that includes many of the largest public relations firms.

-- TVEyes is one of the newer entrants into this space, and tracks a handful of the national networks (Bloomberg, NPR, AP Radio, CBS News Radio).

Arbitron data is most commonly used to provide estimates for reach. In an integrated study, I tend to prefer Cume Persons, which is the average number of different people who listened to a station for a minimum of five minutes in a quarter hour. These persons are counted only once, no matter for how long they listen (so say, they listen for 45 minutes). This differs slightly from Average Quarter-Hour Persons, which is more useful for estimating reach in a rotating ad spots in a specific daypart or timeframe. Since we know when the news item aired, we would use Cume Persons, as it is comparable to other metrics like Unique Monthly Visitors (Web) or Impressions (Print). If you are new to Arbitron data, a handy reference guide can be found here. This data is often segmented within demographic groups, but the extent to which each provider can provide this level of detail would have to be more closely determined.

As with print and online data, radio reach data would be enriched with information about position within a particular show, dominance within a segment and other measures of prominence and dominance that are specific to radio and relate to how individuals remember aurally-received news.

Data providers are slowly getting to the point where segments can be listened to online, which is always preferable in analysis as tone of voice, pacing and similar factors can alter perceptions of information. It took TV and cable monitoring services some time to get to the streaming video point, and as streaming audio becomes more widely available, I consider it an asset to analysis.

Peter Kowalski is Director of Research Strategy at KDPaine & Partners. His research interests include international inter-media influence, agenda setting and network analysis. He holds a Bachelor of Arts in Public Relations from the USC Annenberg School of Journalism.

How Public Relations Measurement Can Win the War on Terror

It's not a new thing to talk about the war on terror as a war in the media for mindshare. But I've never seen it stated as explicitly as in the article "Marketing Osama" which appeared in a recent issue of The Week, and which was reprinted from a story by John Cook originally published in Radar. The point there is that Brand USA and Brand Osama are fighting it out in the media, and this marketing/PR battle is the important front in the war.

Public relations measurement evaluates exactly the sort of metrics that define the strategy that al Qaida has been using to enlarge its support: media impressions, media content, key messages, contributions and new members. Public relations should be not only measuring the war on terror, but, by virtue of tracking the most important metrics, should be defining the war on terror too. (See "Knowing the Enemy: Can social scientists redefine the "war on terror"?")

One of the more interesting aspects of the war on terror is that there is, (so far, that I know of) no way to know if we are winning or losing, or how we will ever know if we do win or lose. And that's where public relations measurement comes in: If we can use public relations metrics to define progress in the war, we can use them to define success and failure. Public relations measurement can win the war on terror by defining when the war will be over.

And here's how we do it: Let's say that al Qaida's media and marketing efforts now generate X impressions of their key messages each year, which in turn result in Y dollars of contributions and Z new members. We gather and track this data for the last ten or twenty years, and correlate it against terrorist activity of all sorts. (Yeah, it's a big job.) We then will understand what level of exposure to key messages (or Osama bin Laden videos) is required to generate what levels of recruits. Or to generate one new suicide bomber.

Then we define success in terms of the metrics. So, we could define "winning the war on terror" as when al-Qaida's impressions, contributions and recruits are dropping. And then we define victory as the point at which these metric fall low enough so that al Qaida can no longer function effectively. If Osama bin Laden doesn't get his message out, or if no one believes it enough to do anything, then we've won. --Bill Paarlberg

The Media Integrity Index

A call for weighting media coverage based on trust and credibility.

by Katie Delahaye Paine

In this article I suggest that media analysis programs would more accurately reflect articles' influence on readers if the integrity of the media outlet was taken into account. This could be achieved by developing a standard measure of media integrity which would used to rate different media outlets for different shareholder groups. The resulting Integrity Index would be used to weight coverage, thereby achieving a more accurate measure of media coverage's impact on consumers.

To understand why an Integrity Index is needed, consider:

Item #1
Why measure what no one believes?

Mazen Nawahi, in an impassioned speech at the 2008 Dubai Measurement Summit, raised the issue of journalistic integrity and the degree to which the integrity of a particular media outlet should be accounted for in any measurement program. Put another way, if:

1. Everyone knows that a given media outlet is going to print whatever a company sends it because they always do or because they're a major advertiser, and
2. Everyone also knows that they will never print stories that don't agree with the governing dogma,

then why would anyone believe anything that was printed?

And if all your media coverage in a particular outlet lacks credibility, why would you include it in your measurement results? For example, in 22 years in business I have yet to have a client want me to include The Weekly World News in its media list. So how do people justify measurement programs that include stories that do nothing to achieve their goals?

Item #2
Credibility is in the eye of the beholder.

Suppose a highly credible blogger gets into a mud-wrestling contest with the blogging equivalent of a pig. The contest attracts a lot of attention, a lot of arguments fly back and forth, and a lot of dirt gets thrown around. Now for the people that are into farm animals, the arguments of the pig might be very credible. However, people interested in learning something of value professionally will no doubt pay more attention to the words of the credible blogger. But the real question is: Do you weight coverage of them both equally in your media analysis reports?

How to correct for influence?

Taking examples of this sort into consideration, it is easy to see why some sort of authority or influence weighting of coverage would be of value. There has been a fair amount of research done in this area. For several years, Angie Jeffrey at VMS has been studying various weightings of media coverage to determine which factors most directly impact sales. Her research first found that share of discussion was effective, but she postulates that advertising rates are more telling because they reflect the importance of the various media. (See this paper and this paper.) I think it's a step in the right direction, but I generally frown on the use of advertising rates (and of AVEs), so I'm skeptical.

The Media Integrity Index: Why not rate coverage on integrity as rated by stakeholders?

I suggest that it's not the ad rate but the integrity of the publication that most determines consumer credibility and thus drives consumer action. And I'm suggesting that to rank a media outlet in importance, we should ask our stakeholders how they perceive the media outlet in terms of integrity.

I raised the question on Twitter and got some interesting responses:

• "It's all relative... What is good for one is bad for the other. It has to be specified for a purpose, customized."

• "Integrity or perceived integrity? Seems there's a difference there that's highly subjective." --Ryan Anderson

• "Integrity is the measure of perceived relationship (believability) of one-to-many listeners. So measure relationships. I would suggest looking at the way political integrity is measured. It would be a perceived metric, measured by polling audience. --Videodred

• "What's the goal? What's the client want it to say? In the case of journalists, what's the relationship?

• "Integrity has to be measured parallel to influence as audience determines credibility of source." --Mike Maney

So in one way, my proposed Integrity Index has to start with the goal of the coverage. What is it that the company or organization is trying to accomplish? Consider it this way: If you don't care who or what your brand is associated with, and you want exposure pure and simple, then you don't care about the integrity or credibility of whatever media outlet is talking about you. On the other hand, if you're trying to establish a reputation, or build brand loyalty and trust, or trying to reach an audience with certain media preferences, then the credibility of media to your audience should matter a great deal.

How to determine integrity?

So how do we determine which media is trustworthy and credible, and which journalists have or do not have integrity? The wrong way is to look at a list of media and assign weights or values to each one yourself. What you think doesn't matter. All that really matters is what your customers or stakeholders or members or constituents think. So you need to ask them. The best way to do that is to use the Grunig Relationship Instrument.

Now you probably don't need to ask them about all 150 publications on your media list. Remember: Never ask a question about something that can't be changed. If The New York Times is what your boss' boss reads every morning, there's no chance in hell you'll ever take it off you top tier media list. Even if your target audience is nine-year-old girls. So start with a list of suspect publications, blogs or any other type of media outlet.

Remember that different stakeholder groups may rate a given media outlet differently. As I pointed out above, integrity is not a fixed standard. For instance, we would expect Republican viewers to rate Fox News and The New York Times differently than Democratic viewers. It is conceivable that you will want to generate different Integrity Indexes for different stakeholder groups.

Below are some survey questions adapted from the Grunig Relationship Instrument. You may wish to develop your own, based on the specific components of relationships you wish to measure (refer to this paper). Ask your stakeholders whether they agree or disagree with each statement as it pertains to each media outlet, then use the responses to rate the outlets.

1. This media outlet treats people like me and organizations like mine fairly and justly.

2. Whenever this media outlet makes an important decision, I know it will be concerned about people like me.

3. This media outlet can be relied on to keep its promises.

4. I believe that this media outlet takes the opinions of people like me into account when making decisions.

5. I feel very confident about this media outlet's skills.

6. This media outlet has the ability to accomplish what it says it will do.

7. Sound principles seem to guide this media outlet's behavior.

8. This media outlet does not mislead people like me.

9. I think it is important to watch this media outlet closely.

10. This media outlet is known to be successful at the things it tries to do.

Industry-wide integrity standards?

Now, the logical question is, "Why aren't we doing this as an industry?" Shouldn't we be factoring in credibility based on some industry-wide standard? It certainly is a more accurate weighting factor than simple eyeballs or ad cost. But the reality is that your stakeholders aren't going to be identical to my stakeholders, and what matters is how a significantly valid sample of your stakeholders feel. It would be awfully complex and difficult to set industry-wide standards for many different stakeholder groups.

On the other hand, if the PR industry wanted to take on a project to try and accurately weight publications based on their integrity, the world would most definitely be a better place. And it is not far-fetched to imagine an industry-standard Integrity Scale or Survey that could be used to determine the Integrity Index for different stakeholder groups.

Cubic Applications Earns Platinum Research Award For Media Analysis

We are proud to note that Mike Daily (a recipient of The Measurement Standard's coveted Measurement Maven of the Month Award) and the company he works for, Cubic Applications, Inc., has won a Platinum Award in the 2007 MarCom Creative Awards. Well done, Mike and Co.! Read more here.

Measure the Media that Are Most Important to Your Customers

The Paine of Measurement

 

 

Make Good Choices
No one says you have to measure every medium on the planet.

When my cousin Caleb was into his "terrible twos" and heading for trouble, his father, who stands an imposing 6 feet 3 inches tall, would look down at him and say in his deep baritone: "CALEB, MAKE GOOD CHOICES." Invariably, Caleb would stop what he was doing, sit down and think for a minute, and then modify his behavior accordingly.

The lesson seems to have taken. Caleb is now taller than his father, and a true teenage heartthrob. With very few exceptions, he makes great choices.

Oh, that PR people had such good survival skills. Sadly, they tend to keep doing the bad stuff rather than embracing the good. Four years ago, a member of the audience at a conference on new media raised his hand and said, "l don't have time to deal with all this on-line stuff, I'm too busy as it is."

I wonder how many PR people in the 30's and 40's said the same thing about radio and television. "Woe is me, one more thing I have to learn how to do." Rather than embrace the new, PR people tend to look at new media as yet another burden to be borne. And once they start measuring and tracking one media, they find it extraordinarily hard to embrace a new one. In the last month, at conference after conference, people have whined about how hard it is to deal with social media, and why they don't have the time or resources to measure it.

My answer is: Make Good Choices. No one says you have to measure every medium on the planet. Just because it's new, it doesn't mean you have to monitor it. You can and should choose to concentrate on, and measure, the media that are most important to your customers. If they're on Facebook and YouTube, you better be monitoring them. If they're sharing ideas on Twitter and photos on Flickr, it would behoove you to sign up and listen in. On the other hand, if they're still reading hard copies of trade magazines, then that may be all you need to monitor.

Better still, measure the impact on the customer directly. As Jim Macnamara writes, the average person sees, consumes and creates dozens of different media every day. If you have an unlimited budget, you measure all of those different media. But it's so much simpler to just measure the behavior of the consumer instead. Ask them how they perceive their relationship with your organization. Follow their behavior on your website. Watch and see how engaged they are with your blog. That's all the measure you need.

As long as PR people obsess about tracking column inches and "hits," of course they are going to be overwhelmed with the possibilities and the burden of measuring them all. But if they focus on the outcome – the actual customer behavior – chances are they'll survive and thrive in this new hyper-media world.

Public Relations Measurement Technique Sheds Light on Companies' Environmental Reputations

Jim Macnamara's 'Measuring Up'

 

Media Analysis Shows Companies Need to Set
and Achieve Measurable
Climate Change Goals
Environmental performance will be a key driver of reputation.

A major new study of public debate and discussion on energy completed by my company, Media Monitors – CARMA Asia Pacific, presents some important warnings and advice for companies, organizations and governments and concludes that environmental performance is likely to increasingly be a key driver of reputation. We used content analysis to examine statements and commentary by organizations, spokespersons and consumers in leading mainstream media and blogs in Australia and major Asian countries including China, Hong Kong and Singapore.

The analysis covered discussions of nuclear, coal, wind, solar and other alternative sources appearing in more than 1,500 print news articles, radio and TV program segments and 170 blogs appearing between May and July 2007. It focused on Asia Pacific because China is fast becoming the world's leading polluter. Australia is the world's largest exporter of coal and uranium, and 16 of 29 nuclear power plants under construction or proposed worldwide are due to come into operation in Asia in the next 10-15 years.

Overall Results:
Companies must achieve measurable targets on the environment or their reputations will suffer.

The report found mild optimism that solutions will be found to balance environmental and economic interests, although serious fears and concerns also abound in relation to safety, the environment and consumer protection.

Specifically of interest to companies and organizations is a finding that they face widespread criticism if they do not implement meaningful measures to reduce carbon emissions. Environmental performance is likely to increasingly be a key driver of reputation, the report says.

The report concluded that companies adopting policies and planning measures to address climate change need to set and achieve significant measurable targets. "Broad unspecific policies and 'aspirational goals' are not going to cut it and may backfire leading to public criticism. There are signs of this already happening," the report warns.

Also:

• Positive messages outweighed negative messages about energy production and use – albeit the margin was narrow. (The analysis examined discussion and comment from the perspective of a balanced position between protecting the environment and maintaining economic development and standards of living.)
• Nuclear energy is the most discussed form of energy and is mired in controversy and near evenly divided opinion for and against, the analysis found. Most opposition and concerns in relation to nuclear power are about safety.
• Solar and wind power are being actively promoted by environmentalists, but are described in most discussions as not being able to make a substantial contribution to energy needs.

Carbon Trading:
Despite a number of initiatives, governments are generally not seen to be doing enough.

Carbon trading is one initiative that is being welcomed by a majority of spokespersons and commentators and is set to become a multi-billion dollar industry over the next few years. However, some are warning that it is still uncertain whether carbon trading will stimulate a net reduction in carbon emissions. Some energy producers and consumer groups have pointed to lessons in Europe where too many permits were issued, resulting in a price collapse below levels necessary to stimulate investment in carbon reducing initiatives.

Also, concerns are being expressed that carbon trading is insufficiently regulated and could lead to scams and fraudulent schemes. One leading environment group has warned that some companies are selling trees without accreditation and that their claims of carbon credits or neutrality are not substantiated. Consumer groups are also expressing concern that consumers may be paying levies and surcharges with no guarantee that their money will be used to address climate change.

The report says:

"While the efforts of some companies to reduce their carbon footprint may be commendable and genuine, there is considerable concern over a lack of safeguards for consumers on how extra charges will be used and whether they will reduce carbon emissions... With carbon trading set to become the next big thing in the financial world, strict regulation of the carbon trading industry, including accreditation and monitoring of carbon offset providers, is seen as necessary to avoid scams and corruption... and to ensure that offsets are real and effective."

Clean coal technology (CCT) is also creating some optimism that a balance can be achieved between reducing greenhouse gases and continuing to exploit vast coal reserves. However, environmental groups and some commentators are cynical, pointing out that the technology is still unproven.

Blog Analysis

Blogs were found to be more unfavorable than mainstream media coverage (48% unfavorable compared with 30% of mainstream media content unfavorable). However, 30% of blog discussion was favorable. While much of this was supportive of renewable forms of energy, some bloggers support clean coal technology and carbon trading – albeit with the same concerns as mainstream media commentators and spokespersons.

Analysis also showed that bloggers often include scientists and technical experts and, therefore, blogs need to be recognized as an increasingly important medium, reflecting the viewpoints of influencers and thought-leaders.

Dr. Jim Macnamara BA, MA, PhD is Group Research Director with Media Monitors – CARMA Asia Pacific which conducts media monitoring, media analysis, e-surveys, reputation research and sponsorship analysis. He has a 30-year career in the media and PR in Asia Pacific and is the author of 11 books on media and communication. In August 2007 he was appointed Professor of Public Communication at the University of Technology, Sydney.