The Measurement Standard: Blog Edition

A Powerful New Set of Public Relations Measurement Standards

This time they're based on business outcomes.

by Katie Delahaye Paine

Last week PRSA announced a call for industry agreement on standards for public relations measurement. A working group, chaired by David Rockland and made up of former or current chairs of the IPR Measurement Commission (Pauline Draper-Watts, Mark Weiner, Don Wright, and myself) has put together a set of recommended metrics and approaches for evaluating PR's influence on business outcomes. (Notice I said "outcomes," we'll talk more about those in a minute.)

You can view our actual recommendations in the slide show below. Or visit the PRSA's ComPRhension blog, where you can read David's post and give us your comments.

Before I list the actual outcome measures recommended, here's some background on recent efforts to develop public relations measurement standards, and a glimpse into why this new effort is more useful and powerful than those in the past.

Guidelines, standards: We've been there before.

Over the past fifteen years or so, I've been personally associated with every serious effort to develop public relations measurement standards. Back in 1995, Jack Felton got a group of us measurement geeks together at a PRSA conference to talk about establishing some kind of measurement standards.

The result was the formation in 1996 of a "Special Commission on Measurement" under the auspices of the IPR. Together we wrote the first IPR white paper (offically authored by Walt Lindemann), "Guidelines and Standards for Measuring the Effectiveness of PR Programs and Activities." You can download it (and/or a somewhat newer and revised version of it) from the IPR's website. At about the same time, the Brits and the Europeans were trying to tackle this problem, and Walt and I went to Europe to be part of their standard-setting confab.

Since that time, various other groups, like CPRS (with whom I consulted) and AMEC, have attempted to set more formal standards, but none gained significant traction. So, despite a decade and a half of serious and well-intentioned efforts, no official, industry-wide standards have resulted.

This time, however, things are a little different, and the new PRSA effort holds great promise.

Eyeballs or outcomes?

Back then, what people wanted public relations measurement to provide was a "GRP" of PR – a way to quantify success as conveniently as the GRP measure used by advertising. But GRPs are only a count of eyeballs, and not every PR campaign is designed to attract eyeballs. In fact, just about every PR program ever launched has a unique set of goals, and thus, a different set of measures of success. Thus, there never can be a single metric that applies to all PR.

Now we have PRSA's most recent effort, and I think this time we'll have some standards that people will agree upon and use. Why? Because this time the standards are based on business outcomes -- metrics that the C-suite can understand and respect.

When CEOs and the rest of the C-suite look to marketing and PR for measures of their success, they expect to be presented with business-oriented metrics -- not AVEs, or clip counts, or even impressions. All those output measures are relatively easy to put together, but are really only useful as a way of describing activity. They are only surrogates for the outcome metrics we wish we had, the real revenue-related numbers like sales or cost savings.

Why outcomes now?

Of course, many of us have been pushing the value of outcome measures for years (myself included). Why have we now, at last, built outcome measures into industry recommendations? It's because PR is now under tremendous pressure to up its game:

• When money is tight, the fat gets trimmed. Never has it been more important to demonstrate the business value that PR brings to the table. If you can't show how your work contributes to the bottom line, chances are you'll have a smaller budget going forward, and you probably won't have a job much longer either.
• There are more alternatives to traditional PR tactics than ever before-- and they come with powerful new metrics. Reaching out to reporters in a traditional sense is now only the start of a good campaign. You need to bring in Twitter, Facebook, a blog or two, and probably some videos as well. And the outcomes for those social media are far more measurable than for traditional PR. They are not impressions, but rather, engagement – measured by comments, clickthroughs, downloads, retweets, votes, ratings, and all the other new metrics that social media is bringing to the table. And, believe me, the hungry new social media industry is not waiting around for PR people to catch up. Just ask anyone in the newspaper business.

So what exactly is it that PRSA is offering for standards?

Below is the presentation developed by our Working Group. It's not just a list of recommendations, rather it's really a philosophy that PR practitioners can and should take to the C-suite to explain exactly how PR can help the bottom line. It's also channel- and industry-neutral; designed to work whether you're measuring social or traditional media. And equally applicable whether you're working for a small local nonprofit or a Fortune 500 corporation.

Documenting The Business Outcomes

View more presentations from Public Relations Society of America .

 

If the above slide show doesn't work, visit the PRSA's ComPRhension blog, where you can see it and leave your comments.

The recommendations are summarized in the table below. They divide the business outcomes of PR into four basic categories:

1. Financial
2. Reputation
3. Employee
4. Public Policy

and offer measurement guidelines for outcomes related to each category. Again we are very eager to hear your comments, so please visit the PRSA blog and let us know what you think.

	Business Outcome	Measurement Method
Financial Outcomes	Marketing Public Relations drives sales Investor Public Relations drives investment Public Relations drives donations and membership for relevant organizations	Consumer Response: Field survey of consumers, determine purchase levels and exposure to PR results, isolate causal effects through statistical analysis Market Mix Modeling / Econometric Modeling: Gather PR output/outcome data in conjunction with other marketing activity by market, by marketing function, by region, by time period and factor by revenue-generation by market, by region and over time. Apply regression analysis.
	Improves efficiency by better audience targeting PR reaches more people with a credible message for less money	Determine comparative cost of different communication approaches; calculate percent of target reached; determine change in purchase cycle resultant from PR activity.
	Avoids catastrophic cost Mitigates impacts of crises	Assess competitors and peers who may have faced similar crises, track emergence of their crises and impact on sales, stock price, and relevant business measures to evaluate the potential impact that was avoided.
Reputation	Increases likelihood to purchase / consider your brand(s) Minimizes the effects of a crisis Reinforces communication of organizational values Rebuilds trust after a crisis Establishes credibility of new products / companies; ease of market entry Commanding higher prices, lower costs, premium on stock price Enhances recommendations / word of mouth leading to faster adoption Increases customer loyalty / renewals / satisfaction Improves the attracting / retaining of talent Lowers legal costs	Benchmark reputation / relationship metrics via survey prior to a campaign, repeat every 3 - 6 months Correlate attitudinal studies with customer purchase attitude and behavior Map conversations (and tone) in traditional and social media to web analytic data, e.g., registrations, requests for information, sales leads, etc. Map conversations / reputation to financial analysts opinions and stock price volatility Correlate share of thought leadership visibility to adoption of policy positions
Employee	Increases employee satisfaction and engagement, leading to greater efficiency, increased retention, lower turnover rates, lower recruitment costs, and higher productivity Lowers legal costs Change employee behaviors such as greater levels of focus on key areas such as safety, quality, call response times Provides greater transparency and commitment to and from employees Creates a platform should it be necessary to communicate bad news at some stage in the future Note: Items here can also refer to other internal publics such as trade association members.	Use control groups and compare to employee populations exposed to PR activities Focus on performance outcomes not attitude or awareness Match / correlate messaging data to: Employee satisfaction and engagement findings Employee turnover statistics and other recruitment data Call response times Customer experience surveys Consider other research tools and data – focus groups, exit interview data, days of sickness, etc. Note: items here can also refer to other internal publics such as trade association members.
Public Policy	Creates public awareness, understanding, and support for legislation, regulation, and political candidates Affects voter behavior Helps pass legislation, regulation, and initiatives Affects specific companies and industries through appropriations, tax impacts, and regulatory changes that can affect any and all aspects of a business Instigates and perpetuates grassroots or grasstops campaigns	Use available public tracking services at national (e.g., major network polls) or local (e.g., university polling centers) to track changes in awareness, understanding, support, and voter intent. Also where possible link to level of PR activity. Conduct tracking survey of key politicians or regulators. Can often use Influentials’ awareness as a proxy for elected officials, as well as to measure the “edge” of a trend. Post-election surveys can isolate specific effects of PR by determining actual voting behavior, as well as levels of exposure to different communication media. Actual public or legislator voting behavior.

The Single PR Metric and Flying Pigs

Jim Macnamara's "Measuring Up"

A series of recent blog posts got me going because they raised – yet again – the hoary old goat of a single PR metric. Or perhaps that should be the hoary old flying pig of PR metrics. If ever there was a creature of folklore and fantasy, this is it. But, rather than being a harmless creature of myth, the single metric of public relations continues to divert the industry from meaningful progress. Its fruitless pursuit soaks up a lot of effort and time that could be productively devoted to getting on and measuring PR.

Despite being disavowed by many leading thinkers in the field, "for many in public relations measurement, the Holy Grail is a single, powerful metric of success," as Don Bartholomew wrote early this year in his MetricsMan blog.

What is intriguing about this ongoing debate and fruitless search is that even a cursory glance at other closely related fields of practice show that the chances of finding a single metric are about the same as finding pigs that fly. For instance, there is no single metric used in market research. Imagine trying to reduce all the various things that market researchers need to measure to a single metric.

Similarly, in social research there is no single metric. Even the highly developed media ratings business has a range of metrics for various media including Target Audience Ratings Points (TARPs), Cost Per Rating Point (CPRP), Cost Per Thousand Members of an audience (CPM), recall, Opportunities to See (OTS), visits, Unique Visitors, duration, return visits, click throughs, and so on. Researchers in other sectors of marketing and corporate communication use a range of methods including surveys, response mechanisms, diaries, peoplemeters, internet logfiles, tags, etcetera.

Why does the public relations industry think that there could be a single metric for expressing the value and contribution of the diverse range of practices deployed under the broad heading of "public relations" ranging from analyst relations to social media? And why does the industry beat itself up trying to find such a single PR metric?

What is recognized as important in other research and media metrics fields is that basic standards are important. These do not specify a particular metric or a particular measurement product, but they establish a framework of what types of things are worth measuring and what methods are reliable in doing so. The Institute for Public Relations in the U.S. and a number of other bodies have, to a significant extent, established standards – although these could be more formalized and simplified into a "how to" and "how not to" guide for practitioners.

From a review of PR measurement literature in both academic and industry journals, Anne Gregory and Tom Watson in the UK concluded in a late 2008 paper published in the Journal of Marketing Communications that there is a substantial gap between what researchers and PR scholars advocate and what practitioners do, and that this gap needs to be closed. Gregory and Watson also concluded that the answer is not a single metric. To the contrary, they concluded that a range of measures are available and need to be more widely promoted and used. For instance, they suggested that practitioner-developed dashboards that collect and display a range of metrics together – both informal and formal – need to be given greater attention, rather than hierarchic models which suggest optimum outcome level measurement.

As the developer of one of those hierarchic models (the Pyramid Model of PR Research), I must point out that it would be a misinterpretation to conclude that such approaches advocate any single measurement. Rather, they emphasize the importance of recognizing outcomes as a more advanced level of measurement than outputs, and in fact the Pyramid Model lists more than 20 metrics available for various PR activities. However, Gregory and Watson have a good point in emphasizing the need to look at practical considerations as well as normative ideals in measurement.

While that does not mean abandoning standards and producing metrics that are questionable (such as so-called Advertising Value Equivalents which are a measure of cost, not value, and do not apply to many online sites) a grounded approach of what is available and doable will always have to be integrated with Best Practice high level metrics when these can be generated. Metrics reported also have to be geared to the expectations and needs of senior management. 

Also, there are significant differences in requirements for evaluating an internal newsletter compared with a multi-million dollar campaign. And measurement must be related to objectives which vary from increasing awareness to creating behavioral change. That means multiple metrics are inevitable, necessary and sensible.

Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC became Professor of Public Communication at the University of Technology Sydney in 2007 after a 30-year career working in journalism, public relations and media research which culminated in selling the CARMA Asia Pacific franchise which he founded to Media Monitors in 2006. He worked as Group Research Director with Media Monitors - CARMA Asia Pacific following the sale and continues as a Consultant with the Group.

NYTimes Article on Web Stats: Is Web Measurement Technology Stymied by Human Beings Just Being Human?

Don't miss this NYTimes article in today's paper that nicely sums up the on-going Web visitor counting controversy (at least from the public's view). Makes me wonder about the reputation of public relations and of public relations measurement: Here we are congratulating ourselves on being able to measure relationships, and being able to measure trust and transparency and various other esoteric things, but we can't nail down something simple like measurement for Web visits.

What's the problem here? It's not lack of data, there's piles of that. And you'd think that there would be plenty of motivation, what with all the money hanging in the balance. Is it as basic as a battle for business dominance; human beings just being human? "My number's better than your's!"

Maybe the real measurement problem lies someplace other than in the Web traffic reports--like between the ears of the people who use them. Geeze, this relationship measurement stuff might come in handy. (And, on a completely different note, how about those Red Sox, eh?) --Bill Paarlberg

PR Measurement Must Trust But Verify: Today it's comScore and Nielsen//NetRatings, tomorrow it's you.

We all know there are as many ways to fudge the numbers as there are measurement programs. Bias, whether conscious or unconscious, can and does creep into even the most careful research. Here at The Measurement Standard, we're certain that the PR profession will jump light-years ahead when everybody's measurement results are verifiable by some third party or technique.

So it is nice to see that the Internet Advertising Bureau (IAB) is requesting that comScore and Nielsen//NetRatings submit to a third-party audit of their web measurement processes. Although ONLINE Media Daily referred to this as "IAB Declares War On Web Measurement Firms," we see it as good news. It's about time the measurement profession begins to face up to the multitude of research sins it constantly condones. Oh, what, so your boss trusts the results when you do your own measurement of your own PR programs? Well he or she shouldn't. And neither should you.
--Bill Paarlberg

Measurement Standards: Why Can't We Decide?

This article was published in our Feb 8 issue
THE PAINE OF MEASUREMENT
Measurement Standards: Why Can't We Decide?
We don't have standard measures because there are no standard PR programs.
We in the PR world have been jabbering on about needing measurement standards for at least the last twenty years. Despite the best efforts of the Canadian PR Society and others, we're no closer to achieving that goal...