The Measurement Standard: Blog Edition

Where Did AVES Come From?

A new paper by Professor Tom Watson presents the history of Advertising Value Equivalency, the discredited-but-still-common public relations measurement technique that the PR world is now in the process of rejecting. Read a summary of the paper, "Advertising Value Equivalence – PR’s orphan metric," on the IPR website, or download it here (fee required).

Don't miss the last couple paragraphs of the summary, in which Watson relates an warning against the use of AVEs from 1949: "This practice has done more to undermine public relations than any other." Today, 64 years later, the demand for AVEs is still strong. Which says a lot about the demand for quick and inexpensive measurement. And, perhaps more importantly, about the unfortunate tendency to view PR as simply a way to get press.  --Bill Paarlberg, editor (Thanks to IPR for the image.)

Our Measurement Mavens of the Month, Part 2: Traveling Beyond AVEs

by Katie Delahaye Paine

As I’ve said many times, the only thing I want on my tombstone are the words: "She killed AVEs." And if the grandchildren of today’s PR professionals happen to wander past my grave and actually read those words, then it will be partly due to the efforts of three pioneering organizations in the travel and tourism space.

To call my recent experiences with these clients revolutionary is a serious understatement. Over the moon is more appropriate. So I am presenting them with an unprecedented second-time-in-the-month Measurement Maven award. (See the first Maven award, already this month.)

Here's Why They're Selling You AVEs: The Biggest Lie In Public Relations

by Diane Lennox

Kudos to K.C. Brown, Cision’s analytics geek, for a compelling alternative argument against Advertising Value Equivalency (AVE). In this old, but not outdated video rant, K.C. explains why agencies continue to push this disproven measure of so-called success – sometimes at the same time they disclaim it.

Because it’s worth a lot of money to them.

It puts an arbitrary – and inflated – metric on their efforts that helps justify the cost of their services.

Barcelona Principles Report Card: How Do 12 PR Awards Programs Measure Up?

by Katie Delahaye Paine

Back in 1999, the IPR Measurement Commission began an initiative to improve the measurement criteria for PR industry awards.

CIPR to Use Multiplier of Zero in Evaluating Award Entries that Use AVEs

The Chartered Institute of Public Relations (CIPR) has announced that any entry in its 29th CIPR Excellence Awards that uses advertising value equivalency (AVE) will be given a score of zero for research, planning, measurement, and evaluation. [sound of enthusiastic applause]

New Hampshire Grand Gets A Measurement Makeover

The real-life lesson of how a tourism organization can measure the ROI of their communications -- without using Advertising Value Equivalency (AVE).

by Katie Delahaye Paine

Normally we would have named New Hampshire Grand a Measurement Menace of the Month, but we decided that it would be a much more useful exercise to give them a Measurement Makeover. In the future we’ll be combining the menace with a makeover so watch these pages for more examples. I welcome your submissions.

This month's Measurement Makeover features New Hampshire Grand. New Hampshire Grand is an organization that promotes tourism in the North Country of the state of New Hampshire. And, since I was intimately involved with its creation, I was appalled when I read this statement in New Hampshire Grand’s recent progress report:

“In 2010, we... published over 369 stories with an earned media value of $512,000. Earned media is the value of our exposure if we had to pay for each column inch of the article.”

"Earned media value" is another name for Advertising Value Equivalence (also known as AVE), an inaccurate and discredited technique for evaluating media coverage. We immediately contacted Montagne Communications, the PR firm for New Hampshire Grand, for more details, but have not heard from them. Nevertheless, thanks to New Hampshire's tourism recordkeeping (see below) we can help New Hampshire Grand express the value of its efforts in a more accurate and accepted fashion. So let's give it a Measurement Makeover. Here goes...

Step #1: Resources Needed

1. Excel spreadsheet. 

2. Data establishing relationship between impressions and actual tourist visits.

Step #2: Initial Data and Assumptions

It turns out all the data necessary is publicly available from the www.nhgrand.com and visitnh.gov websites. We have very useful historical data to work with, because the State of New Hampshire keeps good records on the value of its tourism efforts. They know that for every 100 positive tourism impressions they make (or buy) in the media, 3 people will visit the state as tourists. They also know that the average tourist spends about $82 dollars per visit in a combination of taxes and purchases.

Because New Hampshire Grand deals with the North Country specifically, and not the entire state, those averages need to be adjusted:

• The North Country is harder to get to than the rest of the state, and has fewer hotels and tourism resources. So let's assume that for every 100 favorable impressions only one person visits the state.
• Let's also recognize that prices in the North Country are typically half of what they are in the rest of the state. So let's assume that every one of those potential visitors spends $41, instead of the statewide average of $82.

We can use these numbers to express the value of New Hampshire Grand's efforts in tourist dollars.

We also know that the New Hampshire Grand effort has cost about $1 million so far. We can use that figure to calculate an approximate Return on Investment for the project so far.

Step #3: Collect Some Data

We reviewed the monthly newsletter reports of Montagne that were posted to the New Hampshire Grand website and counted the total number of positive impressions. Since all the stories posted were favorable, we took the reported impressions as our total number of favorable impressions. That number is 5,099,541 impressions. (Montagne also reported a total Unique Visitor count for their efforts of 342 million. This number seems vastly overstated and unreliable, as it exceeds the total population of the U.S. by about 10%. So we did not use it.)

Step #4: Calculate Potential Value of Media Coverage

Using our estimate that every 100 impressions results in one visitor, that gets us to a potential visitor count of 50,995.

And if each of those 50,995 spend $41, that makes a potential spend of about $2 million. Thus the value of New Hampshire Grand's communications can be estimated as $2 million in revenue from tourist visits.

Step #5: Estimate ROI

The formula for ROI is Revenue minus Investment divided by Investment. Revenue in this case is what the potential tourists spend, or $2 million. Investment is about $1 million (which is what the New Hampshire Grand effort has cost thus far). So the ROI estimate would be $2,000,000 - $1,000.000/$1,000,000 or 100%.

Step #6: The Big Reveal

 $2 million in tourist spending and an ROI of 100% sound a whole lot better -- and are far more accurate -- than $512,000 in fake Advertising Value Equivalency (AVE) numbers. Congrats, New Hampshire Grand, on your Measurement Makeover.

(Tip of the Hat to Chris Jensen at New Hampshire Public Radio who’s report on the need for New Hampshire Grand to demonstrate its value alerted us to an organization in need of a measurement makeover.)

(Transparency Alert: NCIC, which provides funding to New Hampshire Grand, is a former client of KDPaine & Partners. We conducted research for NCIC and were part of the discussion prior to the creation of New Hampshire Grand. We love what NCIC and The Tillotson Foundation have done and are on record as supporting New Hampshire Grand.)

###

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.

The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, CEO of KDPaine & Partners, will be glad to talk with you about measurement for your organization.

 KDPaine & Partners' Facebook Page.

David Rockland Sums Up the State of Measurement

David Rockland’s post “PR measurement beyond the Barcelona Principles” is a succinct and informative summary of recent public relations measurement progress. It’s as elegant a statement of the state of the measurement industry as you will find. If you aren't quite sure what happened at the Barcelona and Lisbon summits, go read his post right now. (And—I hate to say it, but...—don’t bother with The Measurement Standard’s substantial coverage, like here and here, unless you really want the details.)

David makes an excellent point about the brouhaha over Advertising Value Equivalency (AVE): It's not really about AVEs, it’s about what comes next. It’s about the much bigger picture of how to do public relations measurement correctly, using “...the tools and methods we need to demonstrate how we drive business for our clients and within our organizations.” Well said. --WTP

###

--Bill Paarlberg is editor of  The Measurement Standard blog and newsletter, and of Katie Paine's new book Measure What Matters. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

Shine Communications Gutless On AVEs

Yesterday, PRWeek made a big deal about Shine Communications becoming ...“the latest agency to scrap AVES [Advertising Value Equivalency].” And we here, who have been arguing against AVEs since the last century, were all set to award Shine a glossy gold star. (As we did last month for Ogilvy, and just a few days ago for Worldcom, when they swore off the discredited public relations measurement technique.)

But No! Read the Fine Print

What’s this? Shine's heroic stance loses its luster a little later in the article:

“Shine joint MD Richard Brett said... Shine would advise all its clients to migrate away from output and AVE measures. However, he conceded that Shine would continue to use the old system if clients 'insisted on it'.”

What??!!

So Shine’s anti-AVE stance amounts to “We promise not to use AVEs, as long as it’s OK with the client.” A shining example of gutlessness. --WTP

###

--Bill Paarlberg is editor of  The Measurement Standard blog and newsletter, and of Katie Paine's new book Measure What Matters. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

 

Another Victory in the War on AVE: Worldcom Says No to AVEs

ZAP! POP! KABAM! Take that, AVEs!

 For years this blog and its previous newsletter incarnations have been ranting against the use of Advertising Value Equivalency (AVE) to measure the effectiveness of public relations. Of late, thanks largely to events at the Barcelona and Lisbon Summits on Measurement, the tide is turning. More and more organizations and agencies are swearing off AVE.

So, it gives us great pleasure to note that Worldcom Public Relations Group, the world’s leading partnership of independently owned public relations counseling firms, has announced that it has adopted the Barcelona Declaration of Measurement Principles as guidelines for performance metrics. That means, in part, no AVE. Read more from Worldcom here.
--WTP

###

Dispatch from the War on AVE: Ogilvy PR Drops AVEs

Last week, Ogilvy PR announced it will stop using the repeatedly-denounced measurement technique Advertising Value Equivalency. Measurement pros have praised the move. Wrapping itself in a cloak of “Spin is Dead” and a promise to roll out new measurement techniques, the agency appears to have successfully directed attention away from why it was using the long-known-to-be-bogus technique in the first place.

AVEs have been denounced by all major industry organizations, most recently at the Lisbon European Summit on Measurement. Now that Ogilvy is first out of the gate, we predict that the rush will be on for more agencies to breathlessly announce they are done with AVEs for good. The longer they wait -- and yes we are talking to you, Ketchum, Kantar, and Hill & Knowlton -- the less credit they will get for taking a stand. (We single out those three agencies because they all offer AVE measurement, and they all had representation at the Lisbon Summit.)

As readers of this blog know, our publisher Katie Delahaye Paine has been a long-time critic of AVEs. (Last year she advocated a boycott of firms who use them.) It's nice to know her crying in the wilderness has at last been heeded. (That's Katie's face in the mash-up image to the right, but otherwise the illustration is inaccurate. As far as I know, she has never worn a watch like that.)

--Bill Paarlberg is editor of  The Measurement Standard blog and newsletter, and of Katie Paine's new book Measure What Matters. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

AVEs Receive $10 Billion in Media Coverage, ROI of 54,321%

For Immediate Release: The Advertising Value Equivalency (AVE) Promotion Council has announced that recent coverage of its Advertising Value Equivalency public relations measurement technique amounts to over $10 billion in media value. This figure takes into account recent voluminous condemnatory coverage in conferences, social media, and mainstream media.

The $10 billion figure was determined by adding up what it would cost to sponsor major denunciation at both the 2010 Barcelona and 2011 Lisbon Summits on Measurement, to buy several hundred inches of castigation in blog posts by prominent public relations experts, and to purchase 15 1/2" of advertising space adjacent to Carl Bialik’s Wall Street Journal column “Publicists Pump Up Value of Buzz; Don't Believe the Hype.”

This sum was then multiplied by 7.4, a generally accepted advertising-to-PR conversion factor. Based on this total, the Promotional Council calculates a 54,321% ROI on its recent program costs.

For more information on AVEs, contact any of the major measurement firms now offering them to their clients, including Kantar, Ketchum, Hill & Knowlton, MediaMonitors, VMS, BurrellesLuce, and Cision. --WTP

###

--Bill Paarlberg is editor of  The Measurement Standard blog and newsletter, and of Katie Paine's new book Measure What Matters. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

Why Are Sex in the City and Ragan’s PR Daily Alike?

Sex in the City and Ragan’s PR Daily receive our Measurement Menace of the Month Award for promoting make-believe public relations.

I read a disturbing post the other day on the Samantha Effect. It suggests that a surge in applications for PR programs in higher education is driven by interest in the glamorous life of Samantha Jones, a character on Sex in the City.

Thousands of the show’s young fans have been Carried away by the alluring yet utterly fictitious career of Samantha, who owns her own PR agency. To them, the PR profession appears to be a series of fun parties and events, requiring only a pair of Jimmy Choo heels, a little black dress, and a “people personality” for success. 

How do you measure that? AVEs, according to Ragan’s PR Daily.

I’ve never met Jessica Epperly, but judging from her recent piece in Ragan’s PR Daily, her role model is Samantha Jones. “Advertising vs. PR: How to measure the value of editorial coverage” has no more credibility than Samantha’s career. 

Need more proof? The very first comment on Jessica’s piece was from Lauren, a journalism student at the University of Missouri, who says, “This article has taught me more than most of my professors.”  Shame on those professors if a publicist’s puff piece full of inaccurate information can replace classes at a leading educational institution.

What is up with Ragan running such a piece? You know and I know that AVEs are bad, and that every reputable organization, PR firm, and measurement provider has signed onto the Barcelona Principles (which discredit the use and practice of comparing PR to advertising). So why did Ragan run it? Don’t they vet anything?

In fairness to the folks at Ragan, the brouhaha that erupted in the comment section did prompt Ragan to solicit an oped from my colleague Shonali Burke who urged us to consign AVEs to the graveyard. (See also Chuck Hemann's response to that article, with many excellent comments. And if you’d like to learn more about AVEs, we have plenty of coverage here at The Measurement Standard. Start with “AVEs Are Porn for PR,” then see the list here.)

Finally, this issue raises an interesting question about the media that cover our industry: What is—and should be—the role of the media in the coverage of PR? Is it to follow the guidance of the leaders of the industry and help lead the profession to a higher plane? (As PR Week did when they banned the use of AVEs as a metric in their annual award submissions.) Or is it to serve the lowest common denominator by recommending a fictional measurement tool?

Our purpose here at The Measurement Standard is to encourage people to base decisions on reality. And so we award both Sex in the City and Ragan’s PR Daily our Menace of the Month Award, for promoting make-believe public relations.

###

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements.

Measurement Menaces of the Month: All of Australian PR

(...except Jim Macnamara and Rochelle Burbury)

In contrast to the IPR (who are our Measurement Mavens of the Month), it seems that the PR industry in Australia is heading in the opposite direction, essentially throwing up its hands and caving to the “I just need a number” crowd. Given that Australia has so many great educational institutions turning out more and more PR professionals every year, this is kind of scary if all of them believe that AVE is the only answer.  --KDP

--Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.  

(Eds. note: That article linked to above does say that the Public Relations Institute of Australia criticizes the use of AVEs. So maybe there are some folks down under who see the light.)

 

The Measurement Mavens of the Month: The IPR Measurement Commission

Last week the IPR Measurement Commission completed a task that many thought was impossible. As a body, they agreed to, approved, and published their final, definitive statement on the controversial subject of AVEs. The gist of it is that AVEs are bad -- very bad -- so don't use them:

The IPR Commission on Public Relations Measurement & Evaluation voted to reject the term, concept and practice of Advertising Value Equivalency (AVE)...

(Okay, in the interest of full disclosure, I’m a founding member of this body, but that’s not to say I haven’t had my disagreements with them over the years (just read the comments).)

First of all congrats and thanks are due to Pauline Draper who chaired the Commission during this difficult time, and to Brad L. Rawlins, Chair, Department of Communications, Brigham Young University, who chaired the task force on AVEs. And to all the other members of the task force:

• Toni Griffin, Director, Public Relations, MetLife;
• Rebecca Harris, Senior Communications Manager, General Motors;
• Fraser Likely, President, Likely Communication Strategies Ltd.;
• Tim Marklein, Executive Vice President, Measurement & Strategy, Weber Shandwick; 
• Mark Weiner, CEO, North America, PRIME Research.   

When you look at the members of the task force and the commission, as well as the folks behind the Barcelona Principles, you come away with the realization that -- while not all the industry is in agreement -- certainly the best and the brightest have achieved consensus: #AVEMUSTDIE. (Thank you Shonali Burke for that wonderful hashtag!) --KDP

--Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.  

 

AVEs Are Really Dead: The IPR Says So

Here at The Measurement Standard we have long argued against the use of AVEs, as has our publisher Katie Delahaye Paine. See these articles about AVEs (Advertising Value Equivalency) if you'd like to review the controversy. (Especially "AVEs Are Porn for PR.")

Now, after a year of deliberation, a task force of the Institute for Public Relations (IPR) has released a report denouncing the use of AVEs: "The IPR Commission on Public Relations Measurement & Evaluation voted to reject the term, concept and practice of Advertising Value Equivalency (AVE)..." 

You can read the entire AVE report over on Katie Paine's blog. And also read what she thinks about it.

I say it's great progress. But AVEs -- fast, cheap, and flashy as they are -- are still going to be sold and used by people who don't know any better. Only when fast and easy alternatives are available will AVEs cease to be a temptation. And for more on that see this summary of a session on AVE alternatives at the recent IPR Measurement Summit.

--Bill Paarlberg, Editor, The Measurement Standard
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. 

General Sentiment Tars BP with AVE Scare-Wash

 
Measurement Menace Crap of the Week

Somebody please tell General Sentiment that their AVE-based Media Value "measurement" product was voted off the island last week in Barcelona. (See the Barcelona research principles, global standards for public relations measurement.)

What's worse, General Sentiment is waving about their bogus estimate of BP's "brand value being lost" to goose the media into noticing them. Crap like this distorts reality, deludes the masses, and does nothing more than win publicity for its creators. It's obvious here that General Sentiment is juicing the BP brand scare to tout themselves.

The really sad thing is that it would be very interesting to track BP's brand value during their current crisis. But sludge like this from General Sentiment just muddies the water.  -- Bill Paarlberg, Editor

(Note: Katie Delahaye Paine (publisher of The Measurement Standard) and KDPaine & Partners (the company that writes my paycheck) offer public relations measurement and social media measurement services. Services that may complete with those of General Sentiment. If General Sentiment offers any valid measurement services.)

5 Things You Should Do With the 7 Barcelona Principles of Public Relations Measurement

 "We haven't raised the bar, we've raised the floor."

by Katie Delahaye Paine

Editor's note: Welcome to our Barcelona Summit coverage:

• For an overview of the Summit, and links to other articles, blogs and photos, see Everything You Need to Know about the Barcelona Summit. 

• For a practical guide to improving your public relations and social media measurement based on the Barcelona principles, see The Barcelona Principles Checklist. 
  •  
• For a discussion of the next steps after Barcelona, read Barcelona Was a Great First Step: Here Are 5 More Issues to Tackle.

At the recent Barcelona Summit, AMEC, IPR and CIPR presented seven basic principles of measurement. 200 or so delegates voted to approve them:

1. Goal setting and measurement are fundamental aspects of any PR programmes.
2. Media measurement requires quantity and quality – cuttings in themselves are not enough.
3. Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity.
4. Social media can and should be measured.
5. Measuring outcomes is preferred to measuring media results.
6. Business results can and should be measured where possible.
7. Transparency and Replicability are paramount to sound measurement.

So what? What does this have to do with me?

It’s a valid question. Many of these principles were probably, for readers of The Measurement Standard, clearly in the “duh!” category. One of my Twitter followers said “Great, it takes that much effort to do something we all learned at university.” He has a point, (although I for one, never studied PR research in college – it wasn’t even offered).

These principles are not a call to arms, and will hardly start a revolution, but they do create a benchmark. They tell the world that if you are a PR professional trying to measure results, this is where you have to start.

AVEs: Dead Again

Without a doubt the most controversial principle -- and the one that should start at least a few skirmishes if not a revolution -- is #3. It drives a wooden stake into the undead heart of Advertising Value Equivalency. And the fact that it passed at all (the vote was by no means unanimous, see the voting tallies here) is a sign of how far we’ve come in just a year. Remember that last June at the Berlin AMEC conference the vast majority of the crowd cheerfully raised its hands when asked if they provided AVEs for their clients. (Read what I had to say about that here and especially here.) This year, most of the same providers are no doubt still offering what I refer to as “the cocaine” of our industry. But at least they admit that it is a flawed metric and want to see it killed off. (Here is a flamboyant reason to say goodbye to AVEs: "General Sentiment Tars BP With AVE Scare-Wash.")

Outcomes over Outputs

The other significant element to the principles is that they make very clear that PR measurement is not just about counting clips. The fact that they clearly state that measuring outcomes is preferred over outputs is significant, because it may mean the beginning of some very significant changes for the people who attended, many of whom make a living selling clips. Not that people will ever stop wanting to know what the media in all its forms says about them, but it means that, increasingly, media analysis will be integrated with web analytics, CRM and Marketing Mix Modeling in order to quantify the value of a public relations effort.

Of Course We Can Measure Social Media

I was also happy to notice that the principles reiterate my assertion that “YES WE CAN Measure Social Media.” The silly notion that just because a conversation happens in a space that's different from what we’re used to makes it somehow “unmeasurable” has always been one of the more ridiculous myths of our time. Since the percentage of coverage appearing in “traditional” media (as opposed to online and social) has dropped from 95% to about 9% in the last three years, we need to change how we define media.

So here are five things you should do with these principles:

1. Read and think about them. Read the coverage on the IPR website and join the discussion. You can get full copies of what was passed in Barcelona at the AMEC or IPR websites. To comment on the Principles, write to [email protected] or [email protected].
2. If you are a member of an organization not represented in Barcelona, contact your organization’s leadership and get them to sign on to the principles – conspicuously absent were IABC, SNCR, IPRA, AEJMC. If you are a member, we need you on board.
3. Stop using AVEs. Explain to your AVE addicted clients and bosses that the all the major industry leaders have pledged to stop using them. So should you.
4. Don’t work with vendors who don’t adhere to these principles.
5. Spread the word amongst your colleagues about The Barcelona Principles and urge them to endorse them.

Continue the Discussion

Most importantly, these principles aren’t the end of this discussion. As someone said in Barcelona, "We haven’t raised the bar, we’ve raised the floor." (If anyone reading this knows who the person is who said this, please tell Katie so we can properly credit them.) The principles are a place to start, a platform from which we can have more discussions about the right ways and the wrong ways to measure PR.

Photo at top from Katie Delahaye Paine's Flickr set.

Measurement Menace of the Month is a Presenter Who Recommended AVEs

The Measurement Menace of the Month Award

Runners Up:
You Who Are Not Measuring

First we need to point to the 84% of people not measuring the ROI of their social media activities. This number is appalling. People who aren’t measuring their social media activities have no idea what is working or not working, so they have no clue as to where to invest more money, or where to stop wasting money. In this economy, this is incomprehensible. So all of you out there—you know who you are—read Jim Sterne's excellent book Social Media Metrics (read my review of Jim Sterne's book Social Media Metrics it here) and get with the program.

And the Winner Is:
You Who Are Still Using AVEs!

The Measurement Menace that sent me over the top this month is a particular presenter who made me feel like all my efforts to educate people on the best, most accurate ways of measuring have been in vain.

Picture this, if you will: 250 people are gathered for an all-day measurement conference. Speaker after speaker takes the stage and provides truly excellent advice about how to accurately and meaningfully measure their programs. Then comes this particular presenter and her talk on how to measure branding. The audience is treated to an in-depth and very insightful presentation on how communications and branding efforts resulted in significant improvement in both understanding of the category as well as awareness of her brand.

Then came the question: "How did you show ROI to your CEO?" And the answer: “Well, we counted up the impressions and calculated the Ad Value Equivalency of the publicity.”

Just shoot me!

But seriously folks, are we so focused on counting column inches that we can’t be bothered to calculate the value of increased awareness or understanding of the brand?

It’s really not that hard to calculate the ROI of a program:

1. First, find out how many leads it takes to make a sale and what the dollar value of the average sale is.
2. Then compare revenue, new business, and leads before and after the program.
3. Now use the increase in new leads after the program to calculate the potential revenue it brought in. (Ideally you would do an analysis of variance to determine correlation.)
4. Finally, take the cost of the campaign, deduct it from the potential revenue, and you have ROI.

Please do your own version of this calculation instead of using AVEs. Why compare yourself to the cost of advertising when you can estimate the actual value in sales? --Katie Delahaye Paine

More on AVEs: Their true -- and very valuable -- significance.

For serious public relations researchers, the whole AVE and advertising-money-value debate is a red herring, a colossal distraction from the real significance and potential that AVEs represent.

In my previous post, I said that Advertising Value Equivalencies are porn for PR. And I also said that they can be an important component of useful and valid public relations measurement. This latter use is a specific case of a brilliant and powerful research technique that public relations measurement can benefit from.

At the recent 2009 IPR Summit on PR Measurement, Angie Jeffrey provided the most marvelous quote of the whole AVE debate. She said that she used AVEs in her research as "a measure of the perceived ability of media to deliver desired outcomes."

Brilliant. Notice that she did not say anything about advertising or money or value. (All terms that have given AVEs a bad name.) What she did say was that AVEs were used to infer something about human nature, and were used to apply this inference to public relations research.

The really important and valuable thing about AVEs is that they are an example of data from a unusual source being applied to shed new light on a research problem. This general technique is fabulously powerful. It's being used, for example, with great fanfare in economics. (Read Levitt and Dubne's Freakonomics if you want to know more. Actually, read my review of Freakonomics. It will be much quicker.)

Unfortunately, in real life AVEs are a research disaster (with, as I noted, some specific exceptions). Crude users of AVEs draw a direct-but-specious line between advertising and money and PR. Because they are based on the advertising market, AVEs present an irresistible temptation for those who want a quick and easy way to compare PR to advertising, and/or to provide a monetary value for PR outputs. 

Serious PR measurement people know that these uses are bogus, but many otherwise ethical measurement vendors provide their clients with crude AVEs because the demand for this sort of metric is huge. This whole mess is something that the IPR and the PR measurement industry seem to be working hard to sort out. 

--Bill Paarlberg, Editor, The Measurement Standard

AVEs Are Porn for PR

AVEs are the Playboy centerfold of public relations measurement; a fold-out fantasy version of ideal research, eagerly consumed by the masses to fulfill their somewhat distasteful but otherwise unattainable measurement desires. 

That's my opinion (actually only part of my opinion, see the following post). And definitely not that of the recent 2009 IPR Summit On Measurement in Portsmouth, NH, which hosted a reasoned and deliberate debate on AVEs. I sat through an excellent panel question-and-answer discussion session (including measurement heavy-hitters David Rockland, David Michaelson, Angie Jeffrey, and Richard Bagnall), followed by much lively debate over lunch.

What was clearly expressed at the Summit was that AVE concepts can be and have been a useful and valid component of public relations research. Unfortunately, this valid and useful AVE-based research addresses only a very limited area of PR activities. And has nothing to do with comparisons to advertising, nor with putting a monetary value on public relations results.

When it comes to the common, garden variety AVEs, serious public relations measurement thinkers consider them to be a pale imitation of valid research, produced for the legion of clients who eagerly lust after a direct, dollar-value comparison to advertising, and who will pay good money for a tarted-up imitation of real measurement. 

Sounds like porn for PR to me.

And therein lies the bitter irony of the AVE: While the measurement cognoscenti know that a very refined version of AVEs can be a valid and useful technique for a narrow range of inquiry, vast masses of clients eagerly purchase crude versions of AVEs to use in completely invalid and inappropriate pseudo-research. 

Which leads us to the dirty secret of the measurement industry: The demand is so strong for some way to compare PR to advertising and to put a monetary value on PR, that otherwise diligent, upstanding, and ethical measurement companies sell AVEs to their clients. Even though they know full well the limitations of the data and the bogus conclusions their clients will draw. 

As one very skilled but pragmatic researcher from a very large measurement company said, "I get AVEs for free, and my clients want them, so why not?" That is, given that a client has signed up for media monitoring or other measurement services, the data is there and the software can make crude AVE calculations at no extra cost. 

So why shouldn't this product be sold to clients, who are, after all, extremely eager to purchase it?  -- Bill Paarlberg, Editor, The Measurement Standard