Barcelona Principles Report Card: How Do 12 PR Awards Programs Measure Up?

by Katie Delahaye Paine

Back in 1999, the IPR Measurement Commission began an initiative to improve the measurement criteria for PR industry awards.

We sent a letter to the awards committees of all the major associations that said, in part:

The xxx Awards program not only rewards excellence in our profession, but because of its status and influence in the field, it helps shape people's definitions of success. A critical part of defining success is the criteria by which you evaluate the entries... we’d like to propose some standard criteria and guidelines... We hope, by helping to standardize the definitions of “success,” to improve the credibility of our profession, and the status of the xxx awards.

Today, almost 15 years later, I thought it would be useful to grade the various industry organizations on their awards programs' commitment to the Barcelona Principles.

 Prominent among these seven Principles for measuring the success of PR are: #2, Measuring the Effect on Outcomes is Preferred to Measuring Outputs, and #5, AVEs are not the Value of Public Relations. 

Grade A

This month MEPRA, the Middle East Public Relations Association, became the most recent in a growing list of prominent communications organizations who have revamped their awards programs to penalize entrants for using AVEs.

Similarly, CIPR (the Chartered Institute for Public Relations) also recently revised their criteria to specifically reject AVEs: "The CIPR's understanding of best practice does not include the use of Advertising Value Equivalent (AVEs) in any form and AVEs will not be scored. Inclusion of AVEs, even alongside other valid measurement metrics, will result in a zero score for Research, Planning, Measurement, and Evaluation."

AMEC (Association for the Measurement and Evaluation of Communications): "In line with AMEC’s commitment to the Barcelona Principles framework, the use of AVEs as the 'value of earned media' will be considered unfavorably by the judges."

PRSA has a similar statement regarding its Silver Anvil judging criteria: "The use of AVEs as the 'value of earned media' will be considered very unfavorably by the judges. The consideration of AVEs as a value or benefit is simply wrong."

IPRA (International Public Relations Association) is equally clear: "...IPRA does not endorse “advertising value equivalency” or “opportunity to see” as quantifiable measurement tools, please do not include these in your evaluation section."

Of SNCR (Society for New Communications Research)’s five judging criteria, three have to do with effectiveness and measurement.  (Transparency alert: As a founding Fellow and now board member I probably had an influence there. We didn’t address AVEs because they are not commonly used in social media.)

IPR (Institute for Public Relations) Jack Felton Golden Ruler Award: The organization that started this all is very specific on what you need to show to win:

• Links public relations outputs to outcomes
• Demonstrates impact of the program on the target audience
• Demonstrates impact on business outcomes
• Aids in the development of better communications programs
• Proves to be cost effective

Grade B

IABC (The International Association of Business Communicators) is not quite as specific but does judge entries based on whether they align measurement with valid objectives: "Entries should demonstrate output-based results that measure increased volumes as an indicator of progress and/or outcome-based results that influence awareness, understanding, opinion, attitude, behaviours or business results." IABC requires a thorough evaluation that supports the results.

Paul Holmes' SABRE Award addresses some of the Barcelona Principles, but leaves out the part about AVE. He actually explained this at the Dublin Measurement Summit, by admitting that the Awards were his bread and butter and excluding AVEs entries would cost him too much money.

The SABRE awards say: “Be clear about the objectives and the results. And make sure they match. State the objectives clearly, whether the client was looking for increased visibility, enhanced awareness, or an improvement in sales. And if the objective was awareness ('more people knew about my company, or product') make sure the results talk about awareness, and not just how many media impressions or opportunities-to-see you delivered.”

Of PR News' Platinum Awards, publisher Diane Schwartz says, “Evaluation of success is critical in the judging process, as I am sure you will agree. If an entrant cannot show return on investment and how they measured their success, they are usually moved to the bottom of the list.” 

Grade D

Bulldog Reporter’s awards clearly emphasize coverage as a measure of success, and seem to make the assumption that coverage is a sufficient outcome: ”How many media—especially the nation's most influential media—picked up your story, particularly those media with tough requirements and little space for PR-driven stories? Relative to your budget, how much impact did you make?"

Grade F

The CPRPA (Consejo Profesional de Relaciones Públicas de la República Argentina) has no specific measurement criteria at all.

So Now What?

It is great to see how much progress has been made by virtually all the major organizations. Thanks to the enlightened measurement criteria of these awards programs a lot of agencies and corporations will change their behavior.

Congrats to us all for moving forward. I've been railing against bogus measurement techniques -- and AVEs in particular -- for many years, and I confess that this progress provides me with some minor feelings of self-justification. : )

Of course, not all organizations or awards programs are listed here, and there are some more that would receive failing grades. I do know that these things take time to change, so I hope that any award committee members that are reading this will communicate the urgency of demanding proper measurement to the rest of their committees.

I would further suggest that if your organization is even thinking about sponsoring an award program, please review their judging criteria first. If the program allows bad measures of success (like AVEs), please put your sponsorship dollars somewhere else. Unless you do, the value of the award -- and your sponsorship -- will be diminished.

###

(Thanks to BuzzChomp for the image.)

Katie Delahaye Paine is the Chief Marketing Officer of News Group, a leader in international media monitoring and evaluation, as well as Chairman & Founder of Salience/KDPaine & Partners, the leader in integrated social and traditional communications measurement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. She writes KDPaine’s Measurement Blog and publishes The Measurement Standard. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to published this year by Wiley.

The Measurement Standard is a publication of Salience/KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine will be glad to talk with you about measurement for your organization.

 KDPaine & Partners' Facebook page