by Katie Delahaye Paine
Alyson Dutch, CEO of Brown + Dutch Public Relations, was the sole holdout, and she advocated for the use of AVEs. For her bravery (albeit misguided) we thought we’d take her through a measurement makeover.
Continue reading "A Measurement Makeover for Brown + Dutch Public Relations " »
A Measurement Makeover for Strawbery Banke, Portsmouth, NH
by Katie Delahaye Paine
It was the ultimate measurement irony. Twenty of measurement's best and brightest were gathered in my living room for the Conclave on social media measurement standards and the future of measurement. Then, that very morning, the local paper ran a front page story about the "value" of a nearby visit by President Obama. And how did they calculate the value of said visit? In AVEs, of course!
There are lots of reasons that I can think of that the New Hampshire Legislature needs a makeover, not the least of which is that it has had the honor of being featured in a hat trick of ridicule by showing up on Comedy Central's Indecision site, The Colbert Report, and The Onion. So it’s not particularly surprising to discover that they don’t check their facts. But Representative Kingsbury's egregious misuse of data to further an anti-kindergarden agenda is over the top.
We were similarly appalled by the Florida Board of Education not validating its school rating data.
There’s a simple makeover solution here: Put real data analysts to work, and keep politicians away from the data until its been fully vetted and approved by several independent data geeks. It would take most graduate students I know about 10 seconds to see the flaws in Representative Kingsbury's logic. It might have taken them a day or two to validate the Floriida data.
The truth is that lots and lots of data points get thrown out to the media on a regular basis without careful validation. Part of the reason is that there probably aren't enough data geeks to cover all the bases. But in the meantime, folks, can't you just take a day or two and run some of this by a local stats student before you put this stuff out there?
### (Thanks to Fearless Competitor for the image.)
Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, CEO of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
by Katie Delahaye Paine
The Background
I recently got caught up in a string of emails between Professor Samra Bufkins at the University of North Texas and one of her students, Stephanie Chan.
Stephanie was working out in the real world, doing public relations for Moody Gardens theme park of Galveston, Texas. She was trying to be an accountable communicator and do measurement the right way, but her company was insisting she use the discredited measurement technique Advertising Value Equivalency.
Stephanie wrote to Professor Bufkins:
As I've mentioned to you, Moody Gardens still uses AVE!!! What's worse is that I have to generate the reports. The reports are for our CEO to bring to the Board members to prove the Marketing department's worth in the company.
My only solution is to revamp their entire PR campaign to make it measurable. And by measurable I mean track tickets sold.How'd Katie convince her clients? I know this must have taken a lot of time and persuasion. And as a non-profit, they'd like to know they're not spending money on the wrong things.
What do you guys think?--Stephanie
The Solution
Here is our six-step solution for Stephanie to wean her company off of AVEs and develop a proper public relations and social media measurement system.
Step 1: Learn what accounting data is available.
First of all, wander over to Accounting around 11:30 a.m. with a scrumptious-looking platter of food. Offer it to anyone who will spend half an hour with you explaining what mechanisms are in place to track ticket sales and/or revenue.
Step 2: Learn what online traffic and revenue data is available.
Repeat the process with your digital, web, or online team – whoever keeps track of online revenue and activity.
Step 3: Get specific numbers on desired outcomes.
Come back the following week with a plate of delicious cookies around 3 p.m. and offer them to whoever will provide you with an Excel spreadsheet showing weekly data for the following:
Step 4: Orgainze your activity and mentions data by week.
Now, go back to your own records -- yes, even the ones with AVEs. Create an Excel spreadsheet showing PR activity on a weekly basis. In otherwords, what you actually did: Press releases, blog posts, Tweets, events, etc.
Now create another spreadsheet showing external mentions of Moody Gardens. This includes all your week-by-week media results that mention individual exhibits, events, people, or anything else important. To get started, any non-paid mention counts.
If you are running paid advertisements, make up a third spreadsheet with your media plan mapped out, also weekly.
Step 5: Look for connections between your PR activities and desired outcomes.
Now, channel Sam and your Stats class: Go into Excel and click on the fx button and select "statistical" as the category. There you will find correlations along with a host of other statistical tools (it depends which version you are using).
For starters, run a correlation between your weekly overall PR exposure and weekly web traffic. This will probably show that your external media activity is associated with increased traffic to your website. From there, you can look at specific on-line activities, including ticket sales, comments, and signups. Next use Facebook Insights and your Twitter statistics. What you want to do is to show that your increased PR efforts are associated with desired business outcomes, including web traffic, signups, ticket sales, comments, and requests for information.
Extra Credit: If you are running advertisements, do correlations for that activity against week-by-week outcomes as well. Now examine your results carefully to see if there is a way to separate the outcome impact of PR from that of advertising. Hopefully there will be instances where PR and advertising are targeted differently, or where one was not used at all, that will allow you to infer the relative effectiveness of each on its own. Your goal is to understand and measure the influence of your PR, separate from the influence of advertising.
Step 6: Show the board that your PR efforts drive measurable business results.
Next board meeting, go in armed with your data and knock their socks off. We'll bet you they’ll never go back to AVEs.
###
Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, CEO of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
The real-life lesson of how a tourism organization can measure the ROI of their communications -- without using Advertising Value Equivalency (AVE).
by Katie Delahaye Paine
Normally we would have named New Hampshire Grand a Measurement Menace of the Month, but we decided that it would be a much more useful exercise to give them a Measurement Makeover. In the future we’ll be combining the menace with a makeover so watch these pages for more examples. I welcome your submissions.
This month's Measurement Makeover features New Hampshire Grand. New Hampshire Grand is an organization that promotes tourism in the North Country of the state of New Hampshire. And, since I was intimately involved with its creation, I was appalled when I read this statement in New Hampshire Grand’s recent progress report:
“In 2010, we... published over 369 stories with an earned media value of $512,000. Earned media is the value of our exposure if we had to pay for each column inch of the article.”
"Earned media value" is another name for Advertising Value Equivalence (also known as AVE), an inaccurate and discredited technique for evaluating media coverage. We immediately contacted Montagne Communications, the PR firm for New Hampshire Grand, for more details, but have not heard from them. Nevertheless, thanks to New Hampshire's tourism recordkeeping (see below) we can help New Hampshire Grand express the value of its efforts in a more accurate and accepted fashion. So let's give it a Measurement Makeover. Here goes...
Step #1: Resources Needed
1. Excel spreadsheet.
2. Data establishing relationship between impressions and actual tourist visits.
Step #2: Initial Data and Assumptions
It turns out all the data necessary is publicly available from the www.nhgrand.com and visitnh.gov websites. We have very useful historical data to work with, because the State of New Hampshire keeps good records on the value of its tourism efforts. They know that for every 100 positive tourism impressions they make (or buy) in the media, 3 people will visit the state as tourists. They also know that the average tourist spends about $82 dollars per visit in a combination of taxes and purchases.
Because New Hampshire Grand deals with the North Country specifically, and not the entire state, those averages need to be adjusted:
We can use these numbers to express the value of New Hampshire Grand's efforts in tourist dollars.
We also know that the New Hampshire Grand effort has cost about $1 million so far. We can use that figure to calculate an approximate Return on Investment for the project so far.
Step #3: Collect Some Data
We reviewed the monthly newsletter reports of Montagne that were posted to the New Hampshire Grand website and counted the total number of positive impressions. Since all the stories posted were favorable, we took the reported impressions as our total number of favorable impressions. That number is 5,099,541 impressions. (Montagne also reported a total Unique Visitor count for their efforts of 342 million. This number seems vastly overstated and unreliable, as it exceeds the total population of the U.S. by about 10%. So we did not use it.)
Step #4: Calculate Potential Value of Media Coverage
Using our estimate that every 100 impressions results in one visitor, that gets us to a potential visitor count of 50,995.
And if each of those 50,995 spend $41, that makes a potential spend of about $2 million. Thus the value of New Hampshire Grand's communications can be estimated as $2 million in revenue from tourist visits.
Step #5: Estimate ROI
The formula for ROI is Revenue minus Investment divided by Investment. Revenue in this case is what the potential tourists spend, or $2 million. Investment is about $1 million (which is what the New Hampshire Grand effort has cost thus far). So the ROI estimate would be $2,000,000 - $1,000.000/$1,000,000 or 100%.
Step #6: The Big Reveal
$2 million in tourist spending and an ROI of 100% sound a whole lot better -- and are far more accurate -- than $512,000 in fake Advertising Value Equivalency (AVE) numbers. Congrats, New Hampshire Grand, on your Measurement Makeover.
(Tip of the Hat to Chris Jensen at New Hampshire Public Radio who’s report on the need for New Hampshire Grand to demonstrate its value alerted us to an organization in need of a measurement makeover.)
(Transparency Alert: NCIC, which provides funding to New Hampshire Grand, is a former client of KDPaine & Partners. We conducted research for NCIC and were part of the discussion prior to the creation of New Hampshire Grand. We love what NCIC and The Tillotson Foundation have done and are on record as supporting New Hampshire Grand.)
###
Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, CEO of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
“Do not believe in anything simply because you have heard it. Do not believe in anything simply because it is spoken and rumored by many… Do not believe in anything merely on the authority of your teachers and elders... But after observation and analysis, when you find that anything agrees with reason and is conducive to the good and benefit of one and all, then accept it and live up to it.” 
