The Measurement Standard: Blog Edition

Establishing the ROI of Social Media

The Paine
of Measurement

 

The true value of social media is in bottom line benefits and improved relationships.

All this week Twitter has been the scene of furious debate, later picked up in conferences and in blogs, over how to establish the ROI of social media. One side says that without a solid monetary return, it's all just "pixie dust." The other side says that social media is all about building relationships, credibility and trust -- things that don't necessarily translate into bottom line benefits.

I argue that both are right.

To assume that there is no bottom line benefit to improved relationships is simply wrong. Vince Hazelton's team at Radford College demostrated in their paper from the 2006 IPRRC that improved internal relationships lowered operating costs and legal fees, and increased efficiencies. Sandra Duhe in her seminal paper on the impact of CSR (given at the IPRRC 2004) showed that there was a direct connection between an organization's relationships with its community and its financial performance.

And beyond the academic literature there are a ton of additional current examples:

• In a field known for its notoriously high employee churn, Best Buy measures the ROI of its internal "Blue Shirts Nation" community in terms of lower turnover rates.
• The National Association of Manufacturers measures the ROI of its blog in terms of greater access to the halls of Capital Hill. What's the value of that access? I suppose you could calculate it in terms of LFEs (Lobbyist Fee Equivalent), but I bet they don't bother.
• Dell measures the success of its IdeaStorm community both in terms of lower support costs and in the number of new ideas generated. How much is a new idea worth? Technically, it probably depends on how much revenue it ultimately brings in. But you could calculate a "consultant fee equivalency." Just calculate what you pay to outside consultants to creatively solve your problems, I bet it's a lot higher than $100 per idea, which is less than what Dell's costs are.
• Sea World reached out to roller coaster enthusiasts with its social media program and measured its ROI in terms of lower outreach costs as well as tickets sold.

Monitoring vs. Measurement

There's been too much confusion of late between monitoring and measurement. It's great that companies like BuzzLogic and Radian6 can calculate your share of discussion, or even that KDPaine & Partners can calculate your tone, positioning, prominence and dominance. That data tells you which programs work and don't work in terms of improving your messaging and positioning. But we need to get our calculators out to go beyond those easy numbers and start calculating the impact social media is having on organizational mission and the bottom line.

Katie Paine's Top Ten Must-Do Items for Your Social Media Measurement To-Do List

#1. Define measurable goals: a business or mission benefit.

#2. Measure your market, not yourself. Always include the competition.

#3. Make sure everyone agrees on your definition of positive and negative.

#4. Measure messages not just tone.

#5. Measure positioning on key issues, not just sentiment.

#6. Look at comments for at least 4 days (but a maximum of 14).

#7. Don't forget to measure YouTube, Facebook and Twitter.

#8. Measure what matters, not what is popular.

#9. Tie results back to your goals.

#10. Repeat on a regular basis.

Katie Delahaye Paine's Top 10 Signs that It's the End of the World as We Know It

Katie Delahaye Paine's
Top 10 Signs that
It's the End of the World
as We Know It

10. I spent more time on Twitter/Flickr/Facebook yesterday than I did on email.

9. Gatekeepers? What's a gatekeeper? 52% of journalists use blogs to research facts and find stories.

8. Deadline? What's a deadline? 92% of journalists say their online editions are allowed to scoop the print version.

7. It's easier to put my message on M&Ms than it is to get it into an A-list blog.

6. The Obama Yes We Can YouTube video is seen by more people than the number who watch an evening of Monday Night Football.

5. IBM receives more leads, sales and exposure from a $500 podcast than it does from an entire ad.

4. Employers no longer check references, they check Facebook and MySpace.

3. Procter & Gamble is preaching "letting go" and "co-creating" all its marketing with its customers

2. Wikipedia is nearly as accurate and just as credible as the Encyclopedia Brittannica. And a lot more people use it.

1. Google has replaced my thesaurus, encyclopedia, dictionary, and long-term memory.

Why Are Political Lawn Signs Like YouTube Downloads?

The Paine of Measurement

How online measures of engagement have predicted recent primary results.

Ever wondered what the effectiveness of political lawn signs is? Supposedly, every lawn sign represents six votes for the candidate. Or maybe ten votes, depending on what you read. And there's a theory in political circles that if you can get someone to put out a lawn sign, then that person is committed enough to not just vote for you, but also to encourage his or her friends to vote for you as well. So, each additional lawn sign means more than just one more vote, it means more of something even more valuable and a lot more difficult to pin down: more loyalty or commitment or what we in communications call engagement.

To my knowledge, no one has ever done a scientific study of how lawn sign displays influence voting habits. But my completely unscientific study of New Hampshire lawns this fall more or less predicted the outcome of our First in the Nation primary: Everywhere you went there were lots of Obama and Ron Paul signs, and both did much better than the polls predicted.

Now let's transfer this scenario into the world of social media. Can online measures of engagement predict votes? I argue that they can and have done so recently:

• Our YouTube study of candidates showed Obama having a significant lead over Hilary Clinton both in terms of viewership, and in terms of the number of videos that were rated by viewers.
• On Facebook we noted that there were some 500,000 plus groups supporting Obama, compared to Hillary's 100 or so. (In fact there are far more groups opposed to Hillary than there are those in favor.)
• In terms of Facebook's US Politics application that has Facebook voters register their opinions on a variety of topics as well as on the candidates themselves, Obama has consistently maintained a 50- to 60-point lead over Clinton.

And in the end, Obama did better in the primaries than the early polls suggested. The primary results have proven that Obama has a stronger than expected following, as hinted at by the strong online engagement we found.

The point here about engagement is bigger than just politics. How and why is engagement a stronger or different measure than just impressions? If, by joining a group, rating a video, or following someone on Twitter, you are actually thinking or behaving differently than if you just viewed an ad or a message, then measuring these signs of engagement is critical to every marketer. In order to hang on to advertising dollars, media companies will need to provide this data. And the good news is that the data is there, they just need to release it.

And finally, I can't help but see engagement as a kind of bridge between measuring outputs and measuring relationships. (Most of you are aware of my recently published book Measuring Public Relationships, learn more here.) If you measure an output like impressions, you only know what has happened to an audience. But if you measure engagement, you are measuring what is done by an audience as the result of their relationship with your output. How does measuring engagement fit in with measuring relationships? That's a good question, let me know if you have the answer.

Here's wishing you large measures of success,

 

My New Year's Resolutions

The Paine
of Measurement

 

Some realistic, some from my wildest dreams.

Friends and I recently performed our own self-measurement analysis by asking ourselves the question that I always ask my clients: "Imagine we're sitting here a year from now, sipping the finest Champagne and toasting to truly outstanding success over the past year. What achievements would you need to have accomplished to result in such exuberant celebration?"

My friends listed things like: run a half-marathon, get a better job, start writing a book and, "Have a more engaged relationship with my husband." I couldn't help but admire them in their clarity.

Then came the moment I was dreading when all eyes turned to me: What would I consider a wildly successful year? To be honest, I had been so busy just making it through 2007 that I hadn't given 2008 much consideration. But I came up with a few resolutions, and, now that I have given it some more thought, I'd like to share them with you.

My off-the cuff answers to my friends are incorporated in the two lists below. First there is the Champagne Dreams List -- those achievements which qualify for the finest bubbly in celebration. And then there is the more sober view of what, with a lot of hard work and a little luck, I can reasonably resolve to achieve in the coming year.

The 2008 Champagne Dreams List

1. The majority of my clients are marketing VPs for whom my company, KDPaine & Partners, is tying PR results to their bottom line or market share. And all of them are measuring relationships.

2. Measuring Public Relationships (my book that has just been published) is in its second printing, and my next book is at the printers.

3. KD Paine & Partners' research becomes the defined standard for social media measurement.

4. The New York Times and NPR call me regularly for quotes on marketing issues, and I have frequent conference calls with CMOs all over the world to talk about measurement.

5. I'm making three times more money, have retired my mortgage, and spend 40% of my time growing things instead of measuring them. (Well, I can dream.)

The 2008 Realistic Resolutions List

1. KDPaine & Partners has developed or contributed to a widely agreed upon metric for social media measurement that includes an engagement index.

2. KDPaine & Partners has developed or contributed to a widely agreed upon metric for transparency.

3. KDPaine & Partners has exceeded its annual sales goal and we have added at least 20 jobs to the Berlin, NH economy.

4. I've started on Measuring Public Relationships, Volume 2. (and by the way, Volume 1 is up on Amazon now, or you can order it direct from measuresofsuccess.com.)

5. We've helped at least a dozen new clients correlate their media coverage with their business outcomes.

6. We've helped at least a dozen new clients measure their public relationships.

7. We are helping at least a dozen new clients measure new forms of social media, including whatever comes next after Twitter, Facebook and YouTube.

8. Concert for a Cure has raised an additional $15,000 for cancer research and support.

9. I've run two half marathons.

10. I've listened more often, talked less, written more, asked more questions, provided more answers, and let go of more stuff.

Here's wishing you all the best with your resolutions,

Don't Ask Me... Ask Your Customers!

The Paine of Public Relations Measurement

The Top Five Questions I'm Really Tired Of

Don't get me wrong, I love being your friendly neighborhood public relations measurement guru. But, I'm getting real tired of people asking me questions that only their customers have the answers to. Here are the top five:

1. Should we start a blog?
2. Should we respond to bloggers?
3. Should we sponsor this race or this concert?
4. What should we do to improve our internal communications?
5. How do we get our customers more engaged in our brand?

All good questions, but why are you asking me? You should be asking your customers!

Let's take these one by one:

1. Should we start a blog?

Answer: I haven't a clue. But here are some questions that will help you figure it out:

Do your customers pay attention to blogs?

Does social media drive their purchase behavior?

Are you trying to recruit from among the Gen Y and YouTube crowd?

If your answers are "Yes," then you should start a blog. In some circles The Huffington Post is a lot more influential than The Washington Post. And among automotive types, there are bloggers that carry as much weight as Road & Track. But, if you're trying to sell cruise missiles to the 150 or so people on the planet that can buy them legally, then chances are pretty good that those 150 aren't going to turn to Facebook -- or your blog -- for their buying decisions.

So ask your customers what they consider reliable sources. Where do they go for information? What sources do they trust? Whose recommendation carries the most weight?

2. Should we respond to bloggers?

Answer: It depends what they're saying. And it really depends on your answers to question 1, above. You should always welcome an opportunity to listen to or have a conversation with your customers. You should also be open to conversations with what Paul Gillin calls The New Influencers.

3. Should we sponsor a race or sponsor a concert?

Answer: Will your customers be there? That's all that matters, (at least until you can measure the actual event). I can remember a very smart CMO asking me whether he should be sponsoring a sailboat race, a NASCAR event or a concert. My response was, again, "Don't ask me... ask your customers!" Where are they most likely to be? What is most likely to improve their loyalty? One key benefit of sponsorship is the hospitality. Are your customers more likely to enjoy a box seat at the Opera, going back stage at a concert, or having inner circle access at Nascar? I don't know, ask them.

4. What should we do to improve internal communications?

Answer: I don't know... ask your employees. Chances are they know a lot more than I ever will about your internal communications and what needs to be improved.

5. How do we get our customers more engaged?

Answer: Listen to them, hear them, pay attention to what they love or hate about you. Ask them what they hate, then fix it. Ask them what they love, and do more of it.

Listening Isn't Hard, It's Just Different

What I'm getting to here is that it's not hard to listen to your customers. They're talking all the time, to each other, to your competition, to others in the industry. You just need to listen.

Sometimes its as easy as monitoring consumer generated media (CGM). In other cases you just might have to do a survey. We recommend using the Grunig relationships instrument as a basis for any survey you do. (It's free and downloadable here.)

The point is that listening isn't hard. It's just different. For years we've spent all the money and focused all the effort and energy on screaming ever louder at our markets. Now it's time for the customers to talk and for us to start listening.

Happy Holidays,

Measure the Media that Are Most Important to Your Customers

The Paine of Measurement

 

 

Make Good Choices
No one says you have to measure every medium on the planet.

When my cousin Caleb was into his "terrible twos" and heading for trouble, his father, who stands an imposing 6 feet 3 inches tall, would look down at him and say in his deep baritone: "CALEB, MAKE GOOD CHOICES." Invariably, Caleb would stop what he was doing, sit down and think for a minute, and then modify his behavior accordingly.

The lesson seems to have taken. Caleb is now taller than his father, and a true teenage heartthrob. With very few exceptions, he makes great choices.

Oh, that PR people had such good survival skills. Sadly, they tend to keep doing the bad stuff rather than embracing the good. Four years ago, a member of the audience at a conference on new media raised his hand and said, "l don't have time to deal with all this on-line stuff, I'm too busy as it is."

I wonder how many PR people in the 30's and 40's said the same thing about radio and television. "Woe is me, one more thing I have to learn how to do." Rather than embrace the new, PR people tend to look at new media as yet another burden to be borne. And once they start measuring and tracking one media, they find it extraordinarily hard to embrace a new one. In the last month, at conference after conference, people have whined about how hard it is to deal with social media, and why they don't have the time or resources to measure it.

My answer is: Make Good Choices. No one says you have to measure every medium on the planet. Just because it's new, it doesn't mean you have to monitor it. You can and should choose to concentrate on, and measure, the media that are most important to your customers. If they're on Facebook and YouTube, you better be monitoring them. If they're sharing ideas on Twitter and photos on Flickr, it would behoove you to sign up and listen in. On the other hand, if they're still reading hard copies of trade magazines, then that may be all you need to monitor.

Better still, measure the impact on the customer directly. As Jim Macnamara writes, the average person sees, consumes and creates dozens of different media every day. If you have an unlimited budget, you measure all of those different media. But it's so much simpler to just measure the behavior of the consumer instead. Ask them how they perceive their relationship with your organization. Follow their behavior on your website. Watch and see how engaged they are with your blog. That's all the measure you need.

As long as PR people obsess about tracking column inches and "hits," of course they are going to be overwhelmed with the possibilities and the burden of measuring them all. But if they focus on the outcome – the actual customer behavior – chances are they'll survive and thrive in this new hyper-media world.

Recent Public Relations Research Sets Up Conflict Between PR Professionals And The Old School Bottom Line

The Paine of Measurement

The Old Bottom Line Collides with the New Public Relations
New public relations research -- old corporate mindset.

Rumor has it that there is an ancient Chinese curse: May you live in interesting times.* If "interesting" can be translated as "full of conflict," then there is no doubt that modern Western business is now feeling the full effect of this curse. PR people in board rooms all over are headed for a confrontation that makes recent skirmishes between presidential candidates look like scuffles in the proverbial sandbox.

For years, the vast majority of organizations have been driven by a single bottom line, profit. This was relatively easy to calculate. You added up what it cost to make something, plus a bit extra for administration and overhead, then subtracted that from what you charged the customer and you had your bottom line.

But life in the 21st Century is anything but simple. Recent research has demonstrated that the old bottom line is not the whole story. First we found out that the way to make bigger profits is to have more loyal employees, and that we have to do good deeds to gain employee loyalty. Then we found out that good communications is a leading indicator of an organization's financial performance.

And now, most recently, Brad Rawlins has demonstrated that in order to best win the trust of our employees, we have to open our kimono to the world and be completely transparent. (See the article in this issue, and my blog post.)

So here's the conflict: Today's diligent and capable PR person walks into the board room urging better communications, more transparency and open, honest conversations with stakeholders. But the room is full of Machiavellian power brokers who believe in secrecy, stonewalling and the single bottom line. This old-fashioned corporate mindset still insists that the only reason to do something is because it makes money or gains a competitive advantage.

We now know that such an approach is outdated, and those who are stuck on it will soon find their businesses as dead as the dinosaurs they resemble.

The reality is that good communications does make money, it just doesn't do it quite as simply and directly as the old bottom line arithmetic implies. Openness and transparency translate into money saved in legal bills, into efficiency gains because information is more readily available, into lower turnover rates and higher customer loyalty. If trust has a value, transparency is its currency in today's marketplace.

Wishing you large measures of success,

* It is supposedly the first of three curses, each escalating in severity. The other two are: May you come to the attention of those in authority, and, May you find what you are looking for.

Isn't it wonderfully paradoxical that those curses are so similar to the job descriptions of most American PR people, and to the results they typically strive for: crisis control, the attention of those above you, and news? And these curses, mind you, are part of the social heritage of the largest, fastest growing and probably soon-to-be-most-powerful society on earth.

A Public Relations Measurement Index Number Based on Ideal Coverage

The Paine of Measurement

Optimum Content Score: An Index Number to Love?
Here's a quick and easy way to design your customized article content score.

For years I've ranted about why a single Index number or a standard PR metric won't work (see "The Problem of PR Indexes: Magic Number or Big Headache?"). My argument has been that each PR program is different, with different goals, different audiences, and different measures of success. So how can you possibly compare them all with one number?

Embrace the Differences

However, I am beginning to believe that there might be a solution to this problem, and that is to embrace the differences--to build the differences into the measure. Here at KDPaine & Partners we now offer our clients a measure that is tailored to match their individual program goals: the Optimum Content Score (OCS). It gauges the success of articles, based on what the client considers to be the ideal article. With the OCS, we distill a potentially large number of variables into one number, based on the relative concept of “ideal" coverage. That way, we can tell you whether your coverage was more ideal than last year (comparing to benchmark), or less ideal than you wanted it to be (comparing to objectives).

What Is the Ideal Article?

For any particular client, we define an ideal article based on their coverage goals at the present time. So, for instance, an ideal article for Client X might be one that:

• contains several key messages,
• leaves a reader more likely to purchase,
• mentions the brand in the headline, and
• appears in the Wall Street Journal.

If an article includes all these attributes, we give it the maximum score, ten out of ten. To determine the score for any given article, points are deducted from the maximum based on how far an article is from ideal, taking points off, for instance, for negative positioning and minor mentions.

The beauty of this system is that the company that is looking to keep out of the headlines can gauge its effectiveness as well as a company that is trying to get into the headlines. For instance, in the middle of a crisis, ideal press might be a minor mention that contains a key message and is balanced.

I realize we're still measuring outputs here (and our long term goal is to measure outcomes), but this sort of customized-to-goals score is a big improvement over imposing one score across a wide variety of organizations and industries that may not have a lot in common. See Ed Moed's "Measuring Up" blog post on the subject here.

OCS allows you to look at those things that are important to your own business, insuring that you are measuring the most relevant elements of what you do. For instance, Sabrina Steele at Raytheon has used it to make better decisions on supporting trade and air shows, using spokespeople, and comparing her effectiveness with her peers.

Another advantage of OCS is that it can be compared to other marketing data to determine what does, in fact, drive outcomes like Web traffic, product preference and consideration. Factor analysis using OCS can tell you exactly which program elements and media efforts are having the most impact on whatever are your ultimate marketing measures of success (typically sales, sales leads or market share).

Design Your Own Article Content Score

You can easily get started with your own version of an optimum content score. Below is a sample list of questions--and you may wish to remove some or add your own--that you can use to determine the characteristics of the ideal article for your own program. Then rate your articles on how closely they come to your ideal.

Wishing you large measures of success,

Sample questions to determine ideal article content:

1. What is the goal of the program? (pick only one):

a. Increase exposure

b. Keep bad news to a minimum

c. Disseminate key messages

d. Increase preference

e. Generate awareness for spokespeople

2. How important is brand or sub-brand visibility?

a. Extremely important

b. Somewhat important

c. Not important at all

3. How important is it that the story be exclusively about your organization or brand?

a. Extremely important

b. Somewhat important

c. Not important at all

4. How visible do you want your spokespeople to be?

a. Highly visible

b. Somewhat visible

c. Doesn't matter

d. Invisible

5. How important is competitive positioning?

a. Extremely important

b. Somewhat important

c. Not important at all

6. How important is the tone of the coverage?

a. Extremely important

b. Somewhat important

c. Not important at all

 

 

PR Measurement Must Move Beyond Measuring Readers and Start Measuring Friends: The New Measure of Success is Popularity

The Paine of Measurement

Why Measure Eyeballs When You Can Measure Friends?

The latest news is the work the IAB is doing to make the numbers agree between the two leading eyeball counters comScore and Nielsen//NetRatings. (We gave these two firms our Measurement Menace of the Month Award this month.) By the time they get those guys to agree, the concept of counting eyeballs will be obsolete.

Google, with its revamp of Google Analytics, is doing a much better job keeping up with the latest demands in measurement. If you're not using it, you should be. It tracks not just traffic but sources of that traffic and conversion rates. It's data you can use to decide which tactics and strategies are paying off and which are a waste of time. Not based on eyeballs, but based on the response from your audience. Now that's useful stuff.

And besides, why measure eyeballs when you can measure friends? Whether it's on LinkedIn, MyRagan, Twitter or MySpace, the new measure of success is popularity. Actually, hasn't that been the measure of success since before the ruler was invented? Among political candidates, the business of comparing the number of friends is rapidly replacing the business of counting the number of column inches and campaign contributions raised. And, I have to say, it's a bummer if I start my day on Twitter and it tells me I have no friends.

The reality is, we all have friends. I'm talking about real friends in real life. When you're going through traumatic times those friends come out in droves to help you. Which is what people should really be measuring. Not how many friends you have on MySpace or Twitter, but how many you have that will help you celebrate during the good times and help you survive through the bad times.

Businesses need to look at these new measurement schemes through the same lens. Do they help the bottom line? Do they defend the reputation in tough times? Do they help spread the word when the news is good?

All the MySpace pals in the world may not boost your stock price, but they just might help prop it up a little longer during a crisis. And that's the big point here: Measure the impact of social media against the true bottom line, not an artificial benchmark created by the media.

Wishing you large measures of success,

Measurement Standards: Why Can't We Decide?

This article was published in our Feb 8 issue
THE PAINE OF MEASUREMENT
Measurement Standards: Why Can't We Decide?
We don't have standard measures because there are no standard PR programs.
We in the PR world have been jabbering on about needing measurement standards for at least the last twenty years. Despite the best efforts of the Canadian PR Society and others, we're no closer to achieving that goal...