KDPaine's PR Measurement Blog

Edelman academic summit, part 2 from Karen Russell's perspective

Karen gives a little more detail to the session on Measurement here: Teaching PR: Edelman academic summit, part2

Interesting study on On-line Reputation Management

Paul Dunay has put out an interesting (I think) Online Reputation Management Survey.
I say "I think" because there's no detail about the methodology, there's no guarantee that the data is at all reliable. Be that as it may, it does give some insight into the current state of the social media measurement and monitoring market. Clearly people aren't doing nearly as much measurement or even monitoring as they should.
Take the data for what its worth but ignore Dunay's information about measurement. He's a bit behind the times. For one, he suggests that you hire a traditional media monitoring company because "They even put a price tag on the media value from an impression count. These are worth the investment if you have the dollars and want/need to."  He's obviously missed the AVE's are dead memo. Why would anyone pay for "media value" under any circumstances? It is very simply bad data based on no scientific thought or research whatsoever.
He also asks his audience if they monitor blogs using a subscription service. To the 77% of respondents that don't he says:

While I would love to have a dashboard filled with media indicators from firms like Cymfony and BuzzMetrics, I feel you need a level of activity in new media (and old) that would require you to have this. My volume on a daily basis is not that onerous that it can’t be handled by Google Alerts and Technorati feeds. Once you hit a certainlevel of volume I do think firms like Cymfony and BuzzMetrics make sense. 

While Paul's right that it does require a hefty budget and therefore alot of volume to justify Cymfony or BuzzMetrics, there are lots of services out there -- okay, there's ours -- that are both affordable and scalable. Allowing clients to get real measurement for a price that is appropriate for their budget. We have numerous clients that start off with our $3600 a year Do It Yourself dashboard, and gradually move up to more robust measurement.
He's also wrong when he says that you can't measure YouTube of Facebook . There's a detailed story here on just how you can measure YouTube. And as to Facebook, we measure reputations there all the time. Look for a detailed article in an upcoming issue of The Measurement Standard if you want to know how.

 

Someone's listening!

Jodi McDermott of Clearspring Techonologies wrote a great piece for  MediaPost on - Social Media Measurement.
We've been saying for years that companies spend far too much of their budget on screaming at people and not nearly enough on listening. Jodi's advocating for more listening and we love her for it.

Interesting stats on the state of the PR Evaluation market

This just in from AMEC:

FIRST WORLDWIDE STUDY INTO EVALUATION

AMEC has carried out the first ever international market study of the evaluation sector which shows year on year growth and an industry confident about its future.

The Business Monitor market study is the biggest initiative AMEC has carried out during 2008. Its aim is to give members insights into how the international evaluation industry is performing and allow them to compare their own company performance against what that the industry is doing internationally.

Participating Full Members will receive their own performance scorecard on 1st July, showing how the results they provided compare against the rest of the industry.

The Business Monitor study shows an international industry size with revenues from evaluation of more than £25million from companies in the UK, Italy, Spain, Holland, Sweden, Australia, US, Middle East and Latin America, representing over 70% of the AMEC member companies.

The real size of the industry could be almost double that figure.

Highlights of the survey are:

• 14% year on year growth since 2004.
• Clients see the value of continuous tracking: the survey shows 85%of work is continuous client business with only 15% from ad hoc assignments.
• 34% of clients commission multi-country assignments, rising to 48% amongst AMEC members with a turnover of more than £1 million.
• 84% of end clients include print media in their evaluation assignments, compared to just 29% including digital media and 5% social media.

Click to enlarge

As shown in the diagram, AMEC members are strongly optimistic of continuing growth in the sector, with 47% of member companies stating they believed the evaluation industry will continue to grow in the next 12 months and a further 32% of members reporting they felt conditions would remain unchanged.

The Business Monitor study was open to AMEC Full Members and will be run again in 2009.

Barry Leggetter, Executive Director of AMEC, said: "The industry is showing strong growth, year on year as more companies recognize the important role that evaluation has to play in giving insights into brand performance and how their brand compares against competitors.

"We have also seen a trend that we believe will continue internationally, with more companies in the traditional press cuttings area (PCA's) expand to become AMEC members and provide analysis and consulting services."

I love it when the "R" is real business

This is a great example of companies tying their promo items to real returns.  The ROI of quality swag - iMediaConnection.com.

More proof that measurement mavens rule!

Link: New Survey: Marketers with ROI Discipline Pull Ahead While Data Access and Measurement Abilities Hold Back Marketing Performance.

A YouTube video isn’t just a YouTube video

We're thrilled to see Sean Moffit and Marcel LeBrun picked up on our 27 types of conversations. So, as video occupies an increasingly important part of the media landscape, we thought we'd also thought we'd share our 19 different classifications of video.

1. Advertisement
2. Animation
3. Demonstration
4. Event/Performance
5. Fiction
6. Film
7. Home Video
8. Instructional Video
9. Interview
10. Lecture
11. Montage
12. Music Video
13. News Broadcast
14. Promotional Video
15. Sightseeing/Tour
16. Slideshow
17. Speech
18. Television Show
19. Video Log

     

Almost as good as being there..

I have to say that I really wish I could have heard Sean Moffit and Marcel Lebrun's session at Edelman's New Media Academic Summit 2008. Their 10 rules of social media measurement is great!

If you've been wondering how to Measure the ROI of Blogs and All Things Web 2.0 listen up

I've gotten alot of questions of late as to how to measure the ROI of social media and I promise to answer them all on July 16th here: Bulldog Reporter Social Media Measurement for PR: How to Measure the ROI of Blogs and All Things Web 2.0.

Off to the 4th Annual “Breast Fest”

It will soon be 5 years since I was diagnosed with breast cancer, and every year since that first awful year, we've put on the Seacoast Concert for a Cure to raise money for cancer research and survivor support. It never ceases to amaze me what the good will, hard word and determination of a small band of dedicated men and women can accomplish. Everything is donated, volunteered or done for a very reduced cost. And every year we raise between $15 and $20K for the causes we support. After months of meetings, worrying and guilt over what we haven't done yet, somehow the day dawns, the people arrive and all is well. This year is no different. We had our share of hiccups, musicians whose schedule changed, venue issues, and a few sponsors who in these tight times didn't return. But ultimately, it all comes together. This year we're adding a plant sale to the mix which has given me something to do with all the excess plants I seem to produce on a regular basis. So if you're in the neighborhood of Redhook Brewery in Portsmouth, NH today. Stop on by. We'd love to see you, and so would the survivors we support.

Another alternative to the Ad Value nightmare

So I've been talked down off the ledge and am led to believe that perhaps ALL my efforts to disabuse the Marketing and Advertising nabobs of their  obsession with counting eyeballs are not in vain, when there are so many more important things that you can and should measure. One more time – who CARES how many eyeballs you reach if they don't do anything with the information you send them?  They all seem to have the direct mail disease that seems to think that it's great if 90% of your effort is wasted.

But here's another take on it. Let's assume that your internal data says that reaching eyeballs really does lead to sales or market share or something meaningful to your bottom line. And lets assume that your competition can draw the same conclusion. Presumably the most important metric, therefore would be your share of the social media conversation. (SOC).  Forget about the actual number of eyeballs – what really matters is that you get more than the other guys, right?

Now obviously you only care about certain conversations that matter – and regardless of how you define "matter"  no one can argue that there are different levels of authority/influence/rankings depending on what industry you're in.  So if you define the universe of conversation of which you need to be a part, based on ranking, authority, or personal preference, your share of  that conversation (can we call it BRP Blog  Rating Points?) would be a key metric.  And, while it might not have a monetary value  like a  Gross Rating Point, GRP) since most of the cost to reach that universe is undefined or free, it pretty quickly tells you how you're doing in the marketplace. And, if you must to put a dollar figure on it, rather than look at CPM, look at a cost per SOC – i.e. what did it cost you to purchase 1 percentage share of that universe. Take the total cost of your social media marketing program and divide it by your share of discussion.

The formula looks like this:  CSoC = Social Media Budget / SOC where CSoC (Cost per Share of Discussion) is equal to the total Social Media Budget divided by your Share of Discussion.

So if you spent $100,000 and your share of discussion in your marketplace was 25% your CSoC would be $4. Which would be great unless of course the competition also spent $100,000 and got 50% share of discussion. It's CSoC would then be $2.

Thanks to Sally Falkow and Dunkin Donuts Caramel Latte for inspiring this entire train of thought before breakfast!

Being off by the population of Texas is not a rounding error, Part 2

This is a great piece on why you can't trust "eyeball counts" any more: New comScore figures suggest fewer people believing comScore

In case you weren't in Toronto last month

Donna Papacosta of Trafcom caughte me after the Third Tuesday event in Toronto last month. Here's a link to the Podcast:

The Ad Equivalency Debate Round 2

Eric Shwartzman makes a good point in his comment to my rant about Ad Value Equivalency in Social Media.

As he says, he's not asking the question, it's brand managers that are asking for the equivalency number as a way to justify their social media spend. There are, however, 2 problems with this.

The first is to paraphrase Jim Grunig:  measurement isn't about reward or punishment, it's about having data to know what programs should be continued as is, modified or scrapped in favor of a more effective approach. Ad Value doesn't give you that information, because it answers the wrong question: how many eyeballs did I reach? Isn't the question really, which eyeballs did I reach and what did they do afterwards?  Put another way, as I told my session at PRSA's Travel & Tourism conference yesterday "So what if 300 million people see a blog posting about your destination? If the blog posting isn't compelling or interesting, and no one visits your state or your destination, how can you put any  "value" on those eyeballs."

Let's assume that those brand managers that Eric is talking about really want to know whether Social Media is working for them. By using Ad Value Equivalency, first they are defining "working" based entirely on reach not action. Worse, they are assuming that a blog posting that is seen by 1 million people has the same impact on the brand as a paid advertisement in a media outlet with a circulation of 1 million. But THERE IS NO SCIENTFIC EVIDENCE THAT THIS IS THE CASE. No one has ever shown that people's reactions are the same.

More to the point, why waste time counting eyeballs when you can measure outcomes? Isn't it much more useful to set up a web analytics program that tells you whether, after reading the posting, people went to your web site and requested more information? Or, if you calculate the ultimate value of that visitor in terms of how much he/she spends at your destination.

Everything I've said in the last three years is wasted

Never have I been so discouraged. Eric Schwartzman (OnTheRecord on Twitter) who is someone I like and admire, just asked the Twitterverse the following question: 

if can anyone recommend an online monitoring/analysis service that provides an ad equivilency figure for online coverage?

This is SO wrong, Eric, it belongs in the Clueless Train Manifesto. Ad Value Equivalencies are based on column inches and audited circulations from traditional media. Never mind that there is no evidence anywhere that consumers behave the same way when they see and ad or editorial, and that there's no science behind the calculations, and most developed countries have rejected this metric, it also has nothing to do with what people are seeing, hearing, believing or doing. So why do you care?
In social media the science is even more dubious. What "Ads" are you comparing your editorial to? Pop Ups, Banners, Ad Words, YouTube?, Facebook ads? Underwriting?  how would you calculate an "equivalent" between a blog that doesn't accept advertising and doesn't share its visitor numbers but has huge influence, such as Shel Israel's and a pop-up ad that every ignores?
More importantly, why would you want to? There is no science here, Eric. So if you or anyone provides this so-called "equivalency" they're lying to the client and delivering unadulterated crap.Why wouldn't you instead spend the time and resources studying what people did after reading that blog?
I'm curious if anyone DOES recommend an agency that provides these bogus numbers. I'd love to warn my clients away from them.

 

 

Great perspective on online readership figures

This blog post on online readership figures backs up my argument that measuring eyeballs in a Social Media world is not where you should be putting your energies.

I can die happy now...

I've long said that I what I want on my tombstone is "She killed  Advertising Value Equivalents"  and apparently according to Steve Ellis it is officially  dead.

I make Karen Russell's Top 10

Karen is one of those really smart PR people who gets social media and also understands measurement, so I was pleased to make her list of The Week's Best.
There's other great stuff on the list as well

KDPaine & Partners proposes new un-standard for social media measurement

It seems that almost all of my conversations lately have been about the need for some sort of standard metrics for social media measurement. My response is always the same as it has been for the last 2 decades of measuring communications: There can be no standard because there is no standard goal for "communications." Yes, there may have been a standard way to count eyeballs in print and television, (and the accuracy of that can be debated) but just reaching eyeballs is seldom the goal for a good strategic communications program, and the goal always drives the selection of your measurement tool. If you're the PR Manager for Mothers Against Drunk Driving, your goal is members, contributions and ultimately fewer drunk drivers, and fewer deaths. If you're Southwest airlines, it might be ticket sales. If you're the National Association of Manufacturers, the goal is influence and access to Capital Hill. If the goal is to improve relationships with your customers, you need to measure engagement. So establishing a standard metric for such diverse goals is clearly a waste of time. Each communicator needs to decide for him or herself what success means in terms of his/her own organization.

However, when it comes to judging media, if one accepts that each organization has a slightly different approach to success, KDPaine & Partners is proposing a non-standard standard for media measurement if you will. We call it the Optimum Content Score (OCS) . The good news is that it works for both traditional and social media and enables you to track progress over time and against the competition or peer institutions. This is how it works:

You define what the perfect article, TV spot, or Radio Mention, Blog Posting , Tweet etc. looks like. Presumably it would feature your brand prominently, it would contain a key message, it would position your brand favorably on whatever issues you care about etc. etc. Not all criteria are applicable in all situations, so you pick and chose from a list that might include the following:

 

1. Visibility – where is the brand mentioned
2. Dominance – how much is the brand mentioned, does it dominate, or is just mentioned in passing
3. Messaging – does it contain one or more of your key messages? Or does it contain the opposite – a negative message or something you don't want to see in print
4. Positioning – does it position your brand favorably on issues of importance to your audience?
5. Who is quoted – are your thought leaders quoted or included?
6. Tonality – does the article/item/posting leave the reader more or less likely to do business with you, support your cause, do whatever it is that you want them to do
7. What type of conversation is taking place – there are 27 distinct types of conversation that we've identified. Understanding the nature of the conversation is critical to knowing what to do about it. Here are the 27: Presumably the most desirable would be expressing support or giving a shout-out, but again, it depends on the purpose of the program.
   1. Acknowledging receipt of information
   2. Advertising something
   3. Answering a question
   4. Asking a question
   5. Augmenting a previous post
   6. Calling for action
   7. Disclosing personal information
   8. Distributing media
   9. Expressing agreement
   10. Expressing criticism
   11. Expressing support
   12. Expressing surprise
   13. Giving a heads-up
   14. Responding to criticism
   15. Giving a shout-out
   16. Making a joke
   17. Making a suggestion
   18. Making an observation
   19. Offering a greeting
   20. Offering an opinion
   21. Putting out a wanted ad
   22. Rallying support
   23. Recruiting people
   24. Showing dismay
   25. Soliciting comments
   26. Soliciting help
   27. Starting a poll

        

        

Once you've determine what constitutes "Optimal content" you also need to decide what constitutes the opposite – the type of story you really wish would never appear.

You can then rate each item, posting, article or transcript according on a scale of + 1 to -1 depending on its content. You then average your score for the month/week/quarter (whatever timeframe is most appropriate). You will also need to do the same for your competitor's items. That way you can quickly see who is doing better or worse depending on your industry.

 

 

 

 

How Dell Measures Success

Listening to Lionel@Dell at Blog Potomac talk about how they measure the success of their blogging efforts:

• Benefit is that interacting and engaging helps change perceptions
• We could write press releases about this stuff, but social media puts it into practice
• Interaction does make a difference. We used to see 50% negative comments, now just 20%
• Ability to report on what's important to the customers

 

Blog Potomac Preso

Since we're not doing Powerpoint at all at BlogPotomac, I thought I'd summarize here what I talked about Friday afternoon.

1. Basic 6 steps.
   1. What do you want to measure? Traffic, content, reputation, relationships, sales? Revenue?
   2. Define audience and put into context of what you can impact
   3. Define metrics
      1. For external blogs: Optimal Content Score – tone+positioning+content+visibility+engagement – but really it's whatever is important to you
      2. For institutional blogs: Eric Peterson's Engagement index
      3. For media: Engagement + Web analytics
      4. For internal blogs: engagement, turnover rates, recruitment costs
      5. Pick 3
   4. Pick a benchmark
      1. Peer institutions
      2. 13-month rolling averageContent sourceool
      3. Quantifiers
         1. Compete vs Quantcast vs ComScore vs Nielsen
      4. Qualifiers
         1. 26 types of conversations
         2. Definitions of tonality
         3. Topics, issues, positioning
   5. Analyze results and glean insight
      1. Without insight it's just trivia
      2. What works, what doesn't work
      3. Must relate to business goals
2. Where are we going?
   1. Need for standard definitions – not going to happen
   2. Need for guidelines – in process
   3. Stop asking me, ask your customers

       

       

       

       

Mainstream London media discovers PR measurement

In the "I wish I'd done that" category -- kudos to Metrica for landing this great story: Measuring PR success by column inches is old hat – welcome to a new analysis -

Almost like being at the IPRA Summit

If like me, you were otherwise occupied during the  IPRA Summit you can check out  what happened on their blog.

The M Word in social media

I love the fact that Jeremiah Owayng has declared this the year of Measurement for social media.  Maybe, it's because  Robert Scoble has said  the big companies are starting to ask whether social media increases sales. His answer is that there are lots of things that organizations do that can't be tied to sales so why should social media be held to a higher standard?  I have to agree. I think using sales as the only measurement is wrong. I also agree with Josh Hallet that the page view is a very dangerous measure. Yes, media uses it to show the "eyeballs" so people are comfortable with it, but  counting eyeballs becomes increasingly meaningless in a world where we "snack" on media and pay so little time with most of it that we can't remember what we saw where.  Similarly, the  fallacies of clickthrus are being brought to light  as organizations realize that attributing all the "sales" to the last URL is ridiculous. Are you telling me that the last pop-up or Google Adwords is the reason I bought my computer? of course not. I talked to Richard@Dell  and a bunch of other people about laptops, looked at on-line reviews, asked about it on Twitter, and finally went to Staples because I wanted to touch it and feel it first.
My alternative to these overly simplistic metrics is to look at what you're trying to achieve. Most social media programs are designed to engage customers -- or some audience -- in a conversation. The reason might be to learn more about what the customer wants, or it might be to improve the relationship, or to get ideas. Does  Dell's Ideastorm increase sales?
It probably stopped them from losing a few, but far more important is the ideas it creates. How do you put a value on an idea? Consultants and Ad Agencies do it every day. Will it increase sales. Maybe, someday. Can companies exist without them. Of course not.
So why the reluctance to measure what you're trying to achieve? The problem is that most organizations can't articulate what it is that they want to achieve. So my suggestion is pick one:
1. Improve relationships with my customers/employees (Metric: Improvement in Relationship Scores)
2. Generate traffic to a web site (Metric: Web Analytic improvement)
3. Persuade someone to do something (Metric: Increased donations, pledges etc.)

I'm sure there are lots more, but unless and until organizations get much clearer about what it is they're trying to achieve in social media. This will only be the year of Measurement theory, not practice.

Aaron Strout and I talk Social Media Measurement at Community 2.0

This is an interview I did back in April at Community 2.0 with Aaron Strout. Check it out

Avinash puts tells you how to pick a KPI

This falls under the "wish I'd written" it list: Pick One, Just One Web Analytics Report, Go! | Occam's Razor by Avinash Kaushik.