The Measurement Standard: Blog Edition

How to Measure YouTube and Other Video Sharing Sites

"Evolution of Dance" is the most watched YouTube video ever.

Set up a simple YouTube measurement program, decide if particular videos are really problems, and prepare for the worst case scenario.

by Peter Kowalski

Quick: How many people have watched the most popular YouTube video? Now, how many people watched the last Superbowl?

If you answered 87 million for the first and 98 million for the second, then you probably already know that video is the most popular type of shared content in Consumer Generate Media outlets. And with video technology becoming simpler to use and bandwidth growing, we can expect huge video sharing growth in the future. So, if you are not measuring YouTube now, then you will be sooner or later.

Actually, YouTube is only the most popular of at least 40 or so video-sharing websites. You will want to check most of these to see if the content is relevant to your organization or products. And maybe you will want to monitor some of them. As a practical matter, however, if you keep an eye on YouTube, you've pretty much got the field covered. Here's how to get started.

Welcome to YouTube

If you are absolutely new to YouTube, then first just go there and look around and watch some videos. Be amazed at a few of them. Be amazed at the inanity of many of them. Notice that some of the videos have been viewed 30 million times, and some have been viewed 30 times. There is even a movie being made about the YouTube video "Battle at Kruger."

Do a few searches. Do a basic search for your name and your teenage daughter's name and the name of your ex. Take a look at the Channels page, and at the TestTube page. And if you want to really get to know the place, create an account and upload your own video. Here's a hot tip: If you are going to upload that video of you doing the Wild Chicken Dance at the company party, don't put your own name on it.

Why Would You Want To Measure YouTube?

Now let's get a little more complicated. There are three basic reasons why you would want to measure YouTube:

1. You want to measure other peoples's videos about you, and need to decide if you should really care about them.
2. You are thinking of making a video, and you want to know best practices.
3. You already made a video and you want to know if it achieved your objectives.

We will leave #2 above to a separate article. But in the meantime there are some resources on the Web that might help. Just google "How to make a good YouTube video."

Measuring Other People's Videos About You, and Should You Care About Them?

Your Five Minute YouTube Measurement Program
First off, here is the quick and easy way to stay on top of YouTube. Do a search for your company and product names every day or every week. If you come up with nothing, great: You can tell your boss there are no videos posted, and you are monitoring YouTube. For each video you do find, decide: "Would we approve this video, or would we not approve this video?" And so put all the videos into those two files. Now you can report on YouTube activity whenever you need to.

(Note that these searches will pick up all text in the headlines, descriptions and user tags [it doesn't search the actual video itself]. Knowing user tag behavior for your organization is key on YouTube and elsewhere for this reason.)

When Should You Get Excited about a Video?
If you find an number of videos that mention or concern your organization or product, then you will want to do a more careful analysis. Do you appear in the title, description, tag, or just in a very quick glimpse? What are the positive points? What are the negative points?

Thumbs Up? The videos that are favorable can often teach you something about your products or image that you didn't know. Keep track of them, and see below for techniques to examine them more closely.

Thumbs Down? The only ones to worry about are the negative ones, and even then, don't go getting all excited until you examine the situation carefully. How many people are actually watching the negative video(s)? Did the video(s) get low ratings and bad comments, or good ratings and lots of links? Maybe a negative video actually proved strong brand loyalty when viewers gave it negative comments.

You can deal with the problem videos on a case-by-case basis, and/or compare them to see if there is a pattern or a larger problem that needs to be addressed.

How to Survive the Worst Case Scenario
So, it comes to pass that there is a video out there that says strongly negative things about your product or company and lots of people are watching it, commenting on it and passing it on. What to do? First, recognize that you have a full blown crisis on your hands; any video that gets really strong response indicates a major problem that needs serious attention. Respond to the video as you would respond to any crisis: Monitor the press, get your replies together and get your messages out. Details depend on the situation. Maybe you do a response video, maybe you do a press release. But the good news is you have plenty of opportunity to learn what people's problems are and, by analyzing comments, to learn specific ways to fix them.

Indepth YouTube Measurement: Basic Metrics

If you find a number of videos about your products or organization, you may decide you want to examine them more closely. If so, here are the basic metrics that you want to collect for YouTube or any video site:

Most important:

• How many times was a video watched?
• How many times was it rated and what were the ratings?
• How many comments did it get?

Also record:

• How many times has it been favorited?
• How many times has it been embedded?
• How many times has it been linked to?

See below for ideas on how to compare coverage to other media, and how to analyze comments. Be aware that it is often very complicated to understand the impact of a video that mentions your product or company. (For instance, KDPaine & Partners looks specifically at what characteristics of brand strength or organizational reputation are affected.)

So You Made a YouTube Video:
Did it Achieve Your Objectives?

The key here is that you want to be able to show that you communicated what you were supposed to. So, right from the start set your video design objectives so that the video expresses what your bosses want to communicate about the company or product. O.K., yes, it seems elementary, but if everyone agrees that your video says to the world what you (and your boss, board or board room) want to say about your organization, then you have a video you can measure. If there is an uncertainty about what your video should communicate, then all the metrics in the world are not going to tell you how effective it was.

Once your video is posted, you want to collect the basic metrics described above. Measure at standard times from posting: say, one hour, one day, a week, two weeks, a month, and then once a month thereafter. After six months your video has probably finished its YouTube lifecycle, but check at one year just to see.

Now compare your video's metrics to those of your competition, or to your videos in the past. Are you doing better or worse? Which stats are different and why? What do comments reveal about each video and what do they teach you about how to make your next video?

Comparisons to Other Media

Your YouTube metrics will be put in context if you can compare them to other media. So, for instance, you can provide some meaning by comparing number of views to newspaper circulation, or TV viewership. It's not exactly apples to apples -- you can make a case for YouTube views as being much more powerful than print or TV impressions, for instance -- but it is a place to start.

For the Advanced Student: Comment Analysis

If you have plenty of comments to analyze, then things get much more interesting. If a viewer leaves a comment, you can argue that the video had a greater impact on that viewer than did some other video on which the viewer was not moved to comment. So for two videos with equal viewership, the one with the most comments can be said to have the greatest impact.

You will find, however, that many comments are not relevant to your objectives or relate in any obvious way to your product or organization. So, analyze comments based on to what extent they achieve your original objectives. Also check to see if they indicate any other important sentiments or reactions that you did not anticipate:

• Positive or negative responses to your organization, brand or products
• Demonstrations of increased brand strength of loyalty
• Unusual engagement with product or company

More advanced analyses, like that those available from KDPaine & Partners, include teasing out exchanges of particular sorts, debates on particular topics, and threads on topics of choice.

How to Impress your Boss and Justify Your Existence, Video-wise

Here are two quick ways you can demonstrate that your video has been particularly effective:

1. Compare your video against other media (ads, newsletters, etc.) for transmission of key messages or brand reputation.

2. Use comment analysis to show that people are interacting with your brand(s) and are more engaged with your company and products. Remind your boss that non-social media -- like advertisements -- can't encourage or measure that sort of engagement at all.

Extra Credit: Do focus group or survey analysis of video viewers to determine if they, after watching the video, are more likely to buy your product or visit your theme park or whatever. If you survey all customers, ask "Do you watch YouTube videos, and did you see ours?"

Peter Kowalski is Director of Research Strategy at KDPaine & Partners. His research interests include international inter-media influence, agenda setting and network analysis. He holds a Bachelor of Arts in Public Relations from the USC Annenberg School of Journalism.

Measuring Naked Relationships: How to evaluate the impact of a social media program on your public relationships.

Your step-by-step guide to using relationship metrics to evaluate the success of your social media program.

by Katie Delahaye Paine

I've talked a lot here -- also in my speeches, in my blog, and of course in my recently published book Measuring Public Relationships -- about the importance of measuring relationships. I believe that without factoring in the impact that your social media program has on your relationships (with employees, community and constituencies) you are undervaluing your efforts.

So, how does one actually measure relationships? Well, now that you asked, my book explains this with regard to most types of public relations programs. Social media being all the rage right now, I thought it would be appropriate to provide a step-by-step guide as to how to actually do it for social media programs.

Whatever type of program you want to measure, the basic technique is similar: You conduct a survey of your audience using a special set of questions designed to specifically measure the different components of relationships. You do this before and after your social media program is in place, and you do it for your organization and as many competing organizations as you can afford to. Then you compare the data before and after, and between your organization and the others, and then you know where your relationship with your audience stands and where you need to go.

The Grunig/Hon Relationship Research

Before we get to the nuts and bolts, here's a bit of background. A decade or so ago, University of Maryland Professors Jim and Laurie Grunig and Linda Hon synthesized communications and sociology research and theory into a paper published by the Institute for PR called "Guidelines for Measuring Relationships in Public Relations." Their feeling was that, amidst all the brouhaha about quantifying public relations, we were forgetting the essential truth that PR was about relationships. And so you needed to measure the impact that your efforts were having on those relationships.

Their research isolated six fundamental components of relationships -- Trust, Satisfaction, Commitment, Control Mutuality and Exchange/Communal -- and they designed and tested 75 statements to measure those components. These statements are of the sort: "This organization treats people like me fairly and justly," and, "I feel a sense of loyalty to this organization." You can see them all, and copy them for your own use, on my blog or in the Grunig/Hon paper. For an example of these statements successfully used to measure public relationships in a non-social media context, see this research by Forrest W. Anderson and Paul Raab.

Nine Steps to Measure Your Social Media Relationships

Before we start, remember that to isolate the effect of your social media program, you must begin your measurement before you launch that program. Then you'll have a benchmark to compare against later: Before Social Media vs. After Social Media. Without a Before benchmark you won't know how your social media program has changed your relationships.

Of course, you can always begin measuring after your social media program has started, and by doing so you will be able to do ongoing evaluation of your relationships. Which is a good thing. But a better thing is to isolate the effect of social media, and to do that you must compare before and after.

So use Steps 1 through 9 below to survey your audience before your program begins, and thus establish your benchmark. Then begin your social media program, and after an appropriate time, say three or six months (depending on your situation), re-survey to see what has changed.

To be sure that whatever change you find is solely attributable to the social media program, you must hold constant any other PR programs that affect your social media audience's perception of your organization. Yes, it's tricky, and it's not always an ideal world. But if you are trying to measure the effect of your new blog at the same time as the Promotions Department decides to give away A Free Cadillac With Every Purchase, then you can kiss your results goodbye.

Step 1: Define the audience for your social media program.
Social media is about conversation and engagement, so decide with whom you want to converse and engage. If you're starting an internal blog, your audience is your employees. If you're starting an IdeaStorm-type customer community, your audience is anyone likely to buy or recommend your product. If your mission is advocacy, your audience might be voters, politicians or industry influencers.

Step 2: Get a list of your audience's email addresses and/or phone numbers.
To get a representative sample you will need at least 500 names for each organization you are testing (more on that later). If you already have a list of your members, subscribers or customers, then you are ahead of the game.

If you have to purchase your list, then potential vendors vary with the type of sample you're looking for (mail, phone, web). Most lists are sorted based on demographic or title data. There are a lot more resources out there for mail addresses and those resources do not necessarily need to be survey sample companies. For email addresses, some reputable firms include Survey Sampling, e-Rewards and Zoomerang.

Step 3: Pick a survey methodology.
The Grunigs would recommend in-person surveys for the best results, but most researchers find that to be very expensive. Phone surveys are fast and provide very accurate results, but again, depending on the audience, may be cost-prohibitive. Email surveys are an increasingly accepted methodology, and for social media can be appropriate and highly reliable, since, presumably, your audience is all on email.

You may be able to piggyback on an existing survey going out to your community. If marketing, customer satisfaction, business development or anyone else in your organization is doing a survey, see if you can add a few of the Grunig/Hon statements to it.

Step 4: Select which of the Grunig/Hon statements are most appropriate to your organization.
You can probably only impose on someone for 7-10 minutes of their time, so you need to pick which statements you will include. Grunig and Hon suggest that if you want to shorten the survey, you use only the boldfaced items. Not all statements are appropriate for all organizations, so pick and chose the ones that will be most meaningful to your audience.

Step 5: Prepare your survey.
If you are using an electronic survey system like Survey Monkey, you need to create an introductory screen that explains what you are doing and how the scoring works. For instance:

In order to better understand the needs and perceptions of our marketplace, we'd like to ask you some questions. Please tell us whether you agree or disagree with the following statements as they apply to X company/organization.

Explain that 1 means "totally disagree" and 7 means "completely agree," and give them an option for "no opinion." You also need to ask them the same questions about a competing company or organization, so you have comparable data on the competition to compare to.

Step 6: Send out the survey.

Step 7. Resend the survey.
Depending on your audience it may take several tries and an incentive to get sufficient responses (I'll do just about anything for an Amazon or Starbucks card). How many is sufficient? Well, it depends on how you plan to break down your analysis, but in general plan to resurvey until you get at least a 10% return. If in doubt, talk to your local survey expert.

Step 8. Analyze and learn from the results.
Calculate a mean score for each relationship component. There are both positive and negative statements in the survey instrument, so make sure you take that into account. Compare your mean on each score to the competition. (And of course to your earlier survey results, if this survey is not the benchmark.)

Step 9. Implement your program and measure again.
If this is your pre-social media program (benchmark) survey, then implement your program now, and measure again in six months. Or, if your program has been running for a while and your analysis indicates you need to make changes, then make the changes now and let them work on your audience for six months before you measure again.

Good luck, and let me know how things go.

How to Measure Relationships with Voters, Legislators and Other Political Constituencies

by Katie Delahaye Paine

Ah Springtime! When a young man's thoughts turn to love, or basketball, or, if you live in New Hampshire, politics. Given that New Hampshire has the third largest legislative body in the English speaking world (after the British Parliament and the US Congress), and that once a year in March a million or so average citizens get together in town meetings to determine the fate of the state, you can understand why the term "March Madness" takes on a whole new meeting up in these parts.

When measuring relationships with voters, as with any measurement program, the type of system you put in place depends on your objectives and your role. Below is a list of tools you need for each type of program. One important point first...

Timing is more critical than usual
Because the length and success of voter measurement programs are so strictly defined by the election cycle, you can't afford to waste time or opportunities. Probably the most important element in any political measurement program is to have the data on hand when you need it. And you need it whenever there's a decision to be made about how much money to spend or what tactics to use. You don't need it after the votes have been counted.

Measuring PR Efforts on Behalf of a Candidate

My company, KDPaine & Partners, measured political PR for one of the country's best political PR pros, Doug Hattaway, former spokesperson for Al Gore and Tom Daschle. Doug was working on Chellie Pingree's effort to unseat Susan Collins as Senator from Maine. What Doug needed was a measurement system that would tell him which tactics were working or not working in a campaign. The challenge was to do it fast enough so the data would be useful, and cheap enough so the campaign could afford it.

The good news about working with political campaigns is that volunteers are generally plentiful, and in this case the volunteers were charged with collecting the clips from the local publications. If you don't have volunteers to do the clipping, you need to hire a local clipping company or someone like Bacons or Burrelle's to collect the clips for you. Small-town newspapers are just too important to miss. Electronic data aggregators like Nexis and Factiva do not work on local or statewide campaigns; they do a terrible job collecting clips from anything smaller than the Boston Globe. Even Cyberalert and Custom Scoop miss far too many of the small town weeklies.

One we had the clips about both Pingree and Collins in hand, we began to analyze them for positioning on key issues, tone of coverage, type of article, subject of the article and who was quoted. We compiled them monthly and issued a report that included charts like this (click the charts to see them bigger):

 

 

 

 

We could quickly identify the tactics that were most successful. For example, a visit by Hilary Clinton in support of Pingree was far more successful in garnering visibility for Pingree than any other action. However, Pingree's visit to Washington in support of her health care reform legislation, while it didn't get anywhere near the visibility, was more successful in positioning her as "the health care candidate."

The analysis also revealed weaknesses in Pingree's campaign and strengths in Collins' campaign. While Pingree was almost entirely focused on her health care message, Collins was making points with the voters on the environment and campaign finance reform.

Additionally, the research enabled the campaign to track the correlation between visibility and contributions. By tracking Pingree's share of exposure over time against the contributions over time, the campaign could determine the level of additional exposure necessary to generate the requisite contributions.

In the end, the earned media that was generated by Hattaway's PR efforts narrowed incumbent Collins' lead substantially.

Measuring Lobbying Efforts

We ran into a government relations person for a major organization who wanted to know how to measure the impact of his efforts. He was trying to get a bill passed that would be highly advantageous to his company. We began to design his "dashboard," and asked him what success meant. He got as far as "If the bill passes, I'm successful." We pointed out that if the bill failed it would be too late to do anything about it.

So what he really needed was to monitor the tone of conversation around the bill in question so that he could tweak his PR program according to its progress. We recommended that he establish a system that gives him feedback all along the way.

There are several ways to do that. The easiest method is to track who is saying what to the media or in local speeches about the bill. Are key influentials supporting it or trashing it? Or is no one talking about it at all? More important to track, however, is the bill's progress through the legislative process. Is it being heard, tabled, or moved? Who are the cosigners to the bill and are they actively promoting it? Is the opposition gaining or weakening? What do bloggers say? What's being discussed in news groups? Are there any list serves you need to track?

Some of these data points can be gathered via media analysis, others should be part of your regular legislative tracking process. Still others may require a poll of the constituency to determine the level of support, and the perceptions of the voters and/or legislators.

If at all possible, a once-a-year survey of your key legislators is recommended to test and evaluate the health of the relationship between your organization and the elected or appointed officials you are striving to measure.

 

Measuring your Relationship with Publics, Both Active and In-Active

Your constituencies come in several forms, from students to senior citizens. You should measure relationships with as many different stakeholders or constituencies groups as you can afford. It may be useful to segment the different publics by gender, age, length of time in the area and political leanings so you can identify any pockets of opportunities or threat within the community.

Clearly active stakeholders who are most likely to protest, boycott or otherwise cause trouble are the most important to understand. However, inactive stakeholders that are clueless about your programs and therefore don't care and won't get engaged can be just as dangerous.

We recommend using the Grunig relationship model to test your relationships and in particular the level of trust that each public feels towards you. Trust is a key component of success in any political campaign. If it's not there, or if it is recently lost, you will have a significantly harder time achieving success. Alternatively, if your constituents trust you, as do those of Alan Greenspan or John McCain, you have a lot more leeway with those publics. Therefore it is critical to establish a benchmark level of trust to begin with. Subsequently you should conduct regular trust/relationship measurement studies to gauge the level of trust and engagement over time. Conduct these studies as often as possible so you can tie any changes in the trust and relationship scores to actual actions you may have taken.

The Ultimate Measure: Votes

There's a ton of great data in any set of election results. The challenge is to use it to make more informed decisions. When I ran for town council here in Durham five years ago, I knew that my efforts to go door-to-door in specific communities paid off since the number of votes I received from those communities far exceeded my expectations. I also knew that my signage (bright purple and white with the slogan "No Paine, No Gain") was successful since I had the highest name recognition, and ultimately the highest vote count of all the candidates.

In larger elections, there's far more granular data to be gathered, and much of it is readily available online. Using this data you can frequently identify the specific areas or pockets or demographics you've targeted and thus measure your success in getting those groups to vote and/or vote your program or candidate.

The Media Integrity Index

A call for weighting media coverage based on trust and credibility.

by Katie Delahaye Paine

In this article I suggest that media analysis programs would more accurately reflect articles' influence on readers if the integrity of the media outlet was taken into account. This could be achieved by developing a standard measure of media integrity which would used to rate different media outlets for different shareholder groups. The resulting Integrity Index would be used to weight coverage, thereby achieving a more accurate measure of media coverage's impact on consumers.

To understand why an Integrity Index is needed, consider:

Item #1
Why measure what no one believes?

Mazen Nawahi, in an impassioned speech at the 2008 Dubai Measurement Summit, raised the issue of journalistic integrity and the degree to which the integrity of a particular media outlet should be accounted for in any measurement program. Put another way, if:

1. Everyone knows that a given media outlet is going to print whatever a company sends it because they always do or because they're a major advertiser, and
2. Everyone also knows that they will never print stories that don't agree with the governing dogma,

then why would anyone believe anything that was printed?

And if all your media coverage in a particular outlet lacks credibility, why would you include it in your measurement results? For example, in 22 years in business I have yet to have a client want me to include The Weekly World News in its media list. So how do people justify measurement programs that include stories that do nothing to achieve their goals?

Item #2
Credibility is in the eye of the beholder.

Suppose a highly credible blogger gets into a mud-wrestling contest with the blogging equivalent of a pig. The contest attracts a lot of attention, a lot of arguments fly back and forth, and a lot of dirt gets thrown around. Now for the people that are into farm animals, the arguments of the pig might be very credible. However, people interested in learning something of value professionally will no doubt pay more attention to the words of the credible blogger. But the real question is: Do you weight coverage of them both equally in your media analysis reports?

How to correct for influence?

Taking examples of this sort into consideration, it is easy to see why some sort of authority or influence weighting of coverage would be of value. There has been a fair amount of research done in this area. For several years, Angie Jeffrey at VMS has been studying various weightings of media coverage to determine which factors most directly impact sales. Her research first found that share of discussion was effective, but she postulates that advertising rates are more telling because they reflect the importance of the various media. (See this paper and this paper.) I think it's a step in the right direction, but I generally frown on the use of advertising rates (and of AVEs), so I'm skeptical.

The Media Integrity Index: Why not rate coverage on integrity as rated by stakeholders?

I suggest that it's not the ad rate but the integrity of the publication that most determines consumer credibility and thus drives consumer action. And I'm suggesting that to rank a media outlet in importance, we should ask our stakeholders how they perceive the media outlet in terms of integrity.

I raised the question on Twitter and got some interesting responses:

• "It's all relative... What is good for one is bad for the other. It has to be specified for a purpose, customized."

• "Integrity or perceived integrity? Seems there's a difference there that's highly subjective." --Ryan Anderson

• "Integrity is the measure of perceived relationship (believability) of one-to-many listeners. So measure relationships. I would suggest looking at the way political integrity is measured. It would be a perceived metric, measured by polling audience. --Videodred

• "What's the goal? What's the client want it to say? In the case of journalists, what's the relationship?

• "Integrity has to be measured parallel to influence as audience determines credibility of source." --Mike Maney

So in one way, my proposed Integrity Index has to start with the goal of the coverage. What is it that the company or organization is trying to accomplish? Consider it this way: If you don't care who or what your brand is associated with, and you want exposure pure and simple, then you don't care about the integrity or credibility of whatever media outlet is talking about you. On the other hand, if you're trying to establish a reputation, or build brand loyalty and trust, or trying to reach an audience with certain media preferences, then the credibility of media to your audience should matter a great deal.

How to determine integrity?

So how do we determine which media is trustworthy and credible, and which journalists have or do not have integrity? The wrong way is to look at a list of media and assign weights or values to each one yourself. What you think doesn't matter. All that really matters is what your customers or stakeholders or members or constituents think. So you need to ask them. The best way to do that is to use the Grunig Relationship Instrument.

Now you probably don't need to ask them about all 150 publications on your media list. Remember: Never ask a question about something that can't be changed. If The New York Times is what your boss' boss reads every morning, there's no chance in hell you'll ever take it off you top tier media list. Even if your target audience is nine-year-old girls. So start with a list of suspect publications, blogs or any other type of media outlet.

Remember that different stakeholder groups may rate a given media outlet differently. As I pointed out above, integrity is not a fixed standard. For instance, we would expect Republican viewers to rate Fox News and The New York Times differently than Democratic viewers. It is conceivable that you will want to generate different Integrity Indexes for different stakeholder groups.

Below are some survey questions adapted from the Grunig Relationship Instrument. You may wish to develop your own, based on the specific components of relationships you wish to measure (refer to this paper). Ask your stakeholders whether they agree or disagree with each statement as it pertains to each media outlet, then use the responses to rate the outlets.

1. This media outlet treats people like me and organizations like mine fairly and justly.

2. Whenever this media outlet makes an important decision, I know it will be concerned about people like me.

3. This media outlet can be relied on to keep its promises.

4. I believe that this media outlet takes the opinions of people like me into account when making decisions.

5. I feel very confident about this media outlet's skills.

6. This media outlet has the ability to accomplish what it says it will do.

7. Sound principles seem to guide this media outlet's behavior.

8. This media outlet does not mislead people like me.

9. I think it is important to watch this media outlet closely.

10. This media outlet is known to be successful at the things it tries to do.

Industry-wide integrity standards?

Now, the logical question is, "Why aren't we doing this as an industry?" Shouldn't we be factoring in credibility based on some industry-wide standard? It certainly is a more accurate weighting factor than simple eyeballs or ad cost. But the reality is that your stakeholders aren't going to be identical to my stakeholders, and what matters is how a significantly valid sample of your stakeholders feel. It would be awfully complex and difficult to set industry-wide standards for many different stakeholder groups.

On the other hand, if the PR industry wanted to take on a project to try and accurately weight publications based on their integrity, the world would most definitely be a better place. And it is not far-fetched to imagine an industry-standard Integrity Scale or Survey that could be used to determine the Integrity Index for different stakeholder groups.

Ten No-Cost Ways to Measure Online Engagement

by Katie Delahaye Paine

1. Set up Google Analytics on your blog to find out how many repeat visitors you have. How many pages per visit do they check out? How many go back more than 3 times a week? How many go back and spend more than a second or two on the site?

2. Post a vizu poll on your blog and see how many people respond.

3. Go to xinure and enter the URL of your choice to find out how well it is doing in search engines, links, social bookmarks and a whole bunch of other stats.

4. With many of the leading blog providers like TypePad, check your stats to find out how many people have subscribed, and how many visits per day you're receiving.

5. What's the Conversation Index (the ratio of postings to comments)? In the blogosphere any comment is a good comment because it shows that people are engaged enough in what you are saying to take the time to respond.

6. If you have posted a video on YouTube, or a photo on Flickr, check to see how many people have rated it, and/or commented on it.

7. If you have a presence on Facebook, how many people have joined your group?

8. Ask a question on Facebook and see how many people respond.

9. If you're on Twitter, how many followers do you have? More importantly, how many responses do you get when you ask them a question?

10. Set up a Technorati, Sphere or Icerocket search to find out whether people are writing or talking about you.

The Interpretation of Dreams: Dreams as Public Relations Measurement -- I Dream of Hillary, I Dream of Barack

Many public relations professionals find that media analysis or survey research are tools sufficient to measure their programs. The more adventurous bring in external data and more esoteric measures of success like donations or number of new memberships or even lives saved. In politics, KDPaine and Partners has recently used YouTube video counts to predict primary results, and our Ms. Paine has ruminated on the possibility of using the number of political lawn signs as an election predictor. Not until now, however, has anyone used dreams to measure political progress.

In this week's New Yorker, Ben McGrath writes about Sheila Heti and her new blog "I Dream of Hillary... I Dream of Barack." This blog, a repository of reader-submitted dreams about the candidates, can be interpreted as a rough poll of interest in those candidates. As Mr. McGrath says, "...what if the recurrences of Presidential candidates in people's dreams were meaningful in the aggregate?" As of the writing of the article, "..Obama's edge in the over-all dream count... was roughly equivalent to his lead in the latest Gallup poll."

Ms. Heti's blog has been so successful she plans to open an I Dream of McCain section, so in the future we will have bipartisan dream data to go on. --Bill Paarlberg, Editor, The Measurement Standard

Why Public Relations Measurement Is Like Phrenology, or... Just because it is easy to measure, doesn't mean it's the best thing to measure.

PR measurement and marketing measurement often find themselves in a situation analogous to the 19th-century practice of phrenology. Way back then, some people used to think that the shape of your skull indicated your personality or intelligence. It was a very easy thing to measure, so they did. And they hung on to it as a serious area of study for much longer than they should have, because it was far more difficult to measure personality or intelligence by other, more effective, means.

Click-throughs aren't quite as useless as bumps on the head, but still... In the news today is a study by Starcom USA, behavioral targeting firm Tacoda, and comScore that suggests that clickthroughs might not be a very effective measure of the usefulness of online ads. See here for the news and here for Sam Harrelson's ReveNews discussion.

The obvious point here is that clickthroughs, while easy to measure, probably are not very effective at measuring what we would like to measure. The more interesting point is: What aren't we measuring what we really want to know?

I have noticed that PR and marketing metrics are often determined by what technology makes easily possible, rather than by what we would really like to know. Thus people tend to measure what technology drops in their laps (the easy things like click-throughs or AVEs) rather than something more relevant but difficult to measure (like, "Did the ad with the red type result in more sales among wealthy women?"). Then at some point they realize that their easy-but-less-relevant metric is not very effective, and then they spend a lot of time asking why the metric doesn't work.

What they should be asking themselves is: "What made us think that that metric would work in the first place?"

Instead of focussing on what technology is available (in this case click-throughs), wouldn't it be more productive to think about human thought and behavior first? (I've written about this before: "Measurement's Empty Head: Measurement ignores the most complex part of PR.") First think about what it is that might make PR work -- why it affects people's thoughts and behaviors -- and then decide what might make a good metric. First determine what we really want to measure (regardless of the technology or data available), then figure out a way to measure it. -- Bill Paarlberg, editor, The Measurement Standard

KDPaine & Partners Releases New Book on Measuring PR and Corporate Communications

Your Measurement Reading List

Measuring Public Relationships Makes Measurement Simple

Measuring Public Relationships, The Data-Driven Communicator's Guide to Success by Katie Delahaye Paine, 228 pages, paperback. ISBN 978-0-9789899-0-3. Order online or by phone: 603-319-1047.

BERLIN, NH, Dec 12, 2007-- The first how-to book on measuring relationships for public relations professionals was released today by KDPaine & Partners, the Berlin, NH-based leader in social media and public relations measurement. Measuring Public Relationships, The Data-Driven Communicator's Guide to Success was written by KDPaine & Partners' CEO, Katie Delahaye Paine.

Paine takes a "cookbook" approach, providing specific steps to measure all forms of public relationships, from social media measurement to tracking relationships with local communities, industry analysts and social networks. It relies heavily on the relationship theories of James and Laurie Grunig and Linda Hon but translates the theory into an easy to read practical guidebook. The book has been well received by academics and professionals alike who have called it "the must-have practical guide to hands-on PR measurement." Emphasizing the role and evaluation of relationships, this book provides every public relations professional with step-by-step research procedures for measuring programs, improving results, and managing relationships.

"It really is an excellent book," said Professor James Grunig, calling it "lively, engaging, accessible, wise, and candid... a remarkable compendium..."

"Katie Paine is the most practical consultant and educator on public relations research that I know. She is brave enough to link PR to return on investment demands and smart enough to provide marketers and PR professionals with a wide range of useful research tools. If there is cost effective way to do research, Katie Paine will tell you about it," said Clark Caywood of the Department of Integrated Marketing Communications at the Medill School at Northwestern University.

"By force of prose and experience, Katie Paine level-sets the practical and now critical disciplines of media and communication measurement. A student of Grunig theory and a champion of the almighty relationship, Paine plops on every communicator's desk the best new reference for measuring the intangible. Read it, and you'll count your successes," said Alan Kelly, CEO & Founder, The Playmaker's Standard, LLC and author of The Elements of Influence. Adding that it's "...the best new reference for measuring the intangible."

Measuring Public Relationships is available at a 25% discount until January 31st, 2008 when ordered directly online or by calling the KDPaine & Partners' office at 603-319-1047.

How To Evaluate Events and Sponsorships

Sponsorship Measurement

Six basic steps, five final lessons and four additional resources.

By Katie Delahaye Paine

"Live from the KDPaine & Partners/Verizon Wireless Arena we have the KDPaine & Partners' Battle of the Sponsorship Suppliers, underwritten by Orville Redenbacher's Microwave Popcorn, the official snack food of KDPaine & Partners; by Starbucks, the official fuel of KDPaine & Partners; and with additional support from Staples, the official office supply company; FedEx, the official overnight delivery system; Diet Coke, the official diet drink; and Catalina, the official pet of KDPaine & Partners."

Perhaps not these exact words, but something like them has passed your ears at least a dozen times in the last few years. From Little League teams to the World Cup, from naming arenas to naming road races, sponsorship of events is now one of the marketing methods of choice. As communications is increasingly impersonal and electronic, the chance to actually touch, feel and experience the brand is ever more important. Increasingly, purchasing is neatly divided into two categories -- on-line and experiential. More and more decisions are based on familiarity with the brand and ease of purchase.

Now, I'm an avid Amazon fanatic, but I still go into Borders' brick and mortar store and end up buying something. Why? Because I get to touch and feel and browse and listen to products. If I need something in a hurry, I go online. If I want "retail therapy," I go to Borders. That's experiential marketing. Marketing that transcends the product and the media and allows direct interaction with the brand.

A 2006 study by Jack Morton Worldwide showed that live event marketing -- experiences where consumers interact with products, brands or "brand ambassadors" face-to-face -- are among the most effective ways to influence coveted consumer audiences. The study, an online survey of 2,574 consumers ages 13-65 in the top 25 US markets, confirmed that this increasingly important marketing medium resonates strongly across all demographic and product categories.

Which is why total spending on experiential marketing is expected to reach $15 billion by the end of 2007, an 11% increase over last year, according to the latest IEG research report. Companies are now allocating on average 17% of their budgets to sponsorships, compared to just 13% last year. However, 76% of companies spend less than 1% of their total sponsorship budgets on research, and in a shaky economy, if you don't measure it, you can't show ROI. And if you can't show ROI, chances are good that the program will be killed.

Another factor in the measurement of experiential marketing is that simply "reaching large numbers of eyeballs" is no more relevant a goal than is "increasing market share in the buggy whip market." Therefore, new alternative metrics -- like relationship measures, engagement indices and cost per touch -- are increasingly demanded.

We suggest that there are six basic steps to developing a solid measurement program for your events and sponsorships.

Step 1: Be clear about what outcomes your communications program is designed to achieve.

As with any effort, you can't start to measure success until you know what success means for you. For any given event the objectives might be:

• Percent of attendees more likely to purchase
• Percent of attendees remembering the brand
• Number of qualified sales leads generated
• Conversion rate of attendees
• Total potential sales = [number of attendees] x [conversion rate] x [average sale]
• For press events: number of key editors and analysts attending, percent of attendees writing or quoted on the issue, total exposure of key messages in resulting press.

Each objective, of course, requires a different type of measurement. Some types isolate the impact of PR from other communications efforts better than others. And the best objectives are specific and measurable.

Step 2: Determine criteria -- quality as well as quantity

Once you've agreed upon your objectives, establish the specific criteria of success that you will measure. If your objective is awareness, the criterion might be the percentage increase of unaided awareness of brand or product. If your objective is to sell product, the criterion might be the incremental sales after a particular PR or promotional program took place. Consider those numbers that best reflect the health of your business, or that best represent characteristics that most affect your business:

• Increase in awareness
• Increase in preference
• Increase in purchase intent
• Increase in customer loyalty
• Percent improvement in customer experience

To a certain extent, your choice of criteria is dependent on the type of event you are evaluating. If the customer experience you are measuring takes place at a trade show or exhibit booth, you might choose as key criteria the percentage of new visitors or the cost per minute spent with a client in the booth. If the experience takes place at an event such as a concert, you need to count how many people were exposed to your brand or the brand experience you were offering. Don't trust the promoter's numbers; do your own counting. Here's why:

I once met with a major auto manufacturer who wanted to measure the effectiveness of different sponsorships. It was spending many millions sponsoring car races, golf tournaments, antique car auctions and a variety of other events. According to the event organizers, attendance and therefore results were always better than the year before. But then I asked them what they were trying to achieve by these efforts. The first response was essentially: "We are always a major sponsor of these types of events." After about two hours of discussion, we agreed that the business objective was to drive potential customers into dealer showrooms.

We began a series of surveys at each event to determine first if the attendees remembered the sponsors, and secondly if they were more or less likely to test drive and/or buy the sponsor's car. We collected names at the event itself and called attendees two weeks after the event. Our results showed that on average 50% of all attendees were more likely to test drive a "sponsor" vehicle after attending the event.

After we had measured three different events, we were able to compare and contrast the cost effectiveness ($$ per person reached) of different events. The results enabled the sponsor to better understand the ROI from each event. By looking at the percent of people more likely to go to a dealership, they could determine the number of potential buyers. By subtracting the cost of the sponsorship from the profit and the projected number of car sales, they determined a projected ROI from the event. These results led the sponsor to dramatically alter their sponsorship strategies.

It's important to look beyond simple quantitative data, especially if your objective is exposure, to assess the quality of your communication. When companies clearly define their objectives it becomes a relatively simple matter to define a set of criteria against which to measure the relationships the company establishes with its constituencies in different events. Then it's a matter of comparing and contrasting results consistently cross a number of different events to ensure that the company has the most effective sponsorship program possible.

Step 3: Decide upon a benchmark

The key point to remember about any evaluation program is that measurement is a comparative tool. You need to compare one set of results to something else. The most meaningful comparisons are between different events, or between you and your peer sponsors at one event. If your fellow sponsors aren't direct competitors you can learn a lot by sharing information and determining what best practices really moved the needle.

4. Select a measurement instrument

Do not rely on data gathered at the event itself. It is rarely useful or actionable. Consider the following instead:

• Post-show awareness survey
  In our opinion, the most reliable way to measure relationships with your customers is to collect names at the event and interview them afterwards to see what they can remember of about the experience. Do they remember being in your booth? Did they even know you were a sponsor? Did they remember your brand, and the brand benefits or brand positioning you were trying to convey? Did they leave more likely to purchase or to recommend?
• Web analytics
  Because customer actions are so easily trackable from website activity, more and more companies are relying on web analytics to measure the impact of sponsorships. Determining not just the increase in traffic but the actual conversion rate is a relatively easily task using Google Analytics, WebTrends or Visual Sciences.
• Cost per touch
  As a way of comparing different marketing opportunities, some organizations are replacing the old "CPM" or cost per contact with Cost per Touch. In other words what does it cost to make contact (by phone, email, in person, or however you define it for your research) with one potential customer?
• Cost per minute spent with prospect
  If the purpose of the event or sponsorship is to collect sales leads you may want to calculate the value of your effort in monetary terms. If you have a product or service that has a long sales cycle and needs time to explain, this is a particularly good way to evaluate marketing efforts. Several years ago, the pharmaceutical company Glaxo figured out that it cost them $300 to get a salesperson into a doctor's office for about five minutes. That's $60 per minute spent in front of a prospect. Now suppose that by giving a speech you get 60 minutes in front of a qualified audience of 100 people. That's $0.83 per minute spent with each potential prospect. Relatively, this appears to be very efficient.
  
  Once you have expressed it in that form, you can compare value to other marketing efforts. So, for example, if a 20-second underwriting spot on NPR costs $5,000, and assuming it communicates your key message and reaches 500,000 listeners, that's $.01 per opportunity to hear a key message and $0.03 per minute spent with a prospect.
  
  You should also monitor the media coverage around the event to see if additional messages are communicated to your audience via the media surrounding the event. You need to look at the reach of the publication and also determine whether the article contained a key message. You can then take the cost of the program and divide it by the total number of opportunities to see or hear a key message to determine the cost per opportunity to see a key message. That way you can decide if the event is more or less efficient than other programs at getting your messages across and achieving your goals

A relationship study

You can also use the Grunig relationship survey instrument (downloadable for free from the IPR web site) to test the health and strength of your customers' relationships to your brand. The Grunig instrument has been thoroughly tested and shown to be an extremely effective measure of how customers perceive their relationships with an organization.

The key thing about whatever measurement tool you use is to make sure that you are tying results back to the desired business outcome. A number of years ago, Country Music Television (CMT) hired tractor trailers to set up country music events in Wal-Mart parking lots. Their first measure was to ask the truck drivers how many people showed up. Realizing that those answers didn't tell them anything about what those potential customers were actually doing when they went home, CMT decided to hire my company to do follow-up studies with the attendees. The stated purpose of the events was to convince attendees to call their local cable company and request CMT or to take some other action on behalf of CMT. CMT used a sweatshirt giveaway at the events to collect names and addresses of attendees, whom we called two weeks after the event. We agreed that the common criteria against which we should measure all events was an ad hoc "level of engagement." Specifically, we asked people to rate how much the event experience affected their relationship with CMT, and then we established the mean level of interest for each event. We further defined the success criterion as the percentage of people willing to take action on behalf of CMT. Our surveys showed that 93 percent of attendees were willing to take some action and 89% were willing to make a phone call to their local cable company. We then compared results between different cities so CMT could determine where to expand the program the following year.

Five final lessons for sponsorship measurement:

1. You become what you measure, so be very clear internally on the objectives of your sponsorship or event that you will measure.

2. Match the event, the speaker, and the metrics to the objectives.

3. Document any immediate or tangible results: media coverage, phone calls or inquiries, messages delivered to key audience, etc.

4. Share and leverage results internally.

5. Repeat your measurement consistently.

Other resources:

www.instituteforpr.org offers several white papers on measuring events, including:

• Measuring the Effectiveness of Speakers Programs
  By Marianne Eisenmann and Katie Paine
• A Guide for Measuring Event Sponsorships
  By Bruce Jeffries-Fox
• Guidelines For Measuring Relationships in Public Relations
  By Linda Childers Hon and James E. Grunig

My just-published book Measuring Public Relationships includes at least a couple chapters that bear on event measurement. You can purchase the book from the KDP&P's office at 603-319-1047 (25% off the list price of $29.95 until 1/31/08), or you can download the pre- press draft at no charge right here.