The Measurement Standard: Blog Edition

Measuring Naked Relationships: How to evaluate the impact of a social media program on your public relationships.

Your step-by-step guide to using relationship metrics to evaluate the success of your social media program.

by Katie Delahaye Paine

I've talked a lot here -- also in my speeches, in my blog, and of course in my recently published book Measuring Public Relationships -- about the importance of measuring relationships. I believe that without factoring in the impact that your social media program has on your relationships (with employees, community and constituencies) you are undervaluing your efforts.

So, how does one actually measure relationships? Well, now that you asked, my book explains this with regard to most types of public relations programs. Social media being all the rage right now, I thought it would be appropriate to provide a step-by-step guide as to how to actually do it for social media programs.

Whatever type of program you want to measure, the basic technique is similar: You conduct a survey of your audience using a special set of questions designed to specifically measure the different components of relationships. You do this before and after your social media program is in place, and you do it for your organization and as many competing organizations as you can afford to. Then you compare the data before and after, and between your organization and the others, and then you know where your relationship with your audience stands and where you need to go.

The Grunig/Hon Relationship Research

Before we get to the nuts and bolts, here's a bit of background. A decade or so ago, University of Maryland Professors Jim and Laurie Grunig and Linda Hon synthesized communications and sociology research and theory into a paper published by the Institute for PR called "Guidelines for Measuring Relationships in Public Relations." Their feeling was that, amidst all the brouhaha about quantifying public relations, we were forgetting the essential truth that PR was about relationships. And so you needed to measure the impact that your efforts were having on those relationships.

Their research isolated six fundamental components of relationships -- Trust, Satisfaction, Commitment, Control Mutuality and Exchange/Communal -- and they designed and tested 75 statements to measure those components. These statements are of the sort: "This organization treats people like me fairly and justly," and, "I feel a sense of loyalty to this organization." You can see them all, and copy them for your own use, on my blog or in the Grunig/Hon paper. For an example of these statements successfully used to measure public relationships in a non-social media context, see this research by Forrest W. Anderson and Paul Raab.

Nine Steps to Measure Your Social Media Relationships

Before we start, remember that to isolate the effect of your social media program, you must begin your measurement before you launch that program. Then you'll have a benchmark to compare against later: Before Social Media vs. After Social Media. Without a Before benchmark you won't know how your social media program has changed your relationships.

Of course, you can always begin measuring after your social media program has started, and by doing so you will be able to do ongoing evaluation of your relationships. Which is a good thing. But a better thing is to isolate the effect of social media, and to do that you must compare before and after.

So use Steps 1 through 9 below to survey your audience before your program begins, and thus establish your benchmark. Then begin your social media program, and after an appropriate time, say three or six months (depending on your situation), re-survey to see what has changed.

To be sure that whatever change you find is solely attributable to the social media program, you must hold constant any other PR programs that affect your social media audience's perception of your organization. Yes, it's tricky, and it's not always an ideal world. But if you are trying to measure the effect of your new blog at the same time as the Promotions Department decides to give away A Free Cadillac With Every Purchase, then you can kiss your results goodbye.

Step 1: Define the audience for your social media program.
Social media is about conversation and engagement, so decide with whom you want to converse and engage. If you're starting an internal blog, your audience is your employees. If you're starting an IdeaStorm-type customer community, your audience is anyone likely to buy or recommend your product. If your mission is advocacy, your audience might be voters, politicians or industry influencers.

Step 2: Get a list of your audience's email addresses and/or phone numbers.
To get a representative sample you will need at least 500 names for each organization you are testing (more on that later). If you already have a list of your members, subscribers or customers, then you are ahead of the game.

If you have to purchase your list, then potential vendors vary with the type of sample you're looking for (mail, phone, web). Most lists are sorted based on demographic or title data. There are a lot more resources out there for mail addresses and those resources do not necessarily need to be survey sample companies. For email addresses, some reputable firms include Survey Sampling, e-Rewards and Zoomerang.

Step 3: Pick a survey methodology.
The Grunigs would recommend in-person surveys for the best results, but most researchers find that to be very expensive. Phone surveys are fast and provide very accurate results, but again, depending on the audience, may be cost-prohibitive. Email surveys are an increasingly accepted methodology, and for social media can be appropriate and highly reliable, since, presumably, your audience is all on email.

You may be able to piggyback on an existing survey going out to your community. If marketing, customer satisfaction, business development or anyone else in your organization is doing a survey, see if you can add a few of the Grunig/Hon statements to it.

Step 4: Select which of the Grunig/Hon statements are most appropriate to your organization.
You can probably only impose on someone for 7-10 minutes of their time, so you need to pick which statements you will include. Grunig and Hon suggest that if you want to shorten the survey, you use only the boldfaced items. Not all statements are appropriate for all organizations, so pick and chose the ones that will be most meaningful to your audience.

Step 5: Prepare your survey.
If you are using an electronic survey system like Survey Monkey, you need to create an introductory screen that explains what you are doing and how the scoring works. For instance:

In order to better understand the needs and perceptions of our marketplace, we'd like to ask you some questions. Please tell us whether you agree or disagree with the following statements as they apply to X company/organization.

Explain that 1 means "totally disagree" and 7 means "completely agree," and give them an option for "no opinion." You also need to ask them the same questions about a competing company or organization, so you have comparable data on the competition to compare to.

Step 6: Send out the survey.

Step 7. Resend the survey.
Depending on your audience it may take several tries and an incentive to get sufficient responses (I'll do just about anything for an Amazon or Starbucks card). How many is sufficient? Well, it depends on how you plan to break down your analysis, but in general plan to resurvey until you get at least a 10% return. If in doubt, talk to your local survey expert.

Step 8. Analyze and learn from the results.
Calculate a mean score for each relationship component. There are both positive and negative statements in the survey instrument, so make sure you take that into account. Compare your mean on each score to the competition. (And of course to your earlier survey results, if this survey is not the benchmark.)

Step 9. Implement your program and measure again.
If this is your pre-social media program (benchmark) survey, then implement your program now, and measure again in six months. Or, if your program has been running for a while and your analysis indicates you need to make changes, then make the changes now and let them work on your audience for six months before you measure again.

Good luck, and let me know how things go.

Should We Trust Web-Based Studies? (This research says Yes.)

This comparative study of Web-based survey samples and paper-and-pencil surveys (by Samuel D. Gosling, Simine Vazire, Sanjay Srivastava and Oliver P. John and which appeared in American Psychologist) is now four years old, but its conclusions are probably still valid: Yes, Web-based self-report and self-selected samples can be trusted for surveys. From the study's summary:

"Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N
361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and age. Moreover, Internet findings generalize across presentation formats, are not adversely affected by nonserious or repeat responders, and are consistent with findings from traditional methods."

Get the whole study here.

Ten No-Cost Ways to Measure Online Engagement

by Katie Delahaye Paine

1. Set up Google Analytics on your blog to find out how many repeat visitors you have. How many pages per visit do they check out? How many go back more than 3 times a week? How many go back and spend more than a second or two on the site?

2. Post a vizu poll on your blog and see how many people respond.

3. Go to xinure and enter the URL of your choice to find out how well it is doing in search engines, links, social bookmarks and a whole bunch of other stats.

4. With many of the leading blog providers like TypePad, check your stats to find out how many people have subscribed, and how many visits per day you're receiving.

5. What's the Conversation Index (the ratio of postings to comments)? In the blogosphere any comment is a good comment because it shows that people are engaged enough in what you are saying to take the time to respond.

6. If you have posted a video on YouTube, or a photo on Flickr, check to see how many people have rated it, and/or commented on it.

7. If you have a presence on Facebook, how many people have joined your group?

8. Ask a question on Facebook and see how many people respond.

9. If you're on Twitter, how many followers do you have? More importantly, how many responses do you get when you ask them a question?

10. Set up a Technorati, Sphere or Icerocket search to find out whether people are writing or talking about you.

The Interpretation of Dreams: Dreams as Public Relations Measurement -- I Dream of Hillary, I Dream of Barack

Many public relations professionals find that media analysis or survey research are tools sufficient to measure their programs. The more adventurous bring in external data and more esoteric measures of success like donations or number of new memberships or even lives saved. In politics, KDPaine and Partners has recently used YouTube video counts to predict primary results, and our Ms. Paine has ruminated on the possibility of using the number of political lawn signs as an election predictor. Not until now, however, has anyone used dreams to measure political progress.

In this week's New Yorker, Ben McGrath writes about Sheila Heti and her new blog "I Dream of Hillary... I Dream of Barack." This blog, a repository of reader-submitted dreams about the candidates, can be interpreted as a rough poll of interest in those candidates. As Mr. McGrath says, "...what if the recurrences of Presidential candidates in people's dreams were meaningful in the aggregate?" As of the writing of the article, "..Obama's edge in the over-all dream count... was roughly equivalent to his lead in the latest Gallup poll."

Ms. Heti's blog has been so successful she plans to open an I Dream of McCain section, so in the future we will have bipartisan dream data to go on. --Bill Paarlberg, Editor, The Measurement Standard

Why Public Relations Measurement Is Like Phrenology, or... Just because it is easy to measure, doesn't mean it's the best thing to measure.

PR measurement and marketing measurement often find themselves in a situation analogous to the 19th-century practice of phrenology. Way back then, some people used to think that the shape of your skull indicated your personality or intelligence. It was a very easy thing to measure, so they did. And they hung on to it as a serious area of study for much longer than they should have, because it was far more difficult to measure personality or intelligence by other, more effective, means.

Click-throughs aren't quite as useless as bumps on the head, but still... In the news today is a study by Starcom USA, behavioral targeting firm Tacoda, and comScore that suggests that clickthroughs might not be a very effective measure of the usefulness of online ads. See here for the news and here for Sam Harrelson's ReveNews discussion.

The obvious point here is that clickthroughs, while easy to measure, probably are not very effective at measuring what we would like to measure. The more interesting point is: What aren't we measuring what we really want to know?

I have noticed that PR and marketing metrics are often determined by what technology makes easily possible, rather than by what we would really like to know. Thus people tend to measure what technology drops in their laps (the easy things like click-throughs or AVEs) rather than something more relevant but difficult to measure (like, "Did the ad with the red type result in more sales among wealthy women?"). Then at some point they realize that their easy-but-less-relevant metric is not very effective, and then they spend a lot of time asking why the metric doesn't work.

What they should be asking themselves is: "What made us think that that metric would work in the first place?"

Instead of focussing on what technology is available (in this case click-throughs), wouldn't it be more productive to think about human thought and behavior first? (I've written about this before: "Measurement's Empty Head: Measurement ignores the most complex part of PR.") First think about what it is that might make PR work -- why it affects people's thoughts and behaviors -- and then decide what might make a good metric. First determine what we really want to measure (regardless of the technology or data available), then figure out a way to measure it. -- Bill Paarlberg, editor, The Measurement Standard

Great Things About A Public Relations Measurement Dashboard

Measurement Tools

13 Reasons Why You Need A Public Relations Measurement Dashboard
And if you have one already, here's a baker's dozen ways in which your dashboard can help you do your job better every day.

1. Your dashboard provides data for making better strategic decisions.
Whether you're being asked to "Put out a press release," or "Put together a press tour," you need to know what's worked and what hasn't worked in the past. The trend data in your dashboard will tell you not just what has worked in terms of generating exposure, but also whether the tactic resulted in more or less positive exposure, more or less message communication, and the impact the effort had on new user registrations.

2. Think of your dashboard as a continuous improvement tool at your finger tips.
A dashboard is essentially a live, searchable, continuous quality improvement tool. You can instantly determine which of your efforts resulted in better performance. By weeding out the tactics that didn't work, you are ensured of increasing effectiveness.

3. Your dashboard tells you if you are winning or losing key battles.
In any PR department, winning or losing the major positioning battles is critical. How you spend your day may depend on whether you're ahead or behind on a key initiative. Your dashboard can tell you right now how you're doing on any given battle, so you'll know where you need to place your resources. It also provides a long term trend view of whether or not you're winning or losing the battles. Furthermore, it helps you figure out why.

4. Your dashboard helps you figure out if your resources--budgets as well as time--are being spent effectively.
These days, chances are time is even scarcer than money in most PR departments, so where you spend your time becomes a critical decision. Checking your results on a regular basis--to determine if your agency is being effective and if your efforts are paying off--enables you to allocate resources more effectively.

5. Your dashboard helps you figure out which of your actions has the biggest impact on business.
By integrating media results with signups, retention, and other key data, you can make decisions based on the expected impact on the business, not just on media exposure.

6. When you're asked for goals and objectives, your dashboard helps you set realistic expectations.
Whenever you go into a meeting these days, people expect you to be able to set specific, numeric objectives. Your dashboard provides the data at your fingertips with which you can confidently set those benchmarks.

7. Your dashboard can tell you which messages are resonating, which are falling on deaf ears, and why.
Despite the best laid plans, there are times when a key message just isn't getting picked up, either because the media doesn't care or because your spokespeople aren't delivering it. Your dashboard helps you determine which messages are or aren't getting into the media. And better yet, it can help you determine the cause.

8. When a reporter calls, you can look up, instantly, how they've been covering you, what they've covered, and whether or not they "get" your key messages.
Before you answer that call from a reporter, you should know if he's been hostile or friendly, whether he gets your key messages or not, and what topics he's been covering. All that information is available at your fingertips in your dashboard.

9. When you're trying to decide on the best spokesperson, your dashboard can tell you who will be most and least effective.
To quickly determine who the best spokesperson is for any particular interview, use your dashboard to look up who is being quoted in your coverage, then note the tone (positive, negative or neutral) of the article, and the extent to which it conveys a key message.

10. Your dashboard will help you make better decisions about which publications to pitch for which story.
Suppose you have an exciting announcement, but you don't know which media to pitch first. Check your dashboard and determine which publications are most likely to cover the topic or product category. If it's not something you've ever talked about before, do a quick Google News search. Compare the top publications that come up with the data in your dashboard to see which ones are most likely to give you favorable coverage.

11. You can make better decisions about which bloggers to engage in a conversation.
Bloggers and other social media influencers are a key element of many communications programs. By checking your dashboard to see who is blogging about which issues and how favorable they are towards you, you'll have a better understanding of which bloggers you need to start a conversation with.

12. Your dashboard enables you to find influential spokespeople and analysts relevant to the issues.
The dashboard will help you not just identify who are your most influential analysts, but also who are the analysts and experts most relevant to your issues, and which ones are talking about the issues most often.

13. Your dashboard gives you ammo to push back against dumb ideas.
Every communications department is asked to take on a wide variety of projects. Some are wonderful, and some are dogs, but without data it's hard to push back against the latter. Your dashboard can save you countless wasted hours by demonstrating what hasn't worked in the past, and providing you with welcome proof that not all ideas are good ones.

Katie Delahaye Paine's Top Ten Measurement Mini-Tools

Measurement Tools

The must-have public relations research things that Katie (and probably you, too) can't live without.

Here at The Measurement Standard, we are always writing about the big measurement research tools, like media analysis, or factor analysis, or surveys. So the other day we asked Katie Delahaye Paine (CEO of KDPaine & Partners and publisher of this newsletter) to consider the many smaller yet vital components of her everyday work: What are the small-but-indispensable tools that she can't live without? Here's her list (and she'd love to hear what yours are, too):

1. A solid, well-researched list of key media
Knowing what media your target audience pays attention to is critical, particularly in today's media-overloaded world.

2. A set of reader instructions that has been thoroughly tested for validity
In order to get consistency in any research program, you need a consistent methodology. Reader instructions are the most fundamental part of getting any media analysis off to a good start. You just can't code for tonality without first properly defining "positive" and "negative."

3. Google
Whether it is verifying the title of a key spokesperson, looking up a missing circulation figure, or using Google Analytics, I couldn't do my job without it.

4. A fundamental understanding of what makes your audience or constituency buy or perform in the way you want them to
Unless you understand what makes your customers tick, your employees productive, your constituency vote, or your members give money, you can't design a relevant measurement program. You need to understand the impact that communications has on their actions and behaviors before you can figure out what to measure.

5. A benchmark
Unless you know where you've come from or how well the competition is doing, you have no idea if your results are improving or declining. So, whether it's a peer organization, the competition, or how you were doing a year ago, you need a benchmark to make measurement meaningful.

6. Trend data
I hate pie charts. Pie charts yield very little useful information unless you want to know how you are doing today at this very moment. Only with trend data can you determine whether a particular program, strategy or tactic was effective at a particular point in time.

7. A telephone and a stamp
Online surveys are great, but there are a great many instances when a phone or mail survey is the only accurate way to get the data you need.

8. Excel
I might be able to get along for awhile without all the sophisticated analysis tools that SQL and SAS and SPSS can provide, but I really couldn't live without my Excel pivot tables for analysis.

9. The verbatim comments, responses, articles, and postings.
The two most important parts of measurement are figuring out the "Why?" and the "So What?" Actually reading people's verbatim comments can tell you a great deal about why people responded the way they did. Analyzing their comments, blog postings, and the actual articles also tells you what contributes to a particular result.

10. A statistical analysis package that can do correlations
Without some form of statistical analysis, it's impossible do draw conclusions or determine if there is any connection at all between your activities.

How To Evaluate Events and Sponsorships

Sponsorship Measurement

Six basic steps, five final lessons and four additional resources.

By Katie Delahaye Paine

"Live from the KDPaine & Partners/Verizon Wireless Arena we have the KDPaine & Partners' Battle of the Sponsorship Suppliers, underwritten by Orville Redenbacher's Microwave Popcorn, the official snack food of KDPaine & Partners; by Starbucks, the official fuel of KDPaine & Partners; and with additional support from Staples, the official office supply company; FedEx, the official overnight delivery system; Diet Coke, the official diet drink; and Catalina, the official pet of KDPaine & Partners."

Perhaps not these exact words, but something like them has passed your ears at least a dozen times in the last few years. From Little League teams to the World Cup, from naming arenas to naming road races, sponsorship of events is now one of the marketing methods of choice. As communications is increasingly impersonal and electronic, the chance to actually touch, feel and experience the brand is ever more important. Increasingly, purchasing is neatly divided into two categories -- on-line and experiential. More and more decisions are based on familiarity with the brand and ease of purchase.

Now, I'm an avid Amazon fanatic, but I still go into Borders' brick and mortar store and end up buying something. Why? Because I get to touch and feel and browse and listen to products. If I need something in a hurry, I go online. If I want "retail therapy," I go to Borders. That's experiential marketing. Marketing that transcends the product and the media and allows direct interaction with the brand.

A 2006 study by Jack Morton Worldwide showed that live event marketing -- experiences where consumers interact with products, brands or "brand ambassadors" face-to-face -- are among the most effective ways to influence coveted consumer audiences. The study, an online survey of 2,574 consumers ages 13-65 in the top 25 US markets, confirmed that this increasingly important marketing medium resonates strongly across all demographic and product categories.

Which is why total spending on experiential marketing is expected to reach $15 billion by the end of 2007, an 11% increase over last year, according to the latest IEG research report. Companies are now allocating on average 17% of their budgets to sponsorships, compared to just 13% last year. However, 76% of companies spend less than 1% of their total sponsorship budgets on research, and in a shaky economy, if you don't measure it, you can't show ROI. And if you can't show ROI, chances are good that the program will be killed.

Another factor in the measurement of experiential marketing is that simply "reaching large numbers of eyeballs" is no more relevant a goal than is "increasing market share in the buggy whip market." Therefore, new alternative metrics -- like relationship measures, engagement indices and cost per touch -- are increasingly demanded.

We suggest that there are six basic steps to developing a solid measurement program for your events and sponsorships.

Step 1: Be clear about what outcomes your communications program is designed to achieve.

As with any effort, you can't start to measure success until you know what success means for you. For any given event the objectives might be:

• Percent of attendees more likely to purchase
• Percent of attendees remembering the brand
• Number of qualified sales leads generated
• Conversion rate of attendees
• Total potential sales = [number of attendees] x [conversion rate] x [average sale]
• For press events: number of key editors and analysts attending, percent of attendees writing or quoted on the issue, total exposure of key messages in resulting press.

Each objective, of course, requires a different type of measurement. Some types isolate the impact of PR from other communications efforts better than others. And the best objectives are specific and measurable.

Step 2: Determine criteria -- quality as well as quantity

Once you've agreed upon your objectives, establish the specific criteria of success that you will measure. If your objective is awareness, the criterion might be the percentage increase of unaided awareness of brand or product. If your objective is to sell product, the criterion might be the incremental sales after a particular PR or promotional program took place. Consider those numbers that best reflect the health of your business, or that best represent characteristics that most affect your business:

• Increase in awareness
• Increase in preference
• Increase in purchase intent
• Increase in customer loyalty
• Percent improvement in customer experience

To a certain extent, your choice of criteria is dependent on the type of event you are evaluating. If the customer experience you are measuring takes place at a trade show or exhibit booth, you might choose as key criteria the percentage of new visitors or the cost per minute spent with a client in the booth. If the experience takes place at an event such as a concert, you need to count how many people were exposed to your brand or the brand experience you were offering. Don't trust the promoter's numbers; do your own counting. Here's why:

I once met with a major auto manufacturer who wanted to measure the effectiveness of different sponsorships. It was spending many millions sponsoring car races, golf tournaments, antique car auctions and a variety of other events. According to the event organizers, attendance and therefore results were always better than the year before. But then I asked them what they were trying to achieve by these efforts. The first response was essentially: "We are always a major sponsor of these types of events." After about two hours of discussion, we agreed that the business objective was to drive potential customers into dealer showrooms.

We began a series of surveys at each event to determine first if the attendees remembered the sponsors, and secondly if they were more or less likely to test drive and/or buy the sponsor's car. We collected names at the event itself and called attendees two weeks after the event. Our results showed that on average 50% of all attendees were more likely to test drive a "sponsor" vehicle after attending the event.

After we had measured three different events, we were able to compare and contrast the cost effectiveness ($$ per person reached) of different events. The results enabled the sponsor to better understand the ROI from each event. By looking at the percent of people more likely to go to a dealership, they could determine the number of potential buyers. By subtracting the cost of the sponsorship from the profit and the projected number of car sales, they determined a projected ROI from the event. These results led the sponsor to dramatically alter their sponsorship strategies.

It's important to look beyond simple quantitative data, especially if your objective is exposure, to assess the quality of your communication. When companies clearly define their objectives it becomes a relatively simple matter to define a set of criteria against which to measure the relationships the company establishes with its constituencies in different events. Then it's a matter of comparing and contrasting results consistently cross a number of different events to ensure that the company has the most effective sponsorship program possible.

Step 3: Decide upon a benchmark

The key point to remember about any evaluation program is that measurement is a comparative tool. You need to compare one set of results to something else. The most meaningful comparisons are between different events, or between you and your peer sponsors at one event. If your fellow sponsors aren't direct competitors you can learn a lot by sharing information and determining what best practices really moved the needle.

4. Select a measurement instrument

Do not rely on data gathered at the event itself. It is rarely useful or actionable. Consider the following instead:

• Post-show awareness survey
  In our opinion, the most reliable way to measure relationships with your customers is to collect names at the event and interview them afterwards to see what they can remember of about the experience. Do they remember being in your booth? Did they even know you were a sponsor? Did they remember your brand, and the brand benefits or brand positioning you were trying to convey? Did they leave more likely to purchase or to recommend?
• Web analytics
  Because customer actions are so easily trackable from website activity, more and more companies are relying on web analytics to measure the impact of sponsorships. Determining not just the increase in traffic but the actual conversion rate is a relatively easily task using Google Analytics, WebTrends or Visual Sciences.
• Cost per touch
  As a way of comparing different marketing opportunities, some organizations are replacing the old "CPM" or cost per contact with Cost per Touch. In other words what does it cost to make contact (by phone, email, in person, or however you define it for your research) with one potential customer?
• Cost per minute spent with prospect
  If the purpose of the event or sponsorship is to collect sales leads you may want to calculate the value of your effort in monetary terms. If you have a product or service that has a long sales cycle and needs time to explain, this is a particularly good way to evaluate marketing efforts. Several years ago, the pharmaceutical company Glaxo figured out that it cost them $300 to get a salesperson into a doctor's office for about five minutes. That's $60 per minute spent in front of a prospect. Now suppose that by giving a speech you get 60 minutes in front of a qualified audience of 100 people. That's $0.83 per minute spent with each potential prospect. Relatively, this appears to be very efficient.
  
  Once you have expressed it in that form, you can compare value to other marketing efforts. So, for example, if a 20-second underwriting spot on NPR costs $5,000, and assuming it communicates your key message and reaches 500,000 listeners, that's $.01 per opportunity to hear a key message and $0.03 per minute spent with a prospect.
  
  You should also monitor the media coverage around the event to see if additional messages are communicated to your audience via the media surrounding the event. You need to look at the reach of the publication and also determine whether the article contained a key message. You can then take the cost of the program and divide it by the total number of opportunities to see or hear a key message to determine the cost per opportunity to see a key message. That way you can decide if the event is more or less efficient than other programs at getting your messages across and achieving your goals

A relationship study

You can also use the Grunig relationship survey instrument (downloadable for free from the IPR web site) to test the health and strength of your customers' relationships to your brand. The Grunig instrument has been thoroughly tested and shown to be an extremely effective measure of how customers perceive their relationships with an organization.

The key thing about whatever measurement tool you use is to make sure that you are tying results back to the desired business outcome. A number of years ago, Country Music Television (CMT) hired tractor trailers to set up country music events in Wal-Mart parking lots. Their first measure was to ask the truck drivers how many people showed up. Realizing that those answers didn't tell them anything about what those potential customers were actually doing when they went home, CMT decided to hire my company to do follow-up studies with the attendees. The stated purpose of the events was to convince attendees to call their local cable company and request CMT or to take some other action on behalf of CMT. CMT used a sweatshirt giveaway at the events to collect names and addresses of attendees, whom we called two weeks after the event. We agreed that the common criteria against which we should measure all events was an ad hoc "level of engagement." Specifically, we asked people to rate how much the event experience affected their relationship with CMT, and then we established the mean level of interest for each event. We further defined the success criterion as the percentage of people willing to take action on behalf of CMT. Our surveys showed that 93 percent of attendees were willing to take some action and 89% were willing to make a phone call to their local cable company. We then compared results between different cities so CMT could determine where to expand the program the following year.

Five final lessons for sponsorship measurement:

1. You become what you measure, so be very clear internally on the objectives of your sponsorship or event that you will measure.

2. Match the event, the speaker, and the metrics to the objectives.

3. Document any immediate or tangible results: media coverage, phone calls or inquiries, messages delivered to key audience, etc.

4. Share and leverage results internally.

5. Repeat your measurement consistently.

Other resources:

www.instituteforpr.org offers several white papers on measuring events, including:

• Measuring the Effectiveness of Speakers Programs
  By Marianne Eisenmann and Katie Paine
• A Guide for Measuring Event Sponsorships
  By Bruce Jeffries-Fox
• Guidelines For Measuring Relationships in Public Relations
  By Linda Childers Hon and James E. Grunig

My just-published book Measuring Public Relationships includes at least a couple chapters that bear on event measurement. You can purchase the book from the KDP&P's office at 603-319-1047 (25% off the list price of $29.95 until 1/31/08), or you can download the pre- press draft at no charge right here.

"...And What Public Relations Measurement Tool Do You Want Santa To Bring You This Christmas?"

It's a Measurement Christmas

We here at The Measurement Standard are measurement experts, after all. So, when it comes to divining secret Xmas desires, we're either quizzing the seasonal elves at the local mall, or we're asking you our faithful readers. What measurement tool gift would you like for the holidays? Here are the answers so far, (let's hear from you):

Pablo Baqués: Perhaps Santa should start at the conceptual end. We have price-to-earnings ratios for stocks, why not develop, for example, a followeD to followeRs ratio (D/R) for twitters? e.g., martinvars 5/495; barackobama 4882/4592. Who's got more leverage, Santa?

Chris Abraham: A meter stick. An interest calculator. I never get it right.

Greg Peverill-Conti: The tool I would love to have is one that could connect the dots: A post here was picked up there, commented on by him, linked to by her, etc. I think it's the organic nature of social media that makes it so hard to measure.

Linda VandeVrede: I agree with Greg's gift idea. Something that would track wet paint footprints over all the links and deliver it to me in one tidy package. If I can't get that for the holidays, then I'd like a week's vacation in wine country.

Todd Van Hoosear: To expand on Greg's answer, I'd love something like BuzzLogic that can map out your influencers, but that includes MSM as well as social media.

Paul Gillin: Something that tells me who's subscribing to RSS feeds. (Ed's note: Read our review of Paul's new book in this issue.)

Lee Odden: A dashboard to monitor online PR efforts influenced by search marketing and social media promotion.

Kevin Dugan: A universally accepted set of metrics for public relations and media relations.

Doug Haslam: A 10-foot pole to keep away people expecting definitive ROI from PR measurement.

Measure the Media that Are Most Important to Your Customers

The Paine of Measurement

 

 

Make Good Choices
No one says you have to measure every medium on the planet.

When my cousin Caleb was into his "terrible twos" and heading for trouble, his father, who stands an imposing 6 feet 3 inches tall, would look down at him and say in his deep baritone: "CALEB, MAKE GOOD CHOICES." Invariably, Caleb would stop what he was doing, sit down and think for a minute, and then modify his behavior accordingly.

The lesson seems to have taken. Caleb is now taller than his father, and a true teenage heartthrob. With very few exceptions, he makes great choices.

Oh, that PR people had such good survival skills. Sadly, they tend to keep doing the bad stuff rather than embracing the good. Four years ago, a member of the audience at a conference on new media raised his hand and said, "l don't have time to deal with all this on-line stuff, I'm too busy as it is."

I wonder how many PR people in the 30's and 40's said the same thing about radio and television. "Woe is me, one more thing I have to learn how to do." Rather than embrace the new, PR people tend to look at new media as yet another burden to be borne. And once they start measuring and tracking one media, they find it extraordinarily hard to embrace a new one. In the last month, at conference after conference, people have whined about how hard it is to deal with social media, and why they don't have the time or resources to measure it.

My answer is: Make Good Choices. No one says you have to measure every medium on the planet. Just because it's new, it doesn't mean you have to monitor it. You can and should choose to concentrate on, and measure, the media that are most important to your customers. If they're on Facebook and YouTube, you better be monitoring them. If they're sharing ideas on Twitter and photos on Flickr, it would behoove you to sign up and listen in. On the other hand, if they're still reading hard copies of trade magazines, then that may be all you need to monitor.

Better still, measure the impact on the customer directly. As Jim Macnamara writes, the average person sees, consumes and creates dozens of different media every day. If you have an unlimited budget, you measure all of those different media. But it's so much simpler to just measure the behavior of the consumer instead. Ask them how they perceive their relationship with your organization. Follow their behavior on your website. Watch and see how engaged they are with your blog. That's all the measure you need.

As long as PR people obsess about tracking column inches and "hits," of course they are going to be overwhelmed with the possibilities and the burden of measuring them all. But if they focus on the outcome – the actual customer behavior – chances are they'll survive and thrive in this new hyper-media world.

A Real-time Dashboard Public Relations Measurement People Can Only Dream About, So Far: The World Clock

If you haven't seen the World Clock, it's one of the most amazing examples of a real-time dashboard.

A Public Relations Measurement Index Number Based on Ideal Coverage

The Paine of Measurement

Optimum Content Score: An Index Number to Love?
Here's a quick and easy way to design your customized article content score.

For years I've ranted about why a single Index number or a standard PR metric won't work (see "The Problem of PR Indexes: Magic Number or Big Headache?"). My argument has been that each PR program is different, with different goals, different audiences, and different measures of success. So how can you possibly compare them all with one number?

Embrace the Differences

However, I am beginning to believe that there might be a solution to this problem, and that is to embrace the differences--to build the differences into the measure. Here at KDPaine & Partners we now offer our clients a measure that is tailored to match their individual program goals: the Optimum Content Score (OCS). It gauges the success of articles, based on what the client considers to be the ideal article. With the OCS, we distill a potentially large number of variables into one number, based on the relative concept of “ideal" coverage. That way, we can tell you whether your coverage was more ideal than last year (comparing to benchmark), or less ideal than you wanted it to be (comparing to objectives).

What Is the Ideal Article?

For any particular client, we define an ideal article based on their coverage goals at the present time. So, for instance, an ideal article for Client X might be one that:

• contains several key messages,
• leaves a reader more likely to purchase,
• mentions the brand in the headline, and
• appears in the Wall Street Journal.

If an article includes all these attributes, we give it the maximum score, ten out of ten. To determine the score for any given article, points are deducted from the maximum based on how far an article is from ideal, taking points off, for instance, for negative positioning and minor mentions.

The beauty of this system is that the company that is looking to keep out of the headlines can gauge its effectiveness as well as a company that is trying to get into the headlines. For instance, in the middle of a crisis, ideal press might be a minor mention that contains a key message and is balanced.

I realize we're still measuring outputs here (and our long term goal is to measure outcomes), but this sort of customized-to-goals score is a big improvement over imposing one score across a wide variety of organizations and industries that may not have a lot in common. See Ed Moed's "Measuring Up" blog post on the subject here.

OCS allows you to look at those things that are important to your own business, insuring that you are measuring the most relevant elements of what you do. For instance, Sabrina Steele at Raytheon has used it to make better decisions on supporting trade and air shows, using spokespeople, and comparing her effectiveness with her peers.

Another advantage of OCS is that it can be compared to other marketing data to determine what does, in fact, drive outcomes like Web traffic, product preference and consideration. Factor analysis using OCS can tell you exactly which program elements and media efforts are having the most impact on whatever are your ultimate marketing measures of success (typically sales, sales leads or market share).

Design Your Own Article Content Score

You can easily get started with your own version of an optimum content score. Below is a sample list of questions--and you may wish to remove some or add your own--that you can use to determine the characteristics of the ideal article for your own program. Then rate your articles on how closely they come to your ideal.

Wishing you large measures of success,

Sample questions to determine ideal article content:

1. What is the goal of the program? (pick only one):

a. Increase exposure

b. Keep bad news to a minimum

c. Disseminate key messages

d. Increase preference

e. Generate awareness for spokespeople

2. How important is brand or sub-brand visibility?

a. Extremely important

b. Somewhat important

c. Not important at all

3. How important is it that the story be exclusively about your organization or brand?

a. Extremely important

b. Somewhat important

c. Not important at all

4. How visible do you want your spokespeople to be?

a. Highly visible

b. Somewhat visible

c. Doesn't matter

d. Invisible

5. How important is competitive positioning?

a. Extremely important

b. Somewhat important

c. Not important at all

6. How important is the tone of the coverage?

a. Extremely important

b. Somewhat important

c. Not important at all

 

 

How To Measure Public Relations Success for Small Organizations

Measurement Tactics

How To Measure Success by the Gram Rather Than by the Ton
Five simple measurement steps for smaller organizations.

by Katie Delahaye Paine

Here at KDPaine & Partners, we've recently seen more and more non-profits seeking ways to measure their marketing success. Organizations as diverse as the The American Society of Plastic Surgeons and the ASPCA are now using KDPaine & Partner's DIY Dashboard as a cost-effective way to track their PR performance. But even these organizations are large and well funded compared to many of the startups and small businesses that we talk to on a regular basis.

Think differently
For these smaller organizations, it's not a matter of counting or analyzing thousands of media clips--getting 10 media clips would be a huge victory. So how do you measure success by the gram rather than the ton? It's actually a lot simpler than you think. You just need to think differently.

In any small organization, public relations and marketing are far more closely tied to business outcomes than they are in large organizations. Can GM measure how many more cars it sells as a result of its CEO's blog? No. But can the owner of a gardening shop track how much more business comes in as a result of starting a blog? Absolutely. The key is in knowing where to look, and knowing what motivates your customers.

Here are five simple steps for smaller organizations to get started in measurement:

1. Understand what motivates your customers.
If you don't know, ask them. Ask them for their email address in exchange for a discount coupon or free gift. Better yet, send them a coupon every month. Email them a survey, and, for people who don't use email, do a paper one at your checkout as well. What you want to know is:

• Why did they walk in the door?
• How did they hear about you: Web site? Blog? Newspaper article? E-mail? Regular mail? Friend?
• What motivated them to stop?
• Did they buy and if not why not?

When you get the responses back, separate the answers into two categories: buyers and just shoppers (non-buyers). Then look at what got them into the store and why they did or didn't buy. You can set up a salesforce.com account for as little as $10 a month to start tracking your customers and prospects. Pay particularly close attention to the repeat visitors. They are the most profitable, so you want to know what got them in the door in the first place, and what keeps them coming back.

2. Make it someone's responsibility to count and measure.
If you're a one-person shop this gets difficult, I know. But if even if there are only two of you, make sure someone owns the job. Without regular tracking of outputs, outtakes and outcomes, measurement isn't possible

3. Know your Web analytics.
With 95% of American's now using the Web and the vast majority of them making buying decisions based on what they find on the Web, this should be where all measurement begins. Whether you use Google Analytics or WebTrends or ClickTracks or some combination of the three, you need to know where people are going on your site, where they're coming from, how long they are spending and what they're doing while they're there.

4. Don't just ask questions, collect data.
Don't just ask someone where they're from or how they found you--write it down! Ideally, you should have a standard form on hand to collect the data. Better still an automated, computerized way to track it on a dashboard. But failing that, at least write it down. Collecting data is the key to knowing what works and what doesn't work. Sure you get busy and forget, but once you realize how valuable the data is, you'll never go back to chit-chat again. As the Victorian genius Francis Galton said, "Whenever you can, count."

5. Report back, even if it's just to yourself.
The big difference between most large organizations and smaller ones is the presence of a board of directors or other governing body that requires regular reporting of data and numbers on your success. Even if you don't have a board, it is critical to the success of any organization to know what is working and not working. Especially if you are a smaller business, because, if you don't have a lot of marketing dollars to waste, you better know just how well each one is working for you. So make sure you spend 5-10% of your time and/or budget collecting and analyzing the data so you can tell whether the other 90-95% of your time and/or budget is working properly.