The Measurement Standard: Blog Edition

How to Develop KPIs and Metrics: An Olympic Case Study

Measuring Events and Sponsorships

Olympic hopeful Caleb Paine rounds a buoy

A young sailor maps his course for the gold.

Editor's Note: This article is about how to work from your goals to develop KPIs and metrics. 

• To learn about how to actually do the measurement, see "KDPaine's Trade Show & Event Measurement Checklist: Your framework for measuring trade shows, events and sponsorships.")
• To learn more about how to measure the ROI of a conference, see "How to Measure the ROI of a Conference," where you can download a sample spreadsheet that Katie Paine uses to measure conferences she speaks at.

by Katie Delahaye Paine

When I heard my cousin Caleb Paine talk about his goal to become an Olympic sailor, I immediately started thinking about metrics that could help him get there. For any public relations measurement program, setting goals and KPIs is a vital beginning step. (In fact, KDPaine & Partners' new product, the KBI Development System, helps companies do just that, among other things.) So I became intrigued about how I could apply some lessons from measurement to Caleb's quest for gold.

I first met Caleb when he was still in diapers. His father Doug, my first cousin, took me along as crew in a sailboat race off San Diego. Caleb spent most of the race wandering cheerfully around the boat, using every piece of hardware as his own private jungle gym. We were sailing very well and approaching the finish line when it became abundantly clear that Caleb’s diaper needed changing. Doug didn't hesitate or waver from his course—he took the tiller in his knees, changed his son’s diaper, and won the race.

No wonder the kid became a sailor!

At the age of seven, when he was already a hot shot Sabot racer, Caleb learned that sailing was an Olympic sport. That’s when his personal goal was established: He would win Olympic Gold in sailing.

Now, 11 years later, he’s a member of the Junior Olympic US Sailing Alphagraphics team, training for London in 2012. I was honored when he asked me to help him with his marketing. That’s when it occurred to me that I could use his campaign as a perfect example of how to measure events and sponsorships.

Defining Strategies, KPIs, and Metrics

Let's leave Caleb to his sailing for a moment and review the public relations measurement basics involved with KPIs and metrics. Your overall goal is pursued by means of one or more strategies. We define each strategy in terms of KPIs (Key Performance Indicators), which indicate progress on the strategy. The metrics are the specific measurements -- the numbers -- that we use to assess the KPIs, and thus assess progress on the strategy. Each metric is provided by a measurement tool.

Consider an example: If, for instance, you were measuring progress on your goal to increase a company's reputation as a thought leader in green technology, then: 

• Your strategy might be to have your company's thought leaders get more exposure and better positioning as a green tech expert relative to the competition;
• Your KPIs would probably include Share of Positioning as Thought Leader, or Share of Company Thought Leaders Quoted vs. the Competition;
• Specific metrics for these KPIs would be the actual share expressed as a percentage; and
• The measurement tool you would use would be qualitative content analysis of articles or tweets or blog posts or other social media. 

Sail On, Sailor

Now let's return to Caleb's Olympic quest, and apply the above framework to analyze his campaign like we would any program. The goal is Olympic gold. I happen to know that he intends to pursue his goal by means of three strategies:

• Strategy #1: Ensure that his body is ready. Caleb sails a Finn, the most demanding class of boat, so extensive physical conditioning is an absolute necessity.
• Strategy #2: Ensure that his sailing skills are ready. He’ll be competing against other sailors with far more years on a Finn, so he needs experience.
• Strategy #3: Ensure that he has the resources. Boat, sails, travel costs – to make it happen costs money.

KPIs for the first two strategies are relatively easy and straight forward.

KPIs for Strategy #1: He clearly needs to achieve certain benchmarks in strength and endurance to know how strong he is getting and by what date. So typical KPIs might be:

• Run xx miles in xx minutes
• Bike xx miles in xx minutes
• Bench press xx pounds (or whatever exercises are appropriate)

In the marketing world these might translate into activity metrics. For instance:

• Produce xx of blog posts, or
• Win xx of speaking engagements at conferences for thought leaders

KPIs for Strategy Part 2: The second set of KPIs are provided by the races in which he competes and the structure of US Sailing. Caleb needs to finish in the top ten or top twenty in every race and show consistent progress.

In the marketing world, these could be considered basic marketing performance metrics, such as:

• Increase web traffic by xx%, or
• Increase number of qualified trade show leads by xx%

KPIs for Strategy Part 3: The KPIs for the third strategy should also sound pretty familiar: Demonstrate marketing clout to attract sponsors. This could be measured by:

• Number of Facebook friends and “likes.” Here's his group. (I know, I know, I rail against using this kind of metric all the time, but in this case it’s a valid metric of Caleb’s value as an endorser of a brand)
• Number of followers on Twitter – another metric that shows sponsors that he has value as a spokesperson (@cpainesailing)
• Number of views, ratings and comments on a YouTube channel
• % increase in visits to his website
• % increase in requests for information
• % increase registrations and subscriptions to his website and blog
• Share of desirable mentions vs. other top Olympic-bound sailors
• Share of desirable discussion vs. other top Olympic-bound sailors on blogs, Facebook, Twitter and other key online sailing news sites
• Achieving a certain number of personal appearances at targeted yacht clubs as measured by the total number of prospects encountered each month
• % increase in online donations month to month
• % increase in number of donors/sponsors month to month

All these metrics can easily be tracked in a simple Excel spread sheet. If you’re a major corporation or non-profit, you’ll probably need a more sophisticated dashboard which will enable you to set dates, alarms, and benchmarks more easily.

But the point is that none of these metrics is particularly difficult to track. You just need agreement on what they are before you begin.

Echo Research: World Cup Sponsors Miss Goal

Surprising new survey research from Echo reveals that over half of English adults cannot correctly identify sponsors of the recent World Cup. Coca-Cola and Adidas actually had recognition drop over the course of the event. In addition, results from this pre-post survey comparison show that the Cup reinforced the British public's skepticism about sponsors’ motives.

Read more at "World Cup Fiasco - Public Unable to Recall Sponsors." 

Was this survey technique, in which respondents had to select World Cup sponsors from a list of well-known brands, the best way to measure the effectiveness of sponsorships?  -- Bill Paarlberg, Editor

How to Evaluate Events and Sponsorships: An Update

Measuring Events and Sponsorships

Practical guides and online resources

One of the most popular articles on The Measurement Standard Blog Edition continues to be "How to Evaluate Events and Sponsorships: Six basic steps, five final lessons and four additional resources." So we thought we'd play to this obvious demand by gathering together a few articles and references on measuring events and sponsorships.

That blog post, by the way, is still state-of-the-art, and, of all the resources listed below, is the only one that provides very practical instructions and explains the use of Grunig and Hon's relationship measurement theory. (It is also the basis of the chapter on event measurement in Katie Delahaye Paine's book Measuring Public Relationships.)

Here are more resources on measuring events and sponsorships:

Articles previously published in The Measurement Standard:

• (mentioned above) How to Evaluate Events and Sponsorships: Six basic steps, five final lessons and four additional resources
• Two Keys to Better Sponsorship Measurement: Be clear on your objectives, and measure outputs, outtakes and outcomes by Jim Macnamara
• Sponsorship Evaluation Services, including the Six Basic Rules of Sponsorship Measurement, and reviews of three sponsorship measuring companies (from July, 2002)
• Roger Roesser's article, How to Make the Most of Your Sponsorships: Pick the right opportunity, leverage your exposure, and aim for 8 to 1 ROI (from January, 2003)

Articles from the IPR website:

• A Guide to Measuring Event Sponsorships by Bruce Jeffries-Fox
• Measuring the Effectiveness of Speakers Programs by Marianne Eisenmann and Katie Paine

A comprehensive and practical program for small non-profits, now somewhat dated, but with a solid approach and logic: Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources

Some odds and ends found on the Web, mostly to do with sports sponsorships:

• Here is an excellent discussion about measuring NASCAR sponsorships on the Marketing Research Roundtable.
• SponsorMap is a company that measures sponsorships. Their site includes some interesting papers and case studies, like Calculating Sponsorship's Financial ROI.
• At MediaPost Blogs, Larry Albus has some savvy things to say about sports marketing and sponsorship measurement.
• Here's a brief article on old-school approaches to measuring sports sponsorships: Sports News You Can Use: Issue 13--Measuring Sponsorship

How To Evaluate Events and Sponsorships

Measuring Events and Sponsorships

Practical guides and online resources

A year and a half after it was posted, one of the most popular articles on The Measurement Standard Blog Edition continues to be "How to Evaluate Events and Sponsorships: Six basic steps, five final lessons and four additional resources." So we thought we'd play to this obvious demand by gathering together a few articles and references on measuring ROI of events and sponsorships.

That blog post, by the way, is still state-of-the-art, and, of all the resources listed below, is the only one that provides very practical instructions and explains the use of Grunig and Hon's relationship measurement theory. (It is also the basis of the chapter on event measurement in Katie Delahaye Paine's book Measuring Public Relationships.)

Here are more resources on measuring events and sponsorships:

Articles previously published in The Measurement Standard:

• (mentioned above) How to Evaluate Events and Sponsorships: Six basic steps, five final lessons and four additional resources
• Two Keys to Better Sponsorship Measurement: Be clear on your objectives, and measure outputs, outtakes and outcomes by Jim Macnamara
• Sponsorship Evaluation Services, including the Six Basic Rules of Sponsorship Measurement, and reviews of three sponsorship measuring companies. From back in 2002, but still full of good stuff.
• Roger Roesser's article, How to Make the Most of Your Sponsorships: Pick the right opportunity, leverage your exposure, and aim for 8 to 1 ROI (from January, 2003)

Articles from the IPR website:

• A Guide to Measuring Event Sponsorships by Bruce Jeffries-Fox
• Measuring the Effectiveness of Speakers Programs by Marianne Eisenmann and Katie Paine

A comprehensive and practical program for small non-profits, now somewhat dated, but with a solid approach and logic: Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources

Some odds and ends found on the Web, mostly to do with sports sponsorships:

• Here is an excellent discussion about measuring ROI of NASCAR sponsorships on the Marketing Research Roundtable.
• SponsorMap is a company that measures sponsorships. Their site includes some interesting papers and case studies, like Calculating Sponsorship's Financial ROI.
• At MediaPost Blogs, Larry Albus has some savvy things to say about sports marketing and sponsorship measurement.
• Here's a brief article on old-school approaches to measuring sports sponsorships: Sports News You Can Use: Issue 13--Measuring Sponsorship 

How To Evaluate Events and Sponsorships

Sponsorship Measurement

Six basic steps, five final lessons and four additional resources.

By Katie Delahaye Paine

"Live from the KDPaine & Partners/Verizon Wireless Arena we have the KDPaine & Partners' Battle of the Sponsorship Suppliers, underwritten by Orville Redenbacher's Microwave Popcorn, the official snack food of KDPaine & Partners; by Starbucks, the official fuel of KDPaine & Partners; and with additional support from Staples, the official office supply company; FedEx, the official overnight delivery system; Diet Coke, the official diet drink; and Catalina, the official pet of KDPaine & Partners."

Perhaps not these exact words, but something like them has passed your ears at least a dozen times in the last few years. From Little League teams to the World Cup, from naming arenas to naming road races, sponsorship of events is now one of the marketing methods of choice. As communications is increasingly impersonal and electronic, the chance to actually touch, feel and experience the brand is ever more important. Increasingly, purchasing is neatly divided into two categories -- on-line and experiential. More and more decisions are based on familiarity with the brand and ease of purchase.

Now, I'm an avid Amazon fanatic, but I still go into Borders' brick and mortar store and end up buying something. Why? Because I get to touch and feel and browse and listen to products. If I need something in a hurry, I go online. If I want "retail therapy," I go to Borders. That's experiential marketing. Marketing that transcends the product and the media and allows direct interaction with the brand.

A 2006 study by Jack Morton Worldwide showed that live event marketing -- experiences where consumers interact with products, brands or "brand ambassadors" face-to-face -- are among the most effective ways to influence coveted consumer audiences. The study, an online survey of 2,574 consumers ages 13-65 in the top 25 US markets, confirmed that this increasingly important marketing medium resonates strongly across all demographic and product categories.

Which is why total spending on experiential marketing is expected to reach $15 billion by the end of 2007, an 11% increase over last year, according to the latest IEG research report. Companies are now allocating on average 17% of their budgets to sponsorships, compared to just 13% last year. However, 76% of companies spend less than 1% of their total sponsorship budgets on research, and in a shaky economy, if you don't measure it, you can't show ROI. And if you can't show ROI, chances are good that the program will be killed.

Another factor in the measurement of experiential marketing is that simply "reaching large numbers of eyeballs" is no more relevant a goal than is "increasing market share in the buggy whip market." Therefore, new alternative metrics -- like relationship measures, engagement indices and cost per touch -- are increasingly demanded.

We suggest that there are six basic steps to developing a solid measurement program for your events and sponsorships.

Step 1: Be clear about what outcomes your communications program is designed to achieve.

As with any effort, you can't start to measure success until you know what success means for you. For any given event the objectives might be:

• Percent of attendees more likely to purchase
• Percent of attendees remembering the brand
• Number of qualified sales leads generated
• Conversion rate of attendees
• Total potential sales = [number of attendees] x [conversion rate] x [average sale]
• For press events: number of key editors and analysts attending, percent of attendees writing or quoted on the issue, total exposure of key messages in resulting press.

Each objective, of course, requires a different type of measurement. Some types isolate the impact of PR from other communications efforts better than others. And the best objectives are specific and measurable.

Step 2: Determine criteria -- quality as well as quantity

Once you've agreed upon your objectives, establish the specific criteria of success that you will measure. If your objective is awareness, the criterion might be the percentage increase of unaided awareness of brand or product. If your objective is to sell product, the criterion might be the incremental sales after a particular PR or promotional program took place. Consider those numbers that best reflect the health of your business, or that best represent characteristics that most affect your business:

• Increase in awareness
• Increase in preference
• Increase in purchase intent
• Increase in customer loyalty
• Percent improvement in customer experience

To a certain extent, your choice of criteria is dependent on the type of event you are evaluating. If the customer experience you are measuring takes place at a trade show or exhibit booth, you might choose as key criteria the percentage of new visitors or the cost per minute spent with a client in the booth. If the experience takes place at an event such as a concert, you need to count how many people were exposed to your brand or the brand experience you were offering. Don't trust the promoter's numbers; do your own counting. Here's why:

I once met with a major auto manufacturer who wanted to measure the effectiveness of different sponsorships. It was spending many millions sponsoring car races, golf tournaments, antique car auctions and a variety of other events. According to the event organizers, attendance and therefore results were always better than the year before. But then I asked them what they were trying to achieve by these efforts. The first response was essentially: "We are always a major sponsor of these types of events." After about two hours of discussion, we agreed that the business objective was to drive potential customers into dealer showrooms.

We began a series of surveys at each event to determine first if the attendees remembered the sponsors, and secondly if they were more or less likely to test drive and/or buy the sponsor's car. We collected names at the event itself and called attendees two weeks after the event. Our results showed that on average 50% of all attendees were more likely to test drive a "sponsor" vehicle after attending the event.

After we had measured three different events, we were able to compare and contrast the cost effectiveness ($$ per person reached) of different events. The results enabled the sponsor to better understand the ROI from each event. By looking at the percent of people more likely to go to a dealership, they could determine the number of potential buyers. By subtracting the cost of the sponsorship from the profit and the projected number of car sales, they determined a projected ROI from the event. These results led the sponsor to dramatically alter their sponsorship strategies.

It's important to look beyond simple quantitative data, especially if your objective is exposure, to assess the quality of your communication. When companies clearly define their objectives it becomes a relatively simple matter to define a set of criteria against which to measure the relationships the company establishes with its constituencies in different events. Then it's a matter of comparing and contrasting results consistently cross a number of different events to ensure that the company has the most effective sponsorship program possible.

Step 3: Decide upon a benchmark

The key point to remember about any evaluation program is that measurement is a comparative tool. You need to compare one set of results to something else. The most meaningful comparisons are between different events, or between you and your peer sponsors at one event. If your fellow sponsors aren't direct competitors you can learn a lot by sharing information and determining what best practices really moved the needle.

4. Select a measurement instrument

Do not rely on data gathered at the event itself. It is rarely useful or actionable. Consider the following instead:

• Post-show awareness survey
  In our opinion, the most reliable way to measure relationships with your customers is to collect names at the event and interview them afterwards to see what they can remember of about the experience. Do they remember being in your booth? Did they even know you were a sponsor? Did they remember your brand, and the brand benefits or brand positioning you were trying to convey? Did they leave more likely to purchase or to recommend?
• Web analytics
  Because customer actions are so easily trackable from website activity, more and more companies are relying on web analytics to measure the impact of sponsorships. Determining not just the increase in traffic but the actual conversion rate is a relatively easily task using Google Analytics, WebTrends or Visual Sciences.
• Cost per touch
  As a way of comparing different marketing opportunities, some organizations are replacing the old "CPM" or cost per contact with Cost per Touch. In other words what does it cost to make contact (by phone, email, in person, or however you define it for your research) with one potential customer?
• Cost per minute spent with prospect
  If the purpose of the event or sponsorship is to collect sales leads you may want to calculate the value of your effort in monetary terms. If you have a product or service that has a long sales cycle and needs time to explain, this is a particularly good way to evaluate marketing efforts. Several years ago, the pharmaceutical company Glaxo figured out that it cost them $300 to get a salesperson into a doctor's office for about five minutes. That's $60 per minute spent in front of a prospect. Now suppose that by giving a speech you get 60 minutes in front of a qualified audience of 100 people. That's $0.83 per minute spent with each potential prospect. Relatively, this appears to be very efficient.
  
  Once you have expressed it in that form, you can compare value to other marketing efforts. So, for example, if a 20-second underwriting spot on NPR costs $5,000, and assuming it communicates your key message and reaches 500,000 listeners, that's $.01 per opportunity to hear a key message and $0.03 per minute spent with a prospect.
  
  You should also monitor the media coverage around the event to see if additional messages are communicated to your audience via the media surrounding the event. You need to look at the reach of the publication and also determine whether the article contained a key message. You can then take the cost of the program and divide it by the total number of opportunities to see or hear a key message to determine the cost per opportunity to see a key message. That way you can decide if the event is more or less efficient than other programs at getting your messages across and achieving your goals

A relationship study

You can also use the Grunig relationship survey instrument (downloadable for free from the IPR web site) to test the health and strength of your customers' relationships to your brand. The Grunig instrument has been thoroughly tested and shown to be an extremely effective measure of how customers perceive their relationships with an organization.

The key thing about whatever measurement tool you use is to make sure that you are tying results back to the desired business outcome. A number of years ago, Country Music Television (CMT) hired tractor trailers to set up country music events in Wal-Mart parking lots. Their first measure was to ask the truck drivers how many people showed up. Realizing that those answers didn't tell them anything about what those potential customers were actually doing when they went home, CMT decided to hire my company to do follow-up studies with the attendees. The stated purpose of the events was to convince attendees to call their local cable company and request CMT or to take some other action on behalf of CMT. CMT used a sweatshirt giveaway at the events to collect names and addresses of attendees, whom we called two weeks after the event. We agreed that the common criteria against which we should measure all events was an ad hoc "level of engagement." Specifically, we asked people to rate how much the event experience affected their relationship with CMT, and then we established the mean level of interest for each event. We further defined the success criterion as the percentage of people willing to take action on behalf of CMT. Our surveys showed that 93 percent of attendees were willing to take some action and 89% were willing to make a phone call to their local cable company. We then compared results between different cities so CMT could determine where to expand the program the following year.

Five final lessons for sponsorship measurement:

1. You become what you measure, so be very clear internally on the objectives of your sponsorship or event that you will measure.

2. Match the event, the speaker, and the metrics to the objectives.

3. Document any immediate or tangible results: media coverage, phone calls or inquiries, messages delivered to key audience, etc.

4. Share and leverage results internally.

5. Repeat your measurement consistently.

Other resources:

www.instituteforpr.org offers several white papers on measuring events, including:

• Measuring the Effectiveness of Speakers Programs
  By Marianne Eisenmann and Katie Paine
• A Guide for Measuring Event Sponsorships
  By Bruce Jeffries-Fox
• Guidelines For Measuring Relationships in Public Relations
  By Linda Childers Hon and James E. Grunig

My just-published book Measuring Public Relationships includes at least a couple chapters that bear on event measurement. You can purchase the book from the KDP&P's office at 603-319-1047 (25% off the list price of $29.95 until 1/31/08), or you can download the pre- press draft at no charge right here.

Two Keys To Better Sponsorship Measurement

Jim Macnamara's 'Measuring Up'

 

 

Two Keys to Better
Sponsorship Measurement
Be clear on your objectives, and measure outputs, outtakes and outcomes.

While much of the urging for better measurement focuses on public relations, another area of below-the-line (i.e., non-advertising) corporate and marketing communications that is often poorly measured is sponsorship. In some cases, sponsorships are managed as part of PR and corporate communication, while in other situations they are the responsibility of advertising or marketing departments. Irrespective, in most cases sponsorships are inadequately measured – and sometimes not at all.

Sponsorships are deserving of particular attention in the measurement debate as many involve large budgets, often in the millions of dollars. Added to cash contributions is expenditure on support materials and often large amounts of staff time. Some sponsorship experts recommend multiplying the cash outlay for sponsorships by two to calculate the total cost to a sponsor.

So how should we measure sponsorships? Like other corporate and marketing communications, there are a number of approaches. But the same number one golden rule applies: Set clear objectives and measure against them.