The Measurement Standard: Blog Edition

5 Non-Financial Ways to Measure the Success of Your Non-Profit Event – and How to Maximize that Success

by Bill Paarlberg, Editor, The Measurement Standard

Congratulations! Your non-profit organization has just pulled off a well-attended fundraising event to kick off a major campaign. It might have been a cocktail party, or an auction, or a walk-a-thon. Maybe a local celebrity spoke to raise enthusiasm and encourage pledges. You took in enough in donations to be pleased with yourself. But that's not the only way to tally up your results.

DoSomething.org Gets 1.5 Million Views, But Strikes Out On Donations

Here's a great post from Jeff Bladt and Bob Filbin at DoSomething.org on the difference between data and metrics. Yep, they got 1.5 million YouTube views for their sports equipment donation video, but totally struck out when it came to donations. The moral of their story is: "...analyzing the data is the easy part. The hard part is deciding what data matters."

Bob Filbin's insights on the use of data by nonprofits are featured in Measuring the Networked Nonprofit, the book published last fall by Beth Kanter and Katie Paine, which devotes Chapter 3 to DoSomething.org. (Thanks for the image to The Musings of an Urban Christian) --WTP

DoSomething.org Are the Measurement Geeks of the Week

by Katie Delahaye Paine

For a measurement geek like me, hanging out with other measurement geeks is heaven. A couple weeks ago Beth Kanter and I were in the offices of www.dosomething.org, a data-informed organization if there ever was one. They were kind enough to host a pizza party for our presentation about the book we co-authored, Measuring the Networked NonProfit.

A Look at the Data Behind DoSomething.org's "Pregnancy Text" Success

by Beth Kanter (This article is repurposed from Beth's Blog.)

With the release of “Measuring the Networked Nonprofit: Using Data to Change the World" (the new book I wrote with Katie Paine), I’m looking forward to participating in the conversation about how nonprofits can use measurement and data for social change. Here's an example from DoSomething.org...

Joseph Kony, "Measuring the Networked Nonprofit," and the Effectiveness of Social Media

Measuring the effectiveness of social media is not always as easy as counting video views, or even donations. If an organization misrepresents its mission, how can/should measurement take that into account?

by Bill Paarlberg, Editor, The Measurement Standard

I have just finished editing the manuscript for Beth Kanter and Katie Paine’s upcoming book, “Measuring the Networked Nonprofit” due out this fall from Wiley publishers. It is a well-written, informative, and practical guide to social media measurement for nonprofits. It will no doubt be a popular and successful book, as Beth and Katie are leading experts in the fields of nonprofit practice and social media measurement, and the book is filled with inspiring case studies from successful nonprofits and plenty of how-to advice for using measurement.

So, I was excited to read "Online, a Distant Topic Soars to #1" in today's NYTimes. It is about the phenomenal recent success of "Kony 2012," a video by Invisible Children, a nonprofit that is working to end Joseph Kony's long reign of terror in Central Africa. The video has achieved amazing success in terms of views and resulting donations, but the whole story is a little more complicated—and more difficult to measure.

Turns out, the video vastly oversimplifies the situation, to the point of making "virtually no effort to inform." Despite what the video implies, Joseph Kony and his army of child soldiers has not, in fact, been in Uganda in six years, and his following is now reduced to a few hundred. Read Michael Wilkerson's article "Joseph Kony is Not in Uganda, and Other Complicated Things" in Foreign Policy for more about the video's (and Invisible Children's) shortcomings. (And for a defense of his actions, see this video for a Joseph Kony interview.)

The complex interface between the first and third worlds has always been fraught with misunderstanding. (I am well acquainted with this, as I lived in Uganda for a couple years while in high school, and nowadays take my family to live in an almost-as-distant culture each year for a month.) I do hope that Invisible Children does good for Uganda, but it is tempting to see them as just another in a long, long line of awkward efforts of Us to help Them. As Mr. Wilkerson says:

"...what are the consequences of unleashing so many exuberant activists armed with so few facts? Defining Uganda in the international conversation by issues that are either geographical misfires (Save northern Uganda!) or an intentional attempt to distract the international community (Death to the gays!), do a disservice to the many critical problems Uganda has."

But the point here is that measuring the effectiveness of social media is not always as easy as counting video views, or even donations. Thanks to the power of social media (tweets from Rihanna and Ryan Seacrest!) Invisible Children now has a pile of money, yet it appears they have misrepresented their mission. How can or should social media measurement for nonprofits take that into account?

3/12/12 Update: See this post from Ragan's PR Daily for how Invisible Children is handling criticism: "5 crisis PR lessons from ‘Kony 2012’ "

3/15/12 Update:

• This post makes the point that "Kony 2012" is using new media to achieve old media ends: "Kony video inspires but misses larger point."
• Ugandans are furious at how the video portrays and uses them: "Ugandans react with anger to Kony video"
• 3/15/12 Update: Conspiracy theory, or actual conspiracy? "The Kony 2012 Scam: Funded by Chase Bank; Big Oil"

 

###

 --Bill Paarlberg is editor of  The Measurement Standard blog and newsletter, and of Katie Paine's book “Measure What Matters.” He is also editor of the book “Measuring the Networked Nonprofit,” by Beth Kanter and Katie Paine, which will be published this year by Wiley. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.
  Follow Bill Paarlberg on Twitter.

 KDPaine & Partners' Facebook Page.

 

5 Steps to Using Measurement to Increase Your Nonprofit’s Success

by Katie Delahaye Paine

So, you do communications for a nonprofit organization, and you want to know which of your efforts are working. Have your events and promotions achieved the results you hoped for? Is social media worth the effort? Just what is the best way to express your results? This article will show you how to use the process of measurement to express, understand, and improve your results.

Actually this article will show you how to start thinking about using measurement. Odds are your particular situation will be quite different from the examples below. You are going to have to adapt these techniques to your own situation. For more help, read the other articles in this blog and refer to its list of references. (Also, it wouldn’t hurt to pick up a copy of Measure What Matters, my new book on public relations and social media measurement. And of course get familiar with Beth Kanter’s Beth’s Blog.)

Remember that measurement is a process that uses research and data to help make improvements, and is very different than just looking at your results. Counting just adds things up and gets a total. Measurement takes those totals, analyzes what they mean, and uses that meaning to improve your practices. Measurement is an iterative process: You make changes based on your research, then gather new data, then make more changes, then gather more data, and on and on. Measurement of your processes and results—where you spend your time and money and what you get out of it—provides the data necessary to make sound decisions. It helps you set priorities, allocate resources, and make choices.

Nonprofit measurement involves five basic steps:

Step #1. Understand the mission. 

Look in the mirror first: What is it you were hired to do? What problems were you hired to solve? If you or your department went away, would it have an impact on the mission? If so, how? Make sure that, whatever measurement you undertake, you are doing it to accomplish your job.

Now, understand which stakeholders make your organization successful. Yes, I know, there are many of them, from volunteers to donors to sponsors to your fellow employees. If you are going to try to compare your results with different stakeholders, then realistically, you’re probably only going to have the resources to look at three of those groups. To decide which are most important, play the following game: Get together everyone who will be involved in using, reading, presenting, or basing decisions on the data that you plan to gather. Have them list all their stakeholder groups. Now give everyone $1 million in old Monopoly money (or you can use sticky dots). Tell them it’s their budget and make them spend it all communicating to the various stakeholders. Where do they spend the money? The three groups that receive the most monetary votes are where you’ll concentrate your research.

Here's an example: I had a meeting with a healthcare nonprofit I was working with. Everyone who came into the room claimed they were focused on media measurement. I asked them to list their stakeholders, passed out the Monopoly money, and told them to spend their budgets. Of a total of $5 million potential dollars, they spent $1,165,000 on employees, $1,185,000 on donors, and $750,000 on corporate partners/sponsors. And it turned out they only wanted to spend $165,000 on media. The exercise made them understand that media was a only a means to an end, not the ultimate goal. They learned that they needed to focus on measuring the stakeholder groups that mattered the most.

Articulate the reasons why those stakeholders make the organization successful. Get everyone in the room to articulate how a good relationship with each particular group helps achieve the mission. Make sure you ask “So what?” at least three times until you get to the real reasons.

For example: Friend-raiser events are good. So what? Because more friends means more volunteers. So what? Because a bigger pool of volunteers means we don't have to hire the labor. So what? We save money.

Step #2. Define specific metrics.

Now you must determine just what data you are going to gather. Generally, your metrics will depend on your goals, and your goals will depend on your mission.

For example: Let’s suppose you are the Greenville Land Trust, your mission is to conserve land, and your goal is to raise money to buy the land. One money-raising strategy would be to grow your dues-paying membership, and a tactic could be to have a really fun event that attracts new people to the organization. Then a short term metric would be Number of New Members, and a long term metric would be Percent Increase in Donations.

(What you probably really want to know is if acquiring new members is a more effective money-raising strategy than are the alternative strategies of, say, direct mail, social media, grant writing, or just putting pressure on a few big donors. In which case you will want to develop metrics for all your strategies.)

To provide useful information, each metric and goal must have three components:

1. Something that is calculable, like money, a percent, or a ratio;
2. Something to compare it to, like last week, last month, or the competition; and
3. A date by when this will occur. 

So, for example, if you want to increase your trained supply of volunteers, then you might have as your metric: “By third quarter 2012, achieve a 5% increase in the number of trained volunteers, at the same or lower cost per volunteer as at present.”

Step #3. Assess your research tool and data situation.

Our experience with nonprofits is that they always have more measurement tools and data available than they think. Check all your organization’s departments—especially development and marketing—to understand what data systems they have in place. You’ll probably be surprised at much of the data you need is already being collected.

Step #4. Get buy-in from the top.

Using your data to make changes is your goal here, and for that to happen you will need to get everyone on board. Your efforts will be worthless if no one takes your results seriously. So before you actually do the research you’ve got to make sure that people at the highest level, including the board, are in agreement with your definition(s) of success and measurement priorities. Tell them the research you plan to do and why you plan to do it. Explain what data and results you are likely to get and how this information will be valuable for making decisions. If you get any pushback or negativity, then you’ve got to deal with it now, or risk having your results ignored.

Step #5. Collect data, analyze results, and draw conclusions. 

Now you can actually go out and collect your data. You might be doing a survey, or getting membership figures from Development, or looking at web traffic stats. Maybe you are correlating media coverage to donations, or tracking social media activity against new memberships, or comparing the annual Auction with the biannual Gala.

When you analyze your results, what is most important is that you understand the context: Why are they the way they are? What do they mean for improvement? We’ve seen more measurement systems fail for want of context than any other single reason. If for years you’ve been hearing: “This bar on the chart is bigger than that bar,” or, “There was a big spike in June,” without the context and the So What?, then your analysis is meaningless.

• Why is that bar bigger, and what are you going to do next time to make it even bigger?
• What caused that spike, and how can we do that on regular basis?
• How will you use the data to do better planning?

That’s what makes a measurement system useful and successful.

Congrats! Now do it all again. 

As I said above, measurement is a continuing process of improvement. So use your conclusions to make changes to your programs or strategies. Next year or season or time period, collect more data and improve your programs again. Happy measurement.

###

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. 

Social Media Case Study Proves the ROI of Emily

How the USO Used Social Media to Greatly Increase Donations

by Katie Delahaye Paine

So many people want to know what the ROI of social media is. Last time I checked there were 27 million references to it on Google. Oliver Blanchard has even written a book about it: “Social Media ROI: Managing and Measuring Social Media Efforts in Your Organization.”

But sometimes the best way to understand the ROI of social media is to look at case studies. Here's the story of how the USO used social media to greatly enhance their fundraising. I call it “The ROI of Emily.”

The USO is a nonprofit non-governmental organization that provides services and support to actively serving members of the military and their families. When you say “USO” many people think of Bob Hope (that's him to the right) and the USO tours, but the tours are just a small piece of what they really do.

One of the problems that the USO faces is that the Bob Hope generation is dying off, which means fewer volunteers and donors. The other problem is that most of the people the USO serves weren’t born when Bob Hope was doing his tours. So, like many other nonprofit organizations facing an  aging donor base, the USO turned to social media to reach out to a new generation of supporters.

Emily Hall Joins the USO

In March of 2009, they hired Emily Hall, known as “Em” as well as @emilyhaha, to manage their social media outreach. At the time, my company was measuring the USO’s media coverage in both traditional and social media. One of the things we tracked was the nature of the conversations. We would look at every mention and decide whether it was expressing support or merely making an observation (for a full explanation of our 27 different types of conversations, read: “5 Reasons to Love 27 Types of Conversation and 19 Types of Videos.”)

After Em arrived, the first thing we noticed was a distinct shift in the nature of the conversations. For months they had been mostly observational, for instance, “I’m near the USO room at the airport,” or, “Just noticed that XYZ was at the USO last night.” But shortly after Em began her social media efforts, the majority of conversations were expressions of support  for the USO, mostly related to her outreach on Facebook and a “Support USO Friday” campaign. See the following chart:

While it was great that people were expressing more support, So what?

But as I always say, in order for measurement to be meaningful you have to ask “So what?” three times. While it was great that people were expressing more support, So what? Did that translate into meaningful benefits to the organization? 

The answer took a bit more digging. One of the key metrics for any nonprofit organization is online donations, so when we matched compared donations to the overall visibility of the organization, there was clearly a correlation between exposure and donations. See the following chart, which plots donations and opportunities to see in the spring of 2009:

But once again, So what? What else could we learn from the data?  

Let’s take the various spikes one at a time. The red line indicates overall media exposure, the blue line indicates dollars. The first major spike was driven by a White House event where the newly elected POTUS, Barack Obama, wrapped care packages for the USO on the White House lawn. It was a great visual event, and was widely covered by broadcast news outlets, including all the morning talk shows. At the time, it spurred the greatest increase in online donations in the history of the organization, generating some $65,000 an hour and actually crashing the USO server!

Then, a month later, the USO made another broadcast media splash by taking a crew and one of their mobile Internet vans to Afghanistan to show exactly how they support the troops by providing them with the ability to communicate with home from remote locations. While the overall exposure level was about the same as with the Obama event, the online revenue wasn’t nearly as high.

Then in June, the USO took the entire cast and crew of The Colbert Report to Baghdad and did a week-long series of broadcasts. Each broadcast included messages of  “Donate to the USO”  with very specific instructions. The news of these broadcasts were picked up widely on other media outlets and once again, donations spiked.  

But, so what?  

The lesson is that it’s a lot cheaper to get President Obama out onto the White House lawn than it is to support The Colbert Report in Baghdad. So on a cost-per-donor basis, the White House event was much more efficient.

But again, so what?  

We needed to understand why the White House event was so much more effective at fundraising than was the Afghanistan event that generated equal amount of exposure. What was the difference?  

The difference was Em

Here’s where Emily comes in again. As it turns out, Em, through her Facebook outreach, had tapped into the enormous network of Obama supporters that was still in place after the election. Her posts emphasized that if people couldn’t come out and support the effort at the White House in person, they could still support the troops. And, her posts included a “Donate Here” button, which clearly had the desired effect.

Now, I call this “The ROI of Emily” because it addresses the revenue/cost benefit situation. Astute readers of The Measurement Standard and my Measurement Blog will notice that I am not adhering here to my usual strict definition of what ROI is: Percent return on money invested. I don’t know what Emily was paid, so we can't calculate proper ROI. I’m sure it’s a significant amount, though, since the donations she pulled in far exceeded whatever she was paid.

So that’s the ROI of Emily.  

###

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements.

70+ Essential Resources for Making a Difference on the Social Web: mashable

Those of you in nonprofits are probably very familiar with The Networked Nonprofit, the book by Beth Kanter and Alison Fine that shows you how to use the power of social media to help your nonprofit grow, change and succeed. Here’s a companion resource for you: mashable’s list of its 70+ Essential Resources for Making a Difference on the Social Web: “Learn how group buying is supporting charity, see why activists are becoming “hacktivists” and brush up on the latest social media tools for non-profits.” --WTP

MomsRising Is The Measurement Maven of the Month

Twenty plus years ago my mission was simply to get people to measure. Once measurement became more common, I focussed on getting people to measure correctly and accurately. Now, thanks to the Barcelona Principles and other efforts, there is official recognition that there is a right way and wrong way to do things. That’s real progress.

Today’s big problem is that there are so many measurement tools and so many measuring systems that most communicators are overwhelmed with the data. They just don’t know what to do with it all. So now the issue is getting people past the data overload so they can actually gain some insight from all their raw information. 

MomsRising has not just solved this problem, but has become a magnificent role model of an organization that uses data effectively. We are proud to recognize MomsRising as our Measurement Maven of the Month.

You can read MomsRising’s full story on Beth Kanter’s blog. (And, of course, in the book that Beth and I are now writing, “Measuring the Networked Nonprofit,” to be published by Wiley early next year.) The short version is that this grassroots networked nonprofit has grown to millions of members primarily because it is a data-driven organization.

MomsRising measures all of its activities, but far more importantly, it uses the data to decide what to do next. It reviews the data, kills programs that don’t work, and then puts more efforts towards those programs that the data supports. What a concept! Who says that non-profits aren’t savvy businesses? --KDP

###

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements.

Nonprofit Measurement, Beth Kanter, and the Nonprofit Technology Network

If you work with or for a nonprofit organization, then Beth Kanter and her Beth’s Blog are probably already high on your go-to list of resources. Click here for her posts specifically on measurement.

If your nonprofit is just beginning to explore how to use data and measurement to improve your results, then don't miss this post of hers: “How Does Your Nonprofit Use Data to Improve Results?” You'll find an inspirational video (which we’ve embedded here, too) and you may very well recognize your own organization in Beth’s taxonomy of measuring nonprofits.

The Nonprofit Technology Network (NTEN) should also be on your short list of resources. Don't miss this post, which Beth also links to: “Four Ways Nonprofits Can Increase Their Impact with Data,” and Beth includes a list of several more useful articles on the NTEN site in the comments to her post.

Note also that Katie Delahaye Paine and Beth Kanter will co-author a book to be published next year: Measuring Networked Nonprofits: Using Data for Social Change.
--WTP