Mitch Wagner has a very informative interview/post on his blog with a guy who works for Chipotle Restaurants and is replying to bloggers about the hepatitis scare. Nice anecdotal measures of the value of social media to a large company.
This comparative study of Web-based survey samples and paper-and-pencil surveys (by Samuel D. Gosling, Simine Vazire, Sanjay Srivastava and Oliver P. John and which appeared in American Psychologist) is now four years old, but its conclusions are probably still valid: Yes, Web-based self-report and self-selected samples can be trusted for surveys. From the study's summary:
"Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N 361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and age. Moreover, Internet findings generalize across presentation formats, are not adversely affected by nonserious or repeat responders, and are consistent with findings from traditional methods."
Jim
Macnamara"s "Measuring
Up"
Welcome to a new age where media are software and the audiences are the networks.
I don't know about you but, as fascinated as I am with media developments, I am fed up with hearing the term "new media." And I am not too enamored with "social media" either.
What's So New About New Media?
Why? First, because many of the media that we are talking about are increasingly not new. OK, so Web 2.0 has upped the ante with interactivity and participation, but newsgroup chat rooms celebrate their 30th anniversary next year, having been conceived by Duke University graduate students Tom Trucott and Jim Ellis in 1979. The term "Weblog" was created in 1997 and bloggers have been blogging for a decade. Google is into its second decade, celebrating its 10th anniversary as a company in 2008, while MySpace will celebrate its 10th anniversary next year. Even YouTube and FaceBook are three and four years old respectively and, with hundreds of millions of users between them, are hardly new.
Apart from being increasingly inaccurate, the term "new media" leads us to an inevitable terminology trap when the next wave of media developments arrive. Web 3.0 is already under construction and new hybrid forms of media are evolving – what Roger Fidler calls "mediamorphosis." Rather than a choice between "old" or "traditional" and "new" media, which suggests a simple two-horse race, we are living through a period of ongoing media and communications change.
What's So Social About Social Media?
"Social media" is also a problematic term. As much as social networking has wide interpretations and social network mapping is all the rage, "social media" suggests to most that these media are primarily used for chat and gossip, friendship, dating, etcetera. It is this confusion that is causing many businesses to ignore these media or underestimate them. In reality, so-called social networking utilities and social media are making and breaking brands and products every day, building and destroying political careers, and shaping corporate reputations. They are used for civic and political engagement, research, job searching, marketing, shopping, knowledge sharing, and a host of other purposes.
While the U.S. progresses through its primaries in preparation for the November 2008 Presidential election, Australian had a national election in late 2007 which was widely dubbed "the YouTube election," and resulted in a new government. The new Prime Minister, Kevin Rudd, and his party were elected in a landmark campaign spearheaded by Kevin07, a Web-based strategy which extensively used MySpace, blogs, YouTube and other so-called social media. Even though many conservative politicians clung to traditional media advertising -- particularly those who lost the election -- Web 2.0 type media were used for political communication and civic engagement by a large number of both politicians and interest groups.
The U.S. Presidential race is also seeing Web 2.0 media used at an unprecedented level -- even more than in the 2004 election which was described as a critical turning point in media use for political electioneering. (See, for instance, this article, previously in The Measurement Standard.)The term "social media" fails to reflect the serious and substantial communication that is flowing through these channels.
To take the point further, a few hours of research will show that many of the so-called social media are also downright anti-social. Political spoofs and parodies that ridicule, mash-ups of children's nursery rhymes with lyrics replaced by obscenities, and various types of pornography, racism, and other abuses are features and challenges of the Internet and the new forms and genre of media that it facilitates.
What's In a Name?
So what do we call emerging media forms and genre? And is it important what we call them? I suggest it is because our way of describing things frames our understanding. Language limits or delimits the concepts we deal with. It seems clear that we need a review of terminology in relation to media as convergence escalates. In preparing a public lecture which I am due to deliver in June, I compiled a list of 32 different terms used for media today. Many of these are based on delivery systems that are increasingly redundant -- such as film, video tape, broadcast, and so on. Even traditional terms such as "newspapers," "press," "broadcast," "radio" and "television" no longer adequately describe our media, as newspapers are less and less provided on paper, radio programs are increasingly distributed as podcasts rather than broadcasts, and television content is being "transmitted" via the Internet and watched on computers and even hand phones. And "phones" are not phones any more.
The benefit of a review of terminology is that we would find we can dispense with more than half the terms in use and simplify discussion considerably. What does it matter that content is distributed on paper, plastic, magnetic tape or disk, celluloid, cable, broadcast waves, or in jello? Only two things seem to matter: content and users -- whether they are producers or consumers, or a combination of both, as reflected in the terms "prosumers" or "produsers."
This raises three points that I will throw out there for comment. The first observation is that media are becoming immaterial. By that I do not mean that media don't matter per se; I mean the materiality of media is becoming unimportant. With convergence, content pays no mind to the medium on which it is distributed -- nor do most users. In the digital art world, Lev Manovich talks about "post-media" referring to the same notion, so I am not alone in this thinking.
In the same way, hardware technology such as computers and telecommunications networks are disappearing and becoming invisible. The invisible computer was first forecast in 1998 by Donald Norman and research continues through the Disappearing Computer Initiative in the U.S. Similarly, cables and wires are disappearing as we move to wireless. And "logging on," which was an often troublesome ritual that regularly reminded us that we were entering a complex world of machines, is increasingly being replaced with "always on." But it is not only the increasing physical invisibility of hardware that is significant; what is most significant is the growing psychological invisibility of hardware. Today, what Marc Prensky calls "digital natives" and assimilated "digital immigrants" move seamlessly and effortlessly between sources of content without a moment's thought to the hardware infrastructure that delivers them.
Today media are software -- intellectual property in the form of both applications and content. And audiences are the network, actively connecting, linking, redirecting, forwarding, and injecting local comment and static into communications.
Welcome
to a new age in which media are software and audiences are the
networks. 
Dr
Jim Macnamara MA, PhD, FPRIA, FAMI, CPM, FAMEC became Professor
of Public
Communication at the University of Technology
Sydney in late
2007. His 30-year career in journalism, public relations
and media research culminated in the 2006 sale of CARMA
Asia Pacific,
which he founded, to Media Monitors. He worked as Group
Research Director with Media Monitors - CARMA Asia Pacific following
the sale
and continues as a Consultant with the Group.
Book
Review:
Strategic Communications
Planning For Effective Public Relations and Marketing, 5th Edition
by Laurie
J. Wilson and Joseph D. Ogden
Kendall/Hunt Publishing, 2008, 284pp
Buy it at Barnes&Noble
Review by Katie Delahaye Paine
A new PR/Communications/Marketing book shows up in my mailbox every other week. If I know the author, I might actually open it up and read it. If not, the first thing I do is see if there's even a mention of measurement anywhere in the text. If not, it goes right into the donations bin at the Durham Public Library.
When Strategic Communications Planning arrived this week I immediately conducted the measurement test. With my usual skeptical and biased perspective I turned to Chapter 10, "Effective Communications Measurement and Evaluation." The first thing I read was a page on measuring social media -- which I apparently authored. Not only does it promote my basic rules of measurement, it offers up "Katie Paine's Nine Immutable Laws for Measuring Social Media."
Flattery gets you everywhere, of course, so I was hooked. I went back to the beginning and read the entire book. It's great. I suggest that this book be required reading, not just in every PR class on the planet, but in most PR firms as well. I know anyone in my organization that even dreams of speaking to a client will have to read this book first. It's a wonderful overview of how to do PR right.
The core premise is that public relations is all about building and maintaining relationships. (Blatant self-promotion warning.) In many ways, this book is the prequel to my own book, Measuring Public Relationships. If you want proof, look no further than the title of Chapter 1: "Trust and the Relationship-Building Approach to Communications."
It starts with a wonderful explanation of the transition from one-way, publicity-driven communications to the synchronous two-way conversations at the heart of PR today. The reason for the new 2008 edition is the impact of social media on PR relationships, and there's a lot more information on social media than any other PR text book I've seen. And it's great, highly practical advice.
My favorite part of the book is its organization and simplicity. Take this, for example:
In public relations we are ultimately trying to get people to:
1. Do something we want them to do
2. Not do something we don't them to do
3. Let us do something we want to do
And of course, once you've identified which of these three outcomes you want, measurement becomes remarkably simple.
The style and format are a key element to the book's success. It's beautifully laid out, with clear, easy-to-read text and lots of great "Tips from the Pros" that reinforce the points being made. It's full of case studies and exercises that prove and back up the lessons.
So whether
you're new to PR or an old pro who wants to freshen
your skills and learn more about social media, this is a great place
to start.
Public relations measurement evaluates exactly the sort of metrics that define the strategy that al Qaida has been using to enlarge its support: media impressions, media content, key messages, contributions and new members. Public relations should be not only measuring the war on terror, but, by virtue of tracking the most important metrics, should be defining the war on terror too. (See "Knowing the Enemy: Can social scientists redefine the "war on terror"?")
One of the more interesting aspects of the war on terror is that there is, (so far, that I know of) no way to know if we are winning or losing, or how we will ever know if we do win or lose. And that's where public relations measurement comes in: If we can use public relations metrics to define progress in the war, we can use them to define success and failure. Public relations measurement can win the war on terror by defining when the war will be over.
And here's how we do it: Let's say that al Qaida's media and marketing efforts now generate X impressions of their key messages each year, which in turn result in Y dollars of contributions and Z new members. We gather and track this data for the last ten or twenty years, and correlate it against terrorist activity of all sorts. (Yeah, it's a big job.) We then will understand what level of exposure to key messages (or Osama bin Laden videos) is required to generate what levels of recruits. Or to generate one new suicide bomber.
Then we define success in terms of the metrics. So, we could define "winning the war on terror" as when al-Qaida's impressions, contributions and recruits are dropping. And then we define victory as the point at which these metric fall low enough so that al Qaida can no longer function effectively. If Osama bin Laden doesn't get his message out, or if no one believes it enough to do anything, then we've won. --Bill Paarlberg
Katie
Delahaye Paine's
Top 10 Signs that
It's the End of the World
as
We Know It
10. I spent more time on Twitter/Flickr/Facebook yesterday than I did on email.
9. Gatekeepers? What's a gatekeeper? 52% of journalists use blogs to research facts and find stories.
8. Deadline? What's a deadline? 92% of journalists say their online editions are allowed to scoop the print version.
7. It's easier to put my message on M&Ms than it is to get it into an A-list blog.
6. The Obama Yes We Can YouTube video is seen by more people than the number who watch an evening of Monday Night Football.
5. IBM receives more leads, sales and exposure from a $500 podcast than it does from an entire ad.
4. Employers no longer check references, they check Facebook and MySpace.
3. Procter & Gamble is preaching "letting go" and "co-creating" all its marketing with its customers
2. Wikipedia is nearly as accurate and just as credible as the Encyclopedia Brittannica. And a lot more people use it.
1. Google
has replaced my thesaurus, encyclopedia, dictionary, and long-term
memory.
by Katie Delahaye Paine
Ah Springtime! When a young man's thoughts turn to love, or basketball, or, if you live in New Hampshire, politics. Given that New Hampshire has the third largest legislative body in the English speaking world (after the British Parliament and the US Congress), and that once a year in March a million or so average citizens get together in town meetings to determine the fate of the state, you can understand why the term "March Madness" takes on a whole new meeting up in these parts.
When measuring relationships with voters, as with any measurement program, the type of system you put in place depends on your objectives and your role. Below is a list of tools you need for each type of program. One important point first...
Timing
is more critical than usual
Because the length and success
of voter measurement programs are so strictly defined by the election
cycle,
you can't
afford to
waste
time or opportunities. Probably
the most important element in any political measurement program is
to
have
the data
on hand
when
you
need it.
And you need it whenever there's a decision to be made about how
much money to spend or what tactics to use. You don't need
it after the votes have been counted.
Measuring PR Efforts on Behalf of a Candidate
My company, KDPaine & Partners, measured political PR for one of the country's best political PR pros, Doug Hattaway, former spokesperson for Al Gore and Tom Daschle. Doug was working on Chellie Pingree's effort to unseat Susan Collins as Senator from Maine. What Doug needed was a measurement system that would tell him which tactics were working or not working in a campaign. The challenge was to do it fast enough so the data would be useful, and cheap enough so the campaign could afford it.
The good news about working with political campaigns is that volunteers are generally plentiful, and in this case the volunteers were charged with collecting the clips from the local publications. If you don't have volunteers to do the clipping, you need to hire a local clipping company or someone like Bacons or Burrelle's to collect the clips for you. Small-town newspapers are just too important to miss. Electronic data aggregators like Nexis and Factiva do not work on local or statewide campaigns; they do a terrible job collecting clips from anything smaller than the Boston Globe. Even Cyberalert and Custom Scoop miss far too many of the small town weeklies.
One we had the clips about both Pingree and Collins in hand, we began to analyze them for positioning on key issues, tone of coverage, type of article, subject of the article and who was quoted. We compiled them monthly and issued a report that included charts like this (click the charts to see them bigger):
We could quickly identify the tactics that were most successful. For example, a visit by Hilary Clinton in support of Pingree was far more successful in garnering visibility for Pingree than any other action. However, Pingree's visit to Washington in support of her health care reform legislation, while it didn't get anywhere near the visibility, was more successful in positioning her as "the health care candidate."
The analysis also revealed weaknesses in Pingree's campaign and strengths in Collins' campaign. While Pingree was almost entirely focused on her health care message, Collins was making points with the voters on the environment and campaign finance reform.
Additionally, the research enabled the campaign to track the correlation between visibility and contributions. By tracking Pingree's share of exposure over time against the contributions over time, the campaign could determine the level of additional exposure necessary to generate the requisite contributions.
In the end, the earned media that was generated by Hattaway's PR efforts narrowed incumbent Collins' lead substantially.
Measuring Lobbying Efforts
We ran into a government relations person for a major organization who wanted to know how to measure the impact of his efforts. He was trying to get a bill passed that would be highly advantageous to his company. We began to design his "dashboard," and asked him what success meant. He got as far as "If the bill passes, I'm successful." We pointed out that if the bill failed it would be too late to do anything about it.
So what he really needed was to monitor the tone of conversation around the bill in question so that he could tweak his PR program according to its progress. We recommended that he establish a system that gives him feedback all along the way.
There are several ways to do that. The easiest method is to track who is saying what to the media or in local speeches about the bill. Are key influentials supporting it or trashing it? Or is no one talking about it at all? More important to track, however, is the bill's progress through the legislative process. Is it being heard, tabled, or moved? Who are the cosigners to the bill and are they actively promoting it? Is the opposition gaining or weakening? What do bloggers say? What's being discussed in news groups? Are there any list serves you need to track?
Some of these data points can be gathered via media analysis, others should be part of your regular legislative tracking process. Still others may require a poll of the constituency to determine the level of support, and the perceptions of the voters and/or legislators.
If at all possible, a once-a-year survey of your key legislators is recommended to test and evaluate the health of the relationship between your organization and the elected or appointed officials you are striving to measure.
Measuring your Relationship with Publics, Both Active and In-Active
Your constituencies come in several forms, from students to senior citizens. You should measure relationships with as many different stakeholders or constituencies groups as you can afford. It may be useful to segment the different publics by gender, age, length of time in the area and political leanings so you can identify any pockets of opportunities or threat within the community.
Clearly active stakeholders who are most likely to protest, boycott or otherwise cause trouble are the most important to understand. However, inactive stakeholders that are clueless about your programs and therefore don't care and won't get engaged can be just as dangerous.
We recommend using the Grunig relationship model to test your relationships and in particular the level of trust that each public feels towards you. Trust is a key component of success in any political campaign. If it's not there, or if it is recently lost, you will have a significantly harder time achieving success. Alternatively, if your constituents trust you, as do those of Alan Greenspan or John McCain, you have a lot more leeway with those publics. Therefore it is critical to establish a benchmark level of trust to begin with. Subsequently you should conduct regular trust/relationship measurement studies to gauge the level of trust and engagement over time. Conduct these studies as often as possible so you can tie any changes in the trust and relationship scores to actual actions you may have taken.
The Ultimate Measure: Votes
There's a ton of great data in any set of election results. The challenge is to use it to make more informed decisions. When I ran for town council here in Durham five years ago, I knew that my efforts to go door-to-door in specific communities paid off since the number of votes I received from those communities far exceeded my expectations. I also knew that my signage (bright purple and white with the slogan "No Paine, No Gain") was successful since I had the highest name recognition, and ultimately the highest vote count of all the candidates.
In larger elections,
there's far more granular data to be gathered,
and much of it is readily available online. Using this data you can
frequently identify the specific areas or pockets or demographics
you've targeted and thus measure your success in getting those groups
to vote
and/or vote your program or candidate.
by Katie Delahaye Paine
In this article I suggest that media analysis programs would more accurately reflect articles' influence on readers if the integrity of the media outlet was taken into account. This could be achieved by developing a standard measure of media integrity which would used to rate different media outlets for different shareholder groups. The resulting Integrity Index would be used to weight coverage, thereby achieving a more accurate measure of media coverage's impact on consumers.
To understand why an Integrity Index is needed, consider:
Item
#1
Why measure what no one believes?
Mazen Nawahi, in an impassioned speech at the 2008 Dubai Measurement Summit, raised the issue of journalistic integrity and the degree to which the integrity of a particular media outlet should be accounted for in any measurement program. Put another way, if:
then why would anyone believe anything that was printed?
And if all your media coverage in a particular outlet lacks credibility, why would you include it in your measurement results? For example, in 22 years in business I have yet to have a client want me to include The Weekly World News in its media list. So how do people justify measurement programs that include stories that do nothing to achieve their goals?
Item
#2
Credibility is in the eye of the beholder.
Suppose a highly credible blogger gets into a mud-wrestling contest with the blogging equivalent of a pig. The contest attracts a lot of attention, a lot of arguments fly back and forth, and a lot of dirt gets thrown around. Now for the people that are into farm animals, the arguments of the pig might be very credible. However, people interested in learning something of value professionally will no doubt pay more attention to the words of the credible blogger. But the real question is: Do you weight coverage of them both equally in your media analysis reports?
How to correct for influence?
Taking examples of this sort into consideration, it is easy to see why some sort of authority or influence weighting of coverage would be of value. There has been a fair amount of research done in this area. For several years, Angie Jeffrey at VMS has been studying various weightings of media coverage to determine which factors most directly impact sales. Her research first found that share of discussion was effective, but she postulates that advertising rates are more telling because they reflect the importance of the various media. (See this paper and this paper.) I think it's a step in the right direction, but I generally frown on the use of advertising rates (and of AVEs), so I'm skeptical.
The Media Integrity Index: Why not rate coverage on integrity as rated by stakeholders?
I suggest that it's not the ad rate but the integrity of the publication that most determines consumer credibility and thus drives consumer action. And I'm suggesting that to rank a media outlet in importance, we should ask our stakeholders how they perceive the media outlet in terms of integrity.
I raised the question on Twitter and got some interesting responses:
• "It's all relative... What is good for one is bad for the other. It has to be specified for a purpose, customized."
• "Integrity or perceived integrity? Seems there's a difference there that's highly subjective." --Ryan Anderson
• "Integrity is the measure of perceived relationship (believability) of one-to-many listeners. So measure relationships. I would suggest looking at the way political integrity is measured. It would be a perceived metric, measured by polling audience. --Videodred
• "What's the goal? What's the client want it to say? In the case of journalists, what's the relationship?
• "Integrity has to be measured parallel to influence as audience determines credibility of source." --Mike Maney
So in one way, my proposed Integrity Index has to start with the goal of the coverage. What is it that the company or organization is trying to accomplish? Consider it this way: If you don't care who or what your brand is associated with, and you want exposure pure and simple, then you don't care about the integrity or credibility of whatever media outlet is talking about you. On the other hand, if you're trying to establish a reputation, or build brand loyalty and trust, or trying to reach an audience with certain media preferences, then the credibility of media to your audience should matter a great deal.
How to determine integrity?
So how do we determine which media is trustworthy and credible, and which journalists have or do not have integrity? The wrong way is to look at a list of media and assign weights or values to each one yourself. What you think doesn't matter. All that really matters is what your customers or stakeholders or members or constituents think. So you need to ask them. The best way to do that is to use the Grunig Relationship Instrument.
Now you probably don't need to ask them about all 150 publications on your media list. Remember: Never ask a question about something that can't be changed. If The New York Times is what your boss' boss reads every morning, there's no chance in hell you'll ever take it off you top tier media list. Even if your target audience is nine-year-old girls. So start with a list of suspect publications, blogs or any other type of media outlet.
Remember that different stakeholder groups may rate a given media outlet differently. As I pointed out above, integrity is not a fixed standard. For instance, we would expect Republican viewers to rate Fox News and The New York Times differently than Democratic viewers. It is conceivable that you will want to generate different Integrity Indexes for different stakeholder groups.
Below are some survey questions adapted from the Grunig Relationship Instrument. You may wish to develop your own, based on the specific components of relationships you wish to measure (refer to this paper). Ask your stakeholders whether they agree or disagree with each statement as it pertains to each media outlet, then use the responses to rate the outlets.
1. This media outlet treats people like me and organizations like mine fairly and justly.
2. Whenever this media outlet makes an important decision, I know it will be concerned about people like me.
3. This media outlet can be relied on to keep its promises.
4. I believe that this media outlet takes the opinions of people like me into account when making decisions.
5. I feel very confident about this media outlet's skills.
6. This media outlet has the ability to accomplish what it says it will do.
7. Sound principles seem to guide this media outlet's behavior.
8. This media outlet does not mislead people like me.
9. I think it is important to watch this media outlet closely.
10. This media outlet is known to be successful at the things it tries to do.
Industry-wide integrity standards?
Now, the logical question is, "Why aren't we doing this as an industry?" Shouldn't we be factoring in credibility based on some industry-wide standard? It certainly is a more accurate weighting factor than simple eyeballs or ad cost. But the reality is that your stakeholders aren't going to be identical to my stakeholders, and what matters is how a significantly valid sample of your stakeholders feel. It would be awfully complex and difficult to set industry-wide standards for many different stakeholder groups.
On the
other hand, if the PR industry wanted to take on a project to
try and accurately weight publications based on their integrity,
the
world would most definitely be a better place. And it is not far-fetched
to imagine an industry-standard Integrity Scale or Survey that could
be used to determine the Integrity Index for different stakeholder
groups.
Search The Measurement Standard
Recent Comments