The Institute for Public Relations will bring its annual Measurement Summit and Conclave back to Durham, New Hampshire October 27 - 29th. The Summit is billed as, “The annual gathering for the best minds and thought leaders in research and measurement, and those who want to learn from them.” The Conclave is a multidisciplinary group working to simplify and unify the measurement of social media. The fee is $400, and attendance is limited to 50 persons.
Summit session topics will include:
For more details, see the Institute for Public Relations website, where you'll find the agenda, online registration, and more details.
###
Couldn't make it to the 2014 AMEC Summit on Measurement in Amsterdam? No worries:
-- Overview: A quick summary of the conference from Barry Leggetter, AMEC CEO. See the entire 2-day program. See photos.
-- Awards: AMEC bestowed its annual Communications Effectiveness Awards. Read a summary, and what Salience Insight won.
-- New PR Measurement Model: Jim Macnamara unveiled his MAIE model. Read a condensed version, or download the entire paper.
-- Social Media Measurement: AMEC released its new SM Measurement Framework. Read Don Bartholomew's summary. Download the handy User Guide. ###
Presented as part of the annual AMEC Summit on Measurement, the AMEC Awards seek to recognise and promote best practice in communication planning, research and evaluation. See the full list of Awards winners.
In a strong display from firms in Asia Pacific, Ogilvy Public Relations scooped four Golds and the Grand Prix International Award with work from its offices in Taipei, Beijing, and Thailand.
Barry Leggetter, AMEC CEO, said: “This year we had more entries than ever, and of such a high calibre, that our judges awarded more than one Gold in several categories.” To top off their Golds success, Ogilvy also won the Best Newcomer Award in a new AMEC Summit live voting category for its work for the Information Office of People's Government of Chengdu.
Hot in contention were three firms who each won three Golds: iSentia from Australia and South East Asia; Pegasus PR from Brighton, UK; and Gorkana Group from the UK and US. Gorkana also won the prestigious Grand Prix Platinum Award for its work for the UK’s Department for Food and Rural Affairs (DEFRA).In other major awards categories, the Chairman’s Award was awarded to two communications professionals to mark their contribution to the work of AMEC in advancing the benefits of measurement and analytics. The winners were Alex Aiken, Executive Director, Government Communications, HM Government UK and Andre Manning, Global Head of Public Affairs & PR, Booking.com.
This year the standard was so high that judges made three awards in the Young Professional category: Ngaire Crawford, of iSentia; Mounir El Agouz, Salience Insight; and Cate Setterfield, The Mediaverse.
###
by Barry Leggetter, AMEC CEO—Dave Senay, global CEO of FleishmanHillard, was one of three global PR leaders to take part in the key segment on PR and the measurement industry at the AMEC International Summit on Measurement 2014 in Amsterdam. Joining him in the Measurement and PR session were Kevin Murray, Chairman, the Good Relations Group and Matt Neale, President, International, GolinHarris.
“Use the Barcelona Principles!”
Senay urged PR firms to pro-actively use the Barcelona Principles in their work in and with clients: “The Barcelona Principles represents a significant undertaking by this community over a period of three years. My question to everyone here in the room is how often do we refer to it when we are developing metrics and refer to it in conversations with our clients to propose new measurement work?”
Senay said that irrespective of any firm’s individual proprietary offering, the Barcelona Principles represented an industry-wide adopted standard which needed to be constantly reaffirmed with clients.
In addition, Kevin Murray said: “Great communication is mission-critical in today’s radically transparent, hyper-fast world. Measuring the effectiveness of communication is absolutely vital to the future success of clients and agencies alike. I came away from the Summit really impressed with the steps the measurement industry is taking to further improve the value of its work.”
Matt Neale added: “I was impressed by the turnout and the professionalism of the AMEC Summit. It is a great organisation which seems to be going from strength to strength.”
New AMEC Social Media Measurement Initiative
A new initiative, developed by AMEC’s Social Media Measurement Committee to measure the impact of social media, was announced at the Summit. It offers practical tools and a consistent, meaningful approach for measuring social media that can be applied in any organisation, regardless of size. By providing a clear link from the planning stage, to setting objectives, to measuring results, the framework and user guide provide straightforward advice to ensure social media measurement success.
The framework’s ability to work across the communications discipline regardless of type and objectives has resulted in industry-wide support. The framework and user guide are available for download as a pdf, on an interactive microsite at www.social-media-measurement-framework.org, and are accompanied by a video with a real case study to bring it to life.
Couple's Therapy
In a packed two-day programme, a lighter session with a serious point was the workshop, “The stormy marriage of PR consultancies and measurement providers – a couple’s therapy.” Giselle Bodie, Chief Executive of Salience Insight, role-played the “partner” at a fast-expanding media intelligence company. David Rockland, Ketchum Partner and AMEC Chair, played the part of the “hard-charging senior manager” from a global PR consultancy. The aim of the workshop was to identify the pressure points between media intelligence and PR firms and discover how to work more effectively.
###
Barry Leggetter is CEO of the Association for Measurement and Evaluation of Communication (AMEC). He has been responsible for initiatives which have turned AMEC into an international body, enjoying rapid growth with members in over 40 countries. He previously spent 25 years in PR consultancy and was UK Managing Director for three major public relations networks, Porter Novelli, Fleishman-Hillard, and Golin/Harris, climaxing his PR career as international Chairman of Bite Communications.
At the AMEC Measurement Summit a couple weeks ago, Jim Macnamara unveiled his MAIE model, a new conceptualization of what public relations measurement can and should be. The MAIE model provides insights into several of the major difficulties that measurement faces, including why there is continuing resistance to measurement, and why it has been so difficult to use measurement to demonstrate the value of PR. Here Jim presents a condensed version of his AMEC presentation.
by Jim Macnamara—With numerous models, principles, guidelines and tools for measurement and evaluation advocated by academics, professional industry bodies, and measurement service providers, it could be asked ‘does the PR and corporate communication sector need yet another model?’
The answer is yes. This new MAIE model offers an approach that overcomes four largely unrecognized and under-researched obstacles to effective measurement and evaluation by undertaking deep analysis of a range of internal and external quantitative and qualitative data. This analysis provides organization management with insights to productively inform future strategy and achieve organizational outcomes, as well as guide future communication planning. Such a forward-looking, value-adding approach changes the timing, scope, and methodology of evaluation in productive ways for both practitioners and organizations.
To begin, let’s examine several obstacles to measurement and evaluation of outcomes (as opposed to outputs) and demonstration of value to an organization that are not addressed in other models.
1. The rear view mirror approach
The first is that most M&E research looks backwards, producing reports that provide a retrospective performance review of what happened in the past. In some cases, M&E is seen by senior management as little more than an exercise in post-rationalization and self-justification by communication practitioners.
This explains a troubling contradiction at the heart of the measurement dilemma – that is, despite demands for results and accountability, employers often will not pay for and sometimes do not seem to want PR and corporate communication to conduct research. Analysis reveals that many employer/client organizations feel that they intuitively or anecdotally know enough about what was done in the past, or they simply feel that ‘what’s done is done’. Many do not want to pay for what they feel they already know and what cannot be changed. However, senior managers are interested in the future and will pay for what they don’t know – factors that point to the need for a new forward-looking approach.
Organization management sees value in terms of contribution to organizational outcomes and, perhaps even more importantly, contribution to future organizational strategy (e.g., identifying opportunities). However, as shown by the European Communication Monitor, linking PR and corporate communication to organizational outcomes remains a major challenge for up to 75 per cent of practitioners and, while communication professionals are regularly called on for advice on organization strategy, studies show that few directly contribute to the formation of strategy.
2. The blurring of measurement and evaluation
The second obstacle preventing demonstration of the value or PR and corporate communication and justifying a new approach is that measurement and evaluation are commonly blurred as a single process and conceived to be broadly the same thing. This is problematic because measurement and evaluation are distinctly different conceptually and methodologically. Measurement is the taking of measures such as counting items, assigning ratings on a scale, or recording comments in interviews, and analyzing these data. Evaluation, as defined in most dictionaries, is “to judge” or “make a judgement” about the value of something.
Measurement involves metrics and analytics, focussed on data and numbers. While it is informed to some extent by metrics, value is a perception. It is a human judgement made from a perspective and in a context. In the case of organizations, value is determined from the perspective of their goals, objectives and needs Likewise, in the case of stakeholders, the value of communication is determined from the perspective of their interests.
By conflating measurement and evaluation, communication practitioners present arbitrary metrics such as the volume and tone of media coverage and the volume of social media ‘likes’, ‘follows’ and retweets, as purported ‘evaluation’. While useful ‘measurements’ of outputs and outtakes, these do not demonstrate value.
Figure 1. Traditional M&E approaches which blur and conflate measurement and evaluation.
3. A narrow data base
A further obstacle arising from the blurring of evaluation with measurement and conducting evaluation concurrently with measurement or as a conjoined linear process is that evaluation is based on a relatively narrow range of data. In most cases, evaluation is based totally on metrics collected by the organization in its measurement which, with budget restrictions that commonly exist, is often limited. Additional sources of relevant data are largely ignored.
4. Over-emphasis on numbers and ‘scientific’ approaches
The inability of many practitioners to demonstrate the value of PR and corporate communication is further compounded by a fourth major obstacle – the over-emphasis on quantitative data and ‘scientific’ research that is characteristic of modernist research generally and PR and communication management in particular. The focus on taking measurements and the search for a ‘magic metric’ such as an ROI of PR has ignored the previous point that value is a perception (not a number) and also ignored the fact that many if not most outtakes and outcomes of communication are humanistic rather than scientific. For example, PR and corporate communication typically seek to create or influence awareness, attitudes, opinion, engagement, trust, loyalty, reputation, and relationships, as well as behaviours. Evaluating these outcomes requires interpretative qualitative research as they are based on human interpretation and feelings which numbers describe in only arbitrary and superficial ways.
A new four-stage approach – the MAIE model
A new four-stage model of measurement and evaluation was presented to and discussed at the 2014 International Summit on Measurement in Amsterdam in June as an approach to overcome these obstacles and demonstrate the value of PR and communication.
Stage 1: Measurement
As in traditional approaches, the MAIE model begins with measurement – the collection of data to provide relevant metrics (numbers) and analytics which are meaningful patterns of data. The one difference from traditional approaches in the MAIE model is that qualitative as well as quantitative data is advocated as essential.
From this point, the MAIE model departs even more fundamentally from traditional M&E approaches by inserting two additional stages into the process.
Stage 2: Analysis
The first of these is in-depth analysis. This differs from data analysis which is conducted as part of measurement. While data analysis focuses specifically on data collected in a measurement activity (e.g., of survey responses), analysis in the MAIE model advocates accessing and examining a range of relevant external as well as internal data that can provide contextualization and sometimes triangulation for comparison and complementation of findings. External data can include:
Such sources provide an expanded data pool which can be used to compare, contrast, question and contextualize findings, rather than relying only on analysis of bespoke metrics. This analysis stage can also incorporate the techniques of market analysis, competitor analysis, business analysis, and academic approaches of critical analysis. This expanded approach to analysis is undertaken for two reasons in the MAIE model, as outlined in stages three and four.
Stage 3: Insights
Before evaluation is undertaken, in-depth analysis should be conducted to identify insights that can inform future organization strategy. Whereas traditional evaluation findings are mostly backwards-looking and descriptive, insights are forward-looking and involve inferences, predictions, suggestions, and recommendations.
For example, insights could include identification of a gap left by competitors, an opportunity to seize thought-leadership on an emerging issue, a likely legislative initiative identified based on patterns of political comment, or an opinion shift among stakeholders that can be productively addressed at an early stage.
Such insights contribute to future business or organization strategy and outcomes, as well as inform PR and communication performance management and planning and, as such, bridge the gap between PR/corporate communication and organizational strategy and outcomes.
Producing insights requires focussed time and effort as well as knowledge of research and analysis. However, the benefits are substantial and there is a range of techniques used by researchers and analysts that can be learned and applied. These include:
Figure 2. A new MAIE model of measurement and evaluation.
Stage 4: Evaluation
Finally, in the MAIE model, evaluation is undertaken as the fourth stage. Evaluation undertaken post the identification, presentation and application of insights is able to capture the value-add provided to the organization in insights and is therefore likely to reflect a much higher level of appreciation and perceived value among internal stakeholders (e.g., management) than retrospective reports. Similarly, insights which recognize external stakeholders’ perspectives and lead to organization actions to improve communication and relationships are likely to lead to increased value in the eyes of external stakeholders.
###
Jim Macnamara PhD, FPRIA, FAMI, CPM, FAMEC is Professor of Public Communication at the University of Technology Sydney, a post he took up in 2007 after a 30-year career working in journalism, PR and media research which culminated in selling the CARMA Asia Pacific franchise which he founded to iSentia (formerly Media Monitors) in 2006. He is the author of 15 books including The 21st Century Media (R)evolution: Emergent Communication Practices( Peter Lang, New York, 2010, 2014) and Public Relations Theories, Practices, Critiques (Pearson Australia, 2012).
Top speakers from NATO, UNICEF, HM UK Government, Lenovo, Booz Allen Hamilton, Cleveland Clinic, ING, and Booking.com are participating in the AMEC International 2014 Summit on Measurement, Upping the Game – from Measurement to Insights in Amsterdam, June 11-12.
Special speaker guests include Ambassador Kolinda Grabar-Kitarovic, Assistant Secretary General for Public Diplomacy, NATO; Professor Jim Macnamara, PhD, Professor of Public Communication, University of Technology, Sydney; and Alex Aiken, Executive Director of Government Communications, HM Government UK.
Other key speakers include Richard Wergan, Executive Vice President, Global Head of Brand, Communications & Digital, Philips, and three global PR agency heads: Dave Senay, Global CEO, FleishmanHillard; Kevin Murray, Chairman, The Good Relations Group; and Matt Neale, President, International, GolinHarris, who will take part in a special session on “Measurement and the PR Industry”.
Salience Insight Chief Executive Giselle Bodie joins AMEC Chairman and Ketchum Partner/Managing Director David Rockland in a light-hearted but serious ‘therapy session’ that attempts to repair the occasionally tense relationship between the measurement and PR worlds.
More Summit information and booking details on http://amecinternationalsummit.org/
###
Angie Jeffrey, Salience Insight's Managing Director, U.S., and a frequent contributor to The Measurement Standard, will be inducted into the PR News Measurement Hall of Fame on April 8th, 2014 at PR News' Measurement Conference in Washington, DC. Congrats to Angie and all the other new Measurement Hall of Famers.
Salience Insight, as one of the sponsors of this event, is pleased to offer a $200 discount to Measurement Standard readers. Just use discount code Salience200. So, enjoy the conference, and ask Angie for an autograph! --WTP
SNCR's 8th Annual Research Symposium included presentations that covered a wide range of emerging communications tools and technologies: From standards to Google Glass to optimizing the classic press release. The longest-standing social media event, SNCR’s annual Symposium brings together an influential group of business leaders, professional communicators, members of the media, futurists, scholars, and technologists from around the globe. The Symposium was moderated by SNCR's founder and president Jen McClure, and hosted by Thomson Reuters in Boston.
The theme that most every one of the seven presentations touched on was that social media is no longer simply another communications medium to be used by PR and marketing, it is a valuable tool that is rapidly becoming integrated throughout all aspects of business and our personal lives. This theme was most deeply explored in Vanessa DiMauro's presentation, but most succinctly stated by Katie Paine: "Social media isn't media—it's your business."
Highlights:
Vanessa DiMauro: Social Business Trends
Ms. DiMauro reviewed three studies to discuss trends in enterprise-level social business. There is a big difference between social business activities, and social media as used by PR/communications. The former includes a broader and organization-wide embrace of SM's advantages. It's clear that, to remain competitive, organizations have to go beyond SM marketing to integrate social experience into core functions and to value SM as related to all business aspects. View the slides from the presentation.
Major points:
Katie Paine: "We Have Standards, Now What?"
Ms. Paine reviewed recent standards progress. View the slides from the presentation.
Best standards usage tips:
Next steps: Clients should pledge to support and adhere to the new standards, and force agencies and vendors to comply. Associations must sign on to the standards and incorporate into award programs.
Best Improve-Your-Measurement Tip: Rank programs on both engagement and resource use, then find the ones that use less resources but get more engagement. Those are your most efficient programs.
Social Media for Social Good: Using Social Media to Raise Awareness About Health Disparities & Health Reform
Moderated by Douglas Spenser, including Aldon Hynes, Elizabeth Krause, Heang Tan, and Alyssa Yacone.
A discussion of how non-profits and governmental organizations are using new media platforms for conversation, ideas, and collective action. In particular, Health Justice CT, a SNCR initiative funded by the CT Health Foundation, has brought together organizations via social media to address racial and ethnic health disparities. See slides here.
Bob Schukai: Google Glass and the Arrival of Contextual Computing
The dynamic and personable Mr. Schukai demonstrated Glass, and gave a preview of some of the apps and experiences that it will enable. Expect public availability at a price in the mid-to-high three figures by the end of next year.
Big point: Wearable computing is about contextual computing. It's always on, and it's always interacting with what’s going on around the wearer.
What Glass means for measurement: Some very precise measurement will be possible. For instance, Google will be able to charge per glance, by serving ads and monitoring wearers' eye movements.
Nora Barnes: Social Media Trends Among the Fortune 500 & Inc. 500
Dr. Barnes presented results of the fourth year of an annual study of SM adoption in the biggest and fastest growing companies. Compared to the Fortune 500, the Inc. 500 is more active in blogging, Pinterest and Foursquare. But the Fortune 500 is more active in Facebook, Twitter, and especially, blogs. View the slides of the presentation.
Highlights for Inc. 500:
Highlights for Fortune 500:
Ingrid Sturgis: The State of Mobile Banking Adoption
Ms. Sturgis examined the adoption of mobile banking among the underserved in the U.S. Thanks to the mobile money system M-PESA, paying for a taxi ride using your mobile phone is easier in Nairobi than it is in New York. I found it fascinating that a valuable business concept originated in the third world and then potential in the first. See the slides of her presentation here.
Sally Falkow: 2013 Online Newsroom and Media Relations Survey
Ms. Falkow discussed the disconnect between journalists who very much want images and video, and companies who don't provide sufficient of these media in their press releases and in their online press rooms. The inclusion of images and especially video greatly increase the views, time on page, and SEO of press releases (and articles), yet companies appear reluctant to supply them. See this video.
A fascinating dynamic behind this situation was discussed. The ranks of journalists have been cut by roughly 1/3 recently. As a result, most journalists now are both overworked and inexperienced, and so they rely almost exclusively on Google searches to do research. And they very much rely on online press rooms for material. So SEO has become more and more important for press materials. Yet few companies do adequate SEO on their press materials. See the slides from the presentation here.
###
Bill Paarlberg, Editor of The Measurement Standard, has been writing about public relations measurement for 20 years. He is editor of the award-winning "Measuring the Networked Nonprofit" by Beth Kanter and Katie Paine, and editor of two other books on measurement by Katie Paine, "Measure What Matters" and "Measuring Public Relationships." Visit Bill Paarlberg's page on LinkedIn.
The Measurement Standard is a publication of Salience Insight. Salience Insight is the media measurement division of News Group International – a global provider of business intelligence and media resource services. Salience is a fresh, new global brand which incorporates the former UK-based Report International and US-based KDPaine & Partners, acquired in 2012.
Here's a video describing research done to test the effectiveness of news releases with text, text and image, or text and video. For this study, Get City Dealz, SEO-PR, and Business Wire won the 2013 SNCR Excellence in New Communications Award in the Visual Media Category of the Corporate Division. Read the whole article here.
Angie Jeffrey's Salient Insights
As measurement becomes more sophisticated and expensive, the gap widens between “Have’s” and “Have-Nots.” Here are some successes from the "Haves," and some low cost options for the "Have-Nots."
The recent PRSA/AMEC Measurement Symposium in Philadelphia, a pre-con session for the PRSA International Conference, built on insights that had been shared at the 2013 AMEC European Summit, and went further in helping attendees learn new ways to link their work to organizational outcomes.
In case you don’t know AMEC, it is the International Association for the Measurement and Evaluation of Communication, a global conclave of leading PR agencies and measurement providers that is taking a leadership role in educating practitioners about research and measurement. Allyson Hugely, Executive Vice President Measurement, Analytics & Insights for Weber Shandwick, chaired the Symposium.
I had the privilege of moderating the second panel of the day, “Creating Business Value through Social Media Measurement and Analysis.” Panelists included David Kellis, Director Public Relations and Social Media for CLOROX; Victor Zaborsky, Marketing Director of MilkPEP; Jaye Hilton, Executive Vice President of Weber Shandwick Consumer; and Noah Krussell, Director Analytic Services for Evolve24. Below are just a few of the great tips they shared.
But first one quick observation: I was struck again by the gap between the measurement “Have’s” and “Have-Nots” in their access to data scientists and/or marketing mix models. By far the coolest options shared by speakers involved analysis programs way beyond the budgetary reach of the average practitioner. In fact, when I asked attendees to raise their hands if they had access to such people/tools, only three of 50 attendees raised their hands.
So, while we can all be inspired by -- and aspire to -- the excellent work being done by the leading organizations, we also need to bridge the gap with options that involve lower costs and less higher education. I will share one such suggestion with you at the end of this article.
Tip #1: David Kellis Introduces Datalogix
David Kellis has worked extensively with a variety of tools and models to link paid, earned, and shared media to outcomes such as sales. He is one of the “Have’s” when it comes to data smarts (his own!) and apparent measurement budget. He introduced us to Datalogix, which is much less expensive than a market mix model, yet can “measure the actual offline sales lift resulting from paid, earned, and owned digital media across a universe of over 100MM consumers at the 1:1 level.” In other words, it measures offline ROI on digital campaigns, which has been elusive outside of market mix models.
Kellis provided a quick case study below (see the slide below), which shows the power in this process. However, a Datalogix study starts at around $50,000 for a campaign and is really limited to consumer packaged goods, automotive, retail, and other industries with quick access to sales data.
Tip #2: Got Marketing-Mix Modeling?
Victor Zaborsky and Jaye Hilton have been key players behind the incredibly successful National Milk Mustache “got milk?” Campaign. Recently they moved from a periodic campaign-oriented model to an ‘always on’ model with some of the most complete “Virtual/Real World” programming I’ve ever seen. For example, they have created a giant “Command Center” that interacts with thousands of social media conversations about milk and breakfast, and sometimes even sends “real world” milk deliveries.
For measurement, the chart below illustrates clear primary KPIs and exactly what metrics will be used for each. All but the last measure are available to us all through low-cost or free tools. But, MilkPEP is one of the lucky “Have’s” with access to a market mix model, so they can see that the campaign contributed a 72% increase in gallons of milk sold in 2012. Exciting, indeed!
Tip #3: Demonstrating that Communications Saves Costs
Noah Krussel focused on “Managing the Non-Market Environment & Social Media,” showing the disastrous financial results that can result from unplanned events. His most-intriguing take-away was the chart below, which illustrates the billions of dollars lost by various companies due partly to serious problems with their products, but also multiplied by activist or regulator activity. Bottom line: Our communications activities have real-world, financial outcomes; whether we have market mix models or not, we can know the power of our work.
Tip #4: Lost Cost Measurement Tools of Choice: Web Analytics, Correlations, and Excel
So, all this high powered -- and high cost -- measurement is great. But what does the average practitioner do to link his/her traditional or social media work to real outcomes without a data scientist or market mix model?
The easiest answers are through web analytics and correlations. We’ve done lots of work correlating media coverage to leads, survey scores, sales and more. Correlations are easily calculated in Excel. How to do them is covered in our mini Powerpoint guide, "Linking to Outcomes through Media Scores & Correlations."
###
Angela Jeffrey is Managing Director U.S. for Salience Insight. A recognized evangelist, thought leader, writer, and speaker for PR measurement and evaluation, Ms. Jeffrey spent two decades in PR, advertising, and marketing before starting her own firm, Houston-based Jeffrey Communications. She also created PRtrak™, one of the first analysis tools to cover print, broadcast and internet coverage. More recently, Ms Jeffrey founded MeasurementMatch.com, a consultancy aimed at matching the right measurement providers with the right clients. She is also a member of the Institute of Public Relations’ (IPR) Commission on PR Measurement & Evaluation and an active speaker at many industry conferences.
The Measurement Standard is a publication of Salience Insight. Salience Insight is the media measurement division of News Group International – a global provider of business intelligence and media resource services. Salience is a fresh, new global brand which incorporates the former UK-based Report International and US-based KDPaine & Partners, acquired in 2012.
The PRSA/AMEC Measurement Symposium "Unlocking Business Performance Through Communications Research and Analytics" is a half-day course on Sunday 10/27 at the One World PRSA 2013 International Conference. Speakers feature measurement stars Mark Weiner, Mark Stouse, Angie Jeffrey, Allyson Hugley, and several others. Here are some details from the conference website, where you can register:
This session will build on insights that were shared at the 2013 AMEC European Summit, and will give you a practical look at the important role of analytics in evaluating and advancing communications performance. You’ll also gain the tools and techniques that will help you better align your organization’s communications with business objectives and results.
Part one of this session will focus on social media measurement, and will include an overview of best practices and case studies from leading brands, research firms and communications agencies, including Weber Shandwick, PRIME Research, IKEA U.S., BMC Software and MasterCard International.
Part two of this session will focus on outcome measurement and linking communications to business results. You will be provided with a mix of best practices and case studies by presenters from leading brands, communications agencies and research companies, including Salience Insight, Clorox, Evolve24, Weber Shandwick, MSLGroup, HCD Research, Verizon Wireless and PriceWaterhouseCoopers.###
A fun and informative day was followed, as always, by lobster dinner and more lively conversation. Look on flickr for almost 300 photos from the event. Thanks to Katie Paine for hosting and Paine Publishing for sponsoring this year.
Bill Paarlberg, Editor of The Measurement Standard, has been writing about public relations measurement for 20 years. He is editor of the award-winning "Measuring the Networked Nonprofit" by Beth Kanter and Katie Paine, and editor of two other books on measurement by Katie Paine, "Measure What Matters" and "Measuring Public Relationships." Visit Bill Paarlberg's page on LinkedIn.
The Measurement Standard is a publication of News Group International.
When Research and Standards Collide
by Katie Delahaye PaineThere were two papers at IPRRC 2013—one by Marianne Eisenmann and Julie O'Neal, and one by Sean Williams—that have significant impact on the Conclave's pursuit of public relations and social media measurement standards.
Eisenmann and O'Neal: Standards for Media Analysis Prove Unreliable—Or Do They?
The paper with the greatest implications for standards-setting efforts was by Marianne Eisenmann and Julie O’Neal: “Testing the reliablility of metrics proposed as standards for traditional media analysis.” The results at first appear to be unpleasant news, but are subject to a couple different interpretations.
The purpose of the research was to test the reliability of coding decisions made by human coders using the standard definitions proposed by the Institute for Public Relations in its June 2012 paper "Proposed Interim Standards for Metrics in Traditional Media Analysis." That paper specified definitions for:
Eisenmann and O'Neal prepared a detailed set of instructions that specified definitions for each element to be coded. The coding books were reviewed by three PR practitioners experienced in media coding and measurement, who made minor modifications.
Then the researchers collected a systematic random sample of clips about Wal-Mart between July 1, 2011 and June 30, 2012, ultimately analyzing 106 items.
Three graduate students with some experience in public relations were selected to do the actual coding and were trained for about two hours. After the training the three coders and two researchers independently coded seven stories and the results were compared to identify discrepancies. The coding book was modified accordingly. A second set of five items were also subjected to a pretest to make sure that the more subjective qualitative items were coded correctly.
After coding the entire sample of items, reliability results indicated the highest level of agreement between coders on elements like media type, prominence, and shared vs. sole mention. There was low to moderate agreement when coding for the presence of reputational messages, and lowest agreement of all was for sentiment.
One way to interpret these results might be to suggest that the standards as currently written yield unreliable results. However, the way I choose to interpret the results—from the perspective of someone who has managed and trained human coders for more than two decades and has written, reviewed, and tested thousands of coding instructions in that time—is to note that developing accurate coding is a complicated process. It requires a code book that has been tested for at least a month, and the best coders require six months of testing and training before they can be relied upon to code accurately on their own. This training was obviously not within the scope of this study.
Thus I don't recommend using the results of this study to draw too many sweeping conclusions about the fate of the standards. That having been said, I would be seriously remiss if we didn’t report these results.
So, it is possible to draw a couple different conclusions from this study:
1. The definitions of sentiment and the use of latent sentiment analysis are fundamentally flawed, or,
2. The definitions are correct, but it takes more than graduate students and the time available to do this study right.
I’m going with #2.
Williams: Social Media Influence
Sean Williams, like many of us, has long been disturbed by the various claims out there regarding influence in social media. His paper, "Is that all there is? A literature review and potential approach to measuring influence in social media," addressed the following aspects of influence:
He conducted a massive literature search on the subject, referencing everyone from the 1955 communications theorists Elihu Katz and Paul Lazarsfeld, to Valdis Krebs, who has lead the charge into defining online influence as it relates to social network analysis.
His research dovetails nicely with the current efforts of the Conclave and WOMMA on the topic. What the standards effort has revealed is just how challenging it can be to achieve consensus on such a multi-faceted topic. One can argue that there are 7 billion influencers on the planet, or that there are just a handful—it depends on the topic or product. Williams’ paper outlines a framework for a far more exhaustive research study that could be the first step towards more definitive consensus around this subject.
Learn more at this summary of his paper on the IPR website, where you will find a link to the actual paper.
###
Katie Delahaye Paine is Chairman, KDPaine & Partners, (a Salience Insight company), and Chief Marketing Officer of News Group International. KDP&P delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, Chairman of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
by Katie Delahaye Paine
After listening to 108 papers in three days, the fact that I can remember anything at all from the most recent International Public Relations Research Conference is pretty astounding. Which is another way of saying that the research that we highlight here in The Measurement Standard really stood out from the rest. Not necessarily because of the quality of the data, but because it was so relevant to the profession. (See also "Katie Paine's 3 Favorite Social Media Research Papers from IPRRC 2013")
Let's start with crisis management. If you’ve followed our prior IPRRC wrap-ups (see, for instance, "Katie Paine's Favorite Papers From IPRRC 2012"), you’ll know that there are always a plethora of papers on crisis management. What emerged this year, however, were a number of new insights that all communicators should keep in mind. Here are five of them:
#1. Customers who post angry or negative comments about companies are doing it mostly to protect themselves and their friends.
According to research presented by Ming-Yi Wu of Gallup, electronic word-of-mouth complaints are motivated more by a desire to protect others from suffering bad experiences than to get discounts or deals. When asked what kind of response they wanted, 72% of complaining customers said that the most effective response would be to change company policy, while only 38% said that a discount on a future purchase would be a good outcome.
#2. Those who apologize best suffer least.
Tim Coombs has proposed that "mortification strategy" -- complete apology and ownership of the problem -- is the best path to restoring and repairing an organization's brand image after a crisis. This year’s crop of research clearly supports that claim (see also #5, below).A study by Erica Johnson at the University of Missouri School of Journalism examined how Victoria's Secret used Facebook to apologize for perceived racist stereotyping in a fashion show. The rapidity and perceived authenticity of its apology, combined with the immediate recall of the ad, lead to greater fan involvement and emotional understanding of the problem. The authors concluded that that this type of apology is an acceptable form of crisis response in social media. Ironically, there were a number of fans who went to Facebook to try to figure out what the company was apologizing for.
#3. People feel betrayed when corporations act out of character.
Shannon Bowen, Associate Professor at the University of South Carolina, and Diana C. Sesson, a PhD student at USC, studied how consumers in the UK responded to the news that Starbucks was a company that hadn’t paid taxes. They found that, after hearing about their failure to be taxed, consumers were more likely to see the coffee giant as inauthentic, dishonest, and lacking in transparency. Other news about the economy and belt tightening contributed to and amplified the negative messages about Starbucks in the UK. However, its global corporate reputation for doing good and benefiting the community helped mitigate the worst of the criticism. Starbucks gained the benefit of the doubt and time to explain itself in part due to the honesty with which it apologized and its reputation for good intent.
A study by Maria De Moya, North Carolina State University, and Rajul Jain, DePaul University, of Netflix' response to its crisis over the doomed “Netquiks” announcement showed that, until the company reversed its decision, social media commentary saw Netquiks not only as a bad value for the money, but discussed every negative aspect of the service as well as the quality of Netflix’ management. Many negative aspects of reputation were raised, including a feeling of betrayal. Prior positive experience with the company made customers expect something different. The move was seen as an indicator that the company was no longer authentic.#4. Product performance plays a key role in crisis mitigation.
A complex experiment by Weiting Tau and Haishi Cui of the University of Florida showed that product association -- i.e., the degree to which consumers have experience with a product and have experienced the company’s ability to deliver as promised -- had a demonstrable impact on perceptions. When participants in the experiment received a message about the company’s abilities first and then saw information about its CSR efforts they were more likely to purchase a product than if they saw the CSR message alone.
#5. It's best to apologize in the same channel in which a problem arose.
Desideria Cempaka Wijaya Murti, of the University of Atma Jaya Yogyakarta, Indonesia, and Colorado State University, authored a fascinating analysis of FedEx’s response to the highly viewed YouTube video of its delivery man tossing a computer monitor onto a customer’s lawn. Her research found that FedEx's strategy of using YouTube to reply to the incident was highly effective. She also found that FedEx’ use of mortification strategy was effective. The CEO said, “I am upset and embarrassed for our customer. Please accept our apology, we will use this incident to be an even better service provider, this goes against all FedEx values.”
She recommends considering the visual aspect of a crisis when you are deciding upon your response strategy. She also found that creating distance with the employee while at the same time answering all questions about ethical standards was an effective strategy.
###
Katie Delahaye Paine is Chairman, KDPaine & Partners, (a Salience Insight company), and Chief Marketing Officer of News Group International. KDP&P delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, Chairman of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
There were so many great papers on social media at this year's IPRRC. Here are my three favs:
Paper #1
DiStaso, McCorkindale, and Agugliaro: America’s most admired companies social media industry divide
Not much to admire in the social media practices of many of Fortune’s Most Admired.
As usual, the “MarTina” team of Marcia DiStaso and Tina McCorkindale produced one of the most interesting papers of the conference. Together with Alexa Agugliaro, they studied how the companies on Fortune's America's Most Admired list are using social media. This is a paper that every corporate communications officer in America should read.
First off, Twitter is by far the most widely used platform, followed by YouTube and then Facebook. That might surprise some people, but the reality is that while 955 of the Most Admired appear to have a Facebook page, 51% of those are just using their Wikipedia page on Facebook.
The authors categorized the Most Admired companies by industry:
Then they judged each one on the following variables;
Using these criteria, they created a list of “Gold Standard Companies," those that met 11 out of the 15 best practices. They then used content analysis to analyze the social media accounts of 417 companies on Facebook, Twitter, and YouTube.
Sadly very few of the Most Admired adhered to the best practices. The industry with the worst scores on every channel was Healthcare / Pharmaceutical, while Consumer Package Goods scored highest overall. Perhaps most surprising was the high performance of Financial Services companies on Facebook and YouTube. There wasn’t a single company in the Healthcare / Pharmaceutical or Manufacturing categories that even came close to Gold Standard practices.
Overall top performers: Charles Schwab, Chevron, Adobe, Intuit, and Nordstrom.
The Gold Standard companies by industry were:
Consumer Packaged Goods:
Entertainment:
Financial Services
Retail
Technology:
Transportation
The authors have some very pointed recommendations for the Most Admired companies. My favorite was this: “Since the healthcare/pharmaceutical industry is highly regulated, engaging in social media can be more challenging than in some of the other industries, this, however should not discourage companies from utilizing social media." The best practices in this study had nothing to do with the regulation concerns, so there is no excuse for not having better account management. .Paper #2
Most PR people don't measure their own social media.
One of the papers I always look forward to at the IPRRC is the annual update from Michelle Hinson and Don Wright of their eight year longitudinal analysis of the use of social and emerging media in PR. Since 2006, an average of 430 PR professionals each year have responded to their questions on the impact of social media on the practice.
The most discouraging response was that 57% of those surveyed are not measuring anything at all in social media (just shoot me!). And, of those who are measuring, only 22% are measuring anything relating to outcomes.
This year’s research revealed an increase in the percentage of time PR practitioners spend on blogs and other social media during their workdays. Twitter was the channel that saw the most increase in usage.
Paper #3
Mulvey and Hill: Motivation to like: Analyzing the motivations of college students liking companies on Facebook
What’s to Like?
Another riveting study was by Tyler Mulvey and Kevin Hill of Rowan University, who set out to determine just why students decide to Like a brand on Facebook. They conducted an online survey that collected 103 responses, 68% were female, and 32% were male. 76% were between 19 and 22, and 24.3% were over 22.
It was the difference in responses between the various demographics that proved most interesting. In general:
###
Katie Delahaye Paine is Chairman, KDPaine & Partners, (a Salience Insight company), and Chief Marketing Officer of News Group International. KDP&P delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, Chairman of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
It’s the time of year again when all of us research types crawl out of our caves and make our way to the International Public Relations Research Conference in Miami. We are eager to find out what the rest of the world has been working on while we’ve been huddling in front of our wood stoves and laptops, slaving in the measurement mines. We are excited to hear 100+ papers in three days, and, also, to lose our pasty white data-geek complexions.
This year already promises some very hot and relevant topics.
One of the most eagerly anticipated papers, at least from those of us who are currently obsessing about standards, is Assessing the reliability metrics proposed as standards for traditional media analysis, by Marianne Eisenmann, Julie O’Neil, and David Geddes. Their idea is to test the current proposed standards to see how easy they are to follow and, more importantly, how accurate the results would be if they were followed.Another much anticipated paper is Don Wright and Michelle Hinson’s annual update of their study Measuring social and emerging media use in public relations practice: insights based upon an eight year longitudinal analysis.
I’ll be there with my paper Is social media: truth or fiction? Why you shouldn’t trust social media data. (Download the 845K PowerPoint presentation here.) It’s about the myths and misperceptions in social media, and what you can learn from more than 300,000 plus posts.
Here is a sampling of the papers I can’t wait to hear:
• Alfredo Arceo and Rosa Torres: Evaluation of the relationship: how to create a monitoring tool
• Fraser Likely: Public relations/communication department reporting: What information do CEOs need?
• Bokyung Kim: How publics respond to a company's voluntary disclosure of crisis on Facebook despite its apparent motivation to get strategic transparency
• Marcia W. DiStaso, Tina McCorkindale, and Alexa Agugliaro: America's most admired companies social media industry divide
• Joe Bob Hester: Sampling Considerations for Social Media Research
• Ming-Yi Wu: Customer Relations in Social Media: Social Media Usage Motives, Expected Responses From Organizations, and Electronic Word of Mouth (eWOM)
• Ansgar Zerfass and Dana Melanie Schramm: Social Media Newsrooms in Public Relations: Developing a Conceptual Framework and Researching Corporate Practices
• Sean Williams: Is That All There is? A Literature Review and Potential Approach to Measuring Influence in Social Media
• Terri Ann Bailey and Joanna Williamson: Best practices in creating content for Twitter: a "how to" template based on a qualitative content analysis of top Twitter brands
• Young Kim: Post-Crisis Communication Strategy: A Comparative Framing Analysis of BP’s Press Release and Social Media Messages after the 2010 Deepwater Horizon Accident
• Tom Watson: How PR faced the challenge of the "information superhighway"
• Lynne M. Sallot: Measuring reputation excellence in academic public relations programs
• Tyler Mulvey and Kevin Hill: Motivation to Like: Analyzing The Motivations of College Students Liking Companies On Facebook
• Arunima Krishna, Soojin Kim, and Seungyoon Lee: Understanding employee WOM using social network approach for strategic change and risk management
• Bilgen Basal: Exploring the indications of rising social media usage in Turkish PR: a survey of Turkish internet users
• W. Timothy Coombs, Sherry J. Holladay, Finn Frandsen, and Winni Johansen: Livestrong to livewrong: Analyzing and conceptualizing sub-arenas of crisis communication in the Lance Armstrong doping crisis
• Richard Stanton: Communicating deep rupturing policy. Strategic uses of new media in nation-building public relations: a comparative measurement of Twitter's impact on carbon campaigns in the United States, the United Kingdom, and the European Union
If you can’t get there, watch my blog, this space, and my Twitter feed (https://twitter.com/kdpaine) for our live reporting. Where else can you get a synopsis of 100+ papers delivered in three days?
###Katie Delahaye Paine is Chairman, KDPaine & Partners, (a Salience Insight company), and Chief Marketing Officer of News Group International. KDP&P delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, Chairman of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
The University of Alabama's Plank Center's Cross-Cultural Study of Leadership In PR and Communication Management is the largest study of its kind ever conducted. Nearly 4,500 PR pros participated in the online survey. A second phase involves interviews with 150 worldwide leaders, which will be completed and analyzed in 2013. Visit this page for details, or just download the slides.
Durham, NH – October, 2012 – A cross-disciplinary coalition of marketing and communications associations has announced several initial standards for social media measurement, based on progress made earlier this month at the second annual #SMMStandards Conclave. The Conclave has brought together leaders representing 11 industry associations and eight major corporations over the past year to help define social media measurement standards that will work across companies, brands, agencies and disciplines.
The initial standards address three specific challenges:
All of these initial standards will be published this month for public comment via www.smmstandards.org and be submitted to a customer panel assembled by the Coalition for Public Relations Research Standards. Katie Delahaye Paine, Chief Marketing Officer for News Group and founder of KDPaine & Partners, who hosted the meeting at her home in Durham said: “Major customers requiring the use of standards is an essential step toward ensuring their impact. Meanwhile, research providers who wish to stay ahead of the marketplace on social media measurement are encouraged to start adopting the standards now.”
The detailed initial standards for Engagement and Conversation measurement will be published this month vis www.smmstandards.org and be available for comment until December 1. Based on public feedback and review by the Coalition’s customer panel, the standards will be modified and finalized by the end of 2012.
###
(Thanks for the image to Scanning Around With Gene at CreativePro.com.)
Originally a simple calculation of quantifiable financial results, today the term Return On Investment (ROI) is commonly used to refer to situations where results cannot be quantified or simply expressed.
by Bill Paarlberg, Editor
A couple weeks ago I wrote about how the term ROI (Return On Investment) had become so watered down from its original meaning that most public relations measurement professionals don't want to use it. People who do use the term ROI mostly don't know what the term really means. Those who actually do know what it means mostly don't use it, because ROI is very difficult to calculate in the context of public relations and social media.
Well here's another nail in the coffin of the original meaning of the term ROI, in the form of Business Insider's Social Media ROI conference coming up in September. The term ROI appears only once in the description of all the programs on the agenda, and it's used both there and in the title of the event to mean "whatever kind of business-related results." Rather than its original meaning.
Don't get me wrong here. I am sure Business Insider's Social Media ROI conference will be an interesting, informative, and valuable event. My point is that this conference demonstrates that in popular usage the meaning of ROI has moved on from it's original financial meaning.
I'll bet you an iced mocha latte that almost nobody at that conference uses the term ROI in its original sense. But that doesn't mean they aren't using the term in a useful fashion. Saying "ROI" is a whole lot easier than saying "results related to business goals." Or (cynical me) "results I wish and hope are somehow related to business goals."
Also, of course, the term "ROI" has a certain business-savvy sense about it that comes in really handy when you're pretty sure you're not going to get any easily quantifiable results.
And that is the key to what is going on here. ROI was originally a calculation that could simply express quantifiable financial results. But now the term is used, at least in common reference to social media, to refer to situations where results cannot be quantified or simply expressed financially. Fascinating.
It's as if we wish we could have real ROI for social media, but we know we don't. So we try to hide that we don't by calling our results "ROI" anyway. Has ROI become anti-ROI?
###
Bill Paarlberg is editor of The Measurement Standard blog and newsletter, and of Katie Paine's book “Measure What Matters.” He is also editor of the book “Measuring the Networked Nonprofit,” by Beth Kanter and Katie Paine, which will be published this year by Wiley.
Follow Bill Paarlberg on Twitter.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, CEO of KDPaine & Partners, will be glad to talk with you about measurement for your organization.
The term Return on Investment (ROI) has become so diluted in meaning as to become useless for public relations and social media measurement.
by Bill Paarlberg, editor
There is a new paper on the IPR site by Professor Tom Watson that demonstrates just how diluted-to-death the term Return on Investment (ROI) has become in PR measurement. Dr. Watson surveyed 2000 European PR pros and discovered the consensus was that ROI was too loosely defined a term to be useful in public relations measurement or evaluation.
He also reports on this year's International Public Relations Research Conference, where the brainiest measurement minds in the business
(a) could not decide if they knew what ROI for PR was, and
(b) didn't think they'd use the term even if they did know what it was.
The problem is that the term "Return on Investment" is mostly used in public relations measurement and social media measurement by people who don't know what the term really means. Those who do know what it means mostly don't use it, because ROI is very difficult to calculate in the context of public relations.
What ROI Means
Return on Investment equals the dollar value of the return gained by an investment, minus the cost of the investment, divided by the cost of the investment. It's usually expressed as a percent (which means you multiply by 100 and put a % sign after it). So if you invested $1 to make $1.25, your ROI would be (1.25 - 1)/1 = 25%.
This is the definition of ROI that's been used in the business world for almost a hundred years, and if you took Business 101 in school you learned it.
The C-suite and every Board of Directors -- heck, every business person -- just loves to know the ROI of every component of their business. Knowing ROI makes a lot of decisions easier, because you can use it to compare apples to oranges. Once you know the ROI it's not apples or oranges anymore, it's all expressed as the growth of money.
Why People Who Don't Know What ROI Means Like to Use the Term Anyway
In fact, ROI is such a valuable thing to know, that if you use the term in conversation, people tend assume you know what you are talking about. Well, they used to, anyway. Over recent years, there have been an awful lot of people in the business world who have used the term ROI to make themselves seem smart, even though they couldn't accurately determine their investment, or the return on it. And they did seem smart and important when they talked about ROI this and ROI that, so that other people began using the term, too. Ah, the slippery slope.
And so it is that ROI has suffered a process of dilution, at first just self-aggrandizing douchebaggery, then more and more people wanting to sound like they had business chops. Today, most of the people who use the term "ROI" actually mean "results." (And I'd guess a lot of them don't even know they are using the term incorrectly.)
Of course the ironic thing is that the C-suite and Board know and use the original meaning of the term, so people who use it incorrectly to report to them are just shooting themselves in the foot. Moral of the story: Don't use the term Return on Investment unless you can calculate it. The word you save may come in handy some day.
###
--Bill Paarlberg is editor of The Measurement Standard blog and newsletter, and of Katie Paine's book “Measure What Matters.” He is also editor of the book “Measuring the Networked Nonprofit,” by Beth Kanter and Katie Paine, which will be published this year by Wiley.
Follow Bill Paarlberg on Twitter.
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, CEO of KDPaine & Partners, will be glad to talk with you about measurement for your organization.