The Measurement Standard: Blog Edition

Smurftastic Design With Personalities in Mind: Usabillia

If your target audience includes Smurfs, then Loucas Papantoniou’s post over at usabillia will help you design your website for maximum appeal and response. “These Smurfalicious personas will engage your users.” And if you tend to deal with more human beings than Smurfs, then the same concepts (based on the DISC quadrant behavioral model for humans) will probably work, too. ; )

And what if you are more interested in public relations measurement or social media measurement than in web design? Perhaps different personality types react in different ways to your PR programs. How could you take that into account in your measurement?

###

--Bill Paarlberg is editor of Katie Paine's new book Measure What Matters and The Measurement Standard blog and newsletter. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

Public Relations Measurement’s Empty Head

Measurement ignores the most complex part of PR.

(This post is condensed from an article that previously appeared in The Measurement Standard newsletter.)

by William T. Paarlberg, 
Editor, The Measurement Standard

We don't know much about how public relations happens to people. About how all those things we measure--like impressions and key messages and tone--actually do something inside a person's head to affect their thinking.

Jim Macnamara makes a similar point when he claims that PR people don't do measurement because we have not developed a useful theory of how PR affects people. (See his "Fork In The Road" paper.)

Contemporary PR measurement treats the mind of the media consumer like it's an empty box to be filled: The media dumps in impressions and we measure the outtakes and outcomes. But human beings aren't just empty heads, and they don't consume media in a simple and rational fashion. They purposefully ignore much of what the media throws at them. And much of what they do pay attention to, they don't believe anyway.

Measuring impressions is a whole lot easier than measuring what happens to those impressions once they're inside a person's head. What happens there is probably the most complex part of how PR happens, yet state-of-the-art PR measurement doesn't take it into account.

Measurement needs to understand a bit of how the consumer’s mind works; it needs a Theory of How PR Happens. Measurement knows what it can measure: outputs, outtakes and outcomes. Perhaps if we had more of a theory, we'd know a little more about what we ought to measure. Let's fill that empty head with something that will help us do better measurement. 

Bill Paarlberg is the editor ofThe Measurement Standard. He has been publishing a newsletter on measurement with Katie Delahaye Paine since 1991.

Why Survey People When You Can Measure Their Behavior?

I notice that in public relations it is very common to survey people about what they did, or are doing. If we really want to know what they did or are doing, isn’t it better to measure their actual behavior?

Take, for instance, yesterday’s report for SAI Chart of the Day: Netflix is Beating YouTube to Win the War for Consumer Mindshare. According to a Goldman Sachs survey of 2000 persons, more people prefer Netflix than YouTube for watching online video.

Which is interesting. But what would be really interesting to know is why Goldman Sachs did a survey instead of just getting the traffic data for Netflix and YouTube. By doing a survey they found out what people say they watched. But by getting the traffic data they could have found out what people actually watched. ( We’ve got to assume that if there was an easy way to get some traffic numbers, they would have done that instead of the survey. So maybe there is a good reason why they couldn't compare actual hours watched, or number of videos downloaded, or number of unique visitors, or some such.)

The point here is that what people say they do and what they actually do are often two different things. There is plenty of research that demonstrates that people are often not very good at measuring or judging their own behavior. Take illusory superiority, for instance, or confirmation bias. Or just watch some American Idol tryouts. It’s clear that there can be a huge gap between how people are and how people think they are.

As data gets easier to come by there is less reason to survey people because we can measure their actual behavior. And that’s especially true for online activity.

--Bill Paarlberg, editor of Measure What Matters and The Measurement Standard. The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

Jim & Lauri Grunig’s Lecture at PRSA International 10/17/10

The Institute for Public Relations, PRSA, and the University of Maryland hosted the PRSA International Conference on 10/17/10 in Washington DC. At this conference the Third Annual Grunig Lecture was given by, yes, Drs. Jim and Lauri Grunig and entitled: “Public Relations Excellence 2010.”

As many of you know, Jim and Lauri Grunig are developers of the Excellence Study and its principles for excellent PR, which they explain and elaborate on in this lecture. They are strong proponents of relationships as the foundation of public relations, and have developed techniques to measure relationships as a way to measure the effectiveness of public relationships. (Their work forms the foundation of much of Katie Paine's book Measuring Public Relationships, and also her new book due out next spring from Wiley Press Measure What Matters.)

Click here to read an introduction to the lecture by Bob Grupp (President and CEO of the Institute for Public Relations) on the IPR website.

Click here to download a transcript of the Third Annual Grunig Lecture by Jim and Lauri Grunig.

The Virtues of a Wandering Mind

Hey You! Quit Spacing Out:
What happens in people's minds when they experience public relations?

"Discovering the Virtues of a Wandering Mind" by John Tierney in today's NYTimes caught my attention, probably because I'd love to have some justification for my own perambulating mental life. For those of you who despair at your ability to keep your mind on track, take heart. There appear to be very good and beneficial reasons why our minds often wander from the task at hand.

It is intriguing to think that the psychological research on how people do or don't pay attention could have relevance to public relations measurement. I've always been convinced that the effectiveness of public relations has as much to do with what goes on in a person's mind as it does with the physical nature of the message. (See, for instance, "Measurement's Empty Head: Public relations measurement ignores the most complex part of PR.") 

According to the "Wandering Mind" article, people's minds wander as much as 30% of the time, and their minds are often wandering even when they are not aware of it:

"When experimenters sporadically interrupted people reading to ask if their minds were on the text at that moment, about 10 percent of the time people replied that their thoughts were elsewhere — but they hadn’t been aware of the wandering until being asked about it."

What happens in and to people's minds when they experience public relations? When they read our press releases or watch our videos or listen to our podcasts? Are they paying attention to what we want them to? Are they thinking about something else? What are you thinking about right now? -- Bill Paarlberg, Editor

Music for Public Relations Measurement: The types of bias in your research.

Movies, Media, and How Your Brain Works

Those of you into film and / or how the human brain works when it pays attention to media will be interested in an article by Natalie Angier in today's NYTimes: "Bringing New Understanding to the Director's Cut." It's about research that shows that the editing of movies seems to be coming to reflect the natural rhythms of the human brain.

It's long been my personal hunch that people's consumption of media (and thus the PR outputs we evaluate in the field of public relations measurement) has a lot to do with how their brains work. In fact, it probably has as much to do with how their brains work as it does with the number of impressions or key messages or whether or not the boss was quoted. And so the field of PR measurement ought to be studying how people consume media. (See my article "PR Measurement's Empty Head: Measurement ignores the most complex part of PR.") Today's NYTimes article is another hint that characteristics of the human brain are far more involved in the consumption of media that we currently take into account. --WTP

Since Color Affects Human Judgement and Perception, Shouldn't We Control for it in Public Relations Measurement?

I've been saying for a long time now that public relations measurement has to somehow begin to account for what goes on inside the heads of its audience. It can't just assume that human readers are passive recipients of impressions. On the contrary, they think and feel and consider what they are reading or hearing.

Well, if you still doubt that the reader brings significant baggage to your message, read the article in today's NYTimes entitled "Reinvent Wheel? Blue Room. Defusing a Bomb? Red Room." Among other interesting results:

"The study also tested responses to advertising, finding that advertisements listing product details or emphasizing “avoidance” actions like cavity prevention held greater appeal on red backgrounds, while ones using creative designs or emphasizing positive actions like “tooth whitening” held more appeal on blue."

Think about that next time you do some media analysis. --WTP

If Liberal and Conservative Brains Work Differently, What Does That Mean for PR Measurement?

 

It has always amazed me that there are two large groups of US voters, who, after reading more or less the same information about candidates or issues, can come away with two quite different interpretations and opinions. Recent research has shed some light on this Liberal/Conservative split: it is starting to become clear that this division is not just a matter of background or upbringing, it is, at least in part, a result of fundamental differences between how people understand the world, and how their brains work.

For example, in today's NYTimes, the article "Basics | Obama and McCain Walk Into a Bar ..." relates the results of research that implies that Conservatives have more of a sense of humor than Liberals, and that they tend to be happier. 

There is much more work to be done on this topic, of course, but it is not too early to speculate that Conservatives and Liberals have brains that function differently at some basic level, a level that affects their understand of all sorts of information and media. 

What might this mean for public relations measurement? Well, consider the possibility that Liberals and Conservatives might understand a press release, or a news article, or a blog post, in two very different ways. If their response to corporate news is as varied as their response to political news, then measurement will have to allow for this, or be very inaccurate. As I have said before, measurement needs to take into account what goes on inside people's heads, not just the messages they are presented with. --Bill Paarlberg, TMS Editor

Why Public Relations Measurement Is Like Phrenology, or... Just because it is easy to measure, doesn't mean it's the best thing to measure.

PR measurement and marketing measurement often find themselves in a situation analogous to the 19th-century practice of phrenology. Way back then, some people used to think that the shape of your skull indicated your personality or intelligence. It was a very easy thing to measure, so they did. And they hung on to it as a serious area of study for much longer than they should have, because it was far more difficult to measure personality or intelligence by other, more effective, means.

Click-throughs aren't quite as useless as bumps on the head, but still... In the news today is a study by Starcom USA, behavioral targeting firm Tacoda, and comScore that suggests that clickthroughs might not be a very effective measure of the usefulness of online ads. See here for the news and here for Sam Harrelson's ReveNews discussion.

The obvious point here is that clickthroughs, while easy to measure, probably are not very effective at measuring what we would like to measure. The more interesting point is: Why aren't we measuring what we really want to know?

I have noticed that PR and marketing metrics are often determined by what technology makes easily possible, rather than by what we would really like to know. Thus people tend to measure what technology drops in their laps (the easy things like click-throughs or AVEs) rather than something more relevant but difficult to measure (like, "Did the ad with the red type result in more sales among wealthy women?"). Then at some point they realize that their easy-but-less-relevant metric is not very effective, and then they spend a lot of time asking why the metric doesn't work.

What they should be asking themselves is: "What made us think that that metric would work in the first place?"

Instead of focussing on what technology is available (in this case click-throughs), wouldn't it be more productive to think about human thought and behavior first? (I've written about this before: "Measurement's Empty Head: Measurement ignores the most complex part of PR.") First think about what it is that might make PR work -- why it affects people's thoughts and behaviors -- and then decide what might make a good metric. First determine what we really want to measure (regardless of the technology or data available), then figure out a way to measure it. -- Bill Paarlberg, editor, The Measurement Standard

When Is An Impression Not An Impression: How Does Public Relations Measurement Take News Timing Into Account?

The other day I was re-reading Ed Moed's blog post about how much media coverage depends on what other events happen to be happening at the same time. His point, in part, was that events get pushed out of the media by other events, and what we think of as news actually depends greatly on what other news is also being covered.

And this got me to thinking about how public relations measurement typically measures media outputs with media content analysis, but -- to my knowledge -- does not typically take the news environment into account.

My point is that Article X might result in ten zillion impressions, but if it was published on a day with really big other news (an extreme example would be 9/11/01) then those impressions just aren't the same as they'd be on a slow news day.

We all recognize that the news hole shrinks when something else big is hogging all the coverage. But what we're talking about here is not whether or not there's room for an article to get published, but whether or not there is room in the reader's head for another article. Or conversely, if there is no other news in a reader's head, does an article somehow gain extra importance?

I asked Katie Paine about this and she said she recognized the phenomenon from experience. As to how to deal with it, measurement-wise, she emphasized the importance of human experience behind data analysis: "That's actually why [KDPaine & Partners'] human intervention and hybrid approach is so important. The automated systems just give you numbers, we give you an explanation."

Does anyone out there adjust or modify their media analysis results to take into account the news environment? (I mean formally, with some kind of multiplier or variable that corrects raw impressions data for the impact of other news activity on the reader.)

Some might argue that correcting for the news environment doesn't really belong in media analysis, that it is part of some later stage of public relations' impact on the public's mind. If we put that in terms of outputs, outtakes, and outcomes, then what we are talking about here happens somewhere between outputs and outtakes: Between the outputs and outtakes lies the news environment, which may enhance or inhibit the results of impressions.

To my knowledge, measurement does not formally take this "news environment" phenomenon into account. How does the news environment affect PR's effectiveness? How does PR affect a person differently according to what other news they are experiencing?

In "Measurement's Empty Head: Measurement ignores the most complex part of PR" I wrote something that's pertinent here:

Contemporary PR measurement treats the mind of the media consumer like it's some sort of an empty box to be filled: The media dumps in impressions and we measure the outtakes and outcomes. But human beings aren't just empty heads, and they don't consume media in a simple and rational fashion... Measuring impressions is a whole lot easier than measuring what happens to those impressions once they're inside a person's head... What happens there is probably the most complex part of how PR happens, yet state-of-the-art PR measurement doesn't take it into account.

--Bill Paarlberg