My Photo

Measuring the Networked Nonprofit

How to introduce me

  • For those who bear the burden of introducing me at a conference...
    Katie Delahaye Paine (twitter: KDPaine) is the CEO and founder of KDPaine & Partners LLC and author of, Measuring Public Relationships, the data-driven communicators guide to measuring success. She also writes the first blog and the first newsletters dedicated entirely to measurement and accountability. In the last two decades, she and her firm have listened to millions of conversations, analyzed thousands of articles, and asked hundreds of question in order to help her clients better understand their relationships with their constituencies. People talk, we listen..

Become a Fan

« Blogging the Public Relations Measurement Summit | Main | Key messaging in PR tied to prescription sales »

September 28, 2006


Feed You can follow this conversation by subscribing to the comment feed for this post.

Nicolas Boillot

This is a really interesting topic, and great research. However, I would urge people to base conclusions with a strong understanding of the market the study reached -- 351 people in malls who read newspapers at least once a week. Given the malls chosen, there's a great potential that The New York Times may not have been a credible source for them. Would the same thing be true in B2B technology markets -- for instance, the biomedical market -- weighing editorial in Journal of the American Medical Association against advertising in that same publication?

I think it's critical to add this research to the body of evidence on the PR/Advertising dilemma. However, the discussion cannot take place without a parallel discussion about specific target audiences and markets -- some of which will clearly respond to advertising in the same manner as they do to PR, and others who won't. I firmly believe that some audiences will actually respond better to advertising than PR, particularly as mistrust of news organizations continues to grow in certain markets and demographics.

Thank you for furthering this critical discussion.

Robert J Holland, ABC

Thank you for clearing up the issue. I believe in this day and age, when people are so skeptical of "research," especially as it relates to PR, we owe more of an explanation of methodology than "the study was conducted in shopping malls..."

Katie Paine

Here's the response to Mr. Hollands comments from the authors of this study:The research method for this project interviewed 351 respondents in five shopping malls that were geographically dispersed throughout the U.S. Malls were used for this method because it was necessary to show respondents test materials in a printed format.

Respondents were recruited for participation based on several criteria. These criteria included getting a mix of both genders, different ages and different ethnicities as well has being a regular reader of newspapers. In this instance, regular reader was defined as weekly or more often. Qualified respondents were, in turn, randomly assigned to one of three cells -- advertising exposure, editorial exposure or control group. Each respondent (except for the control group) were monadically exposed to the appropriate test materials and then completed a self-administered questionnaire.

A "random sample," given today's problems with contacting respondents is almost impossible and that we very carefully qualified our responses to this sample, which is by design as close to the national average as possible. In order assure that the data was as accurate as possible each test and control cell in each market was statistically weighted into correct proportions that reflected the actually demographic profile of weekly newspaper readers. These weights were based on a national survey of newspaper readers conducted in 2005.

David Michaelson & Don Stacks

Katie Paine

Robert, I'm not sure where you got the notion that this was a mall intercept study, but it was in fact designed by Don Stacks, author of The Primer of PR Research, and David Michaelson,another highly respected PR researcher. They sampled 351 individuals. The complete methodology can be seen here:

Robert J Holland, ABC

I can't believe that, so far, no one has questioned the methodology in this "research." Asking shoppers at five malls? You can't get much more of a self-selected sample than that.

Without having more details on the methodology, I would seriously question the validity of this research -- and certainly wouldn't tout it as "rigorous."

Of course, I can be convinced otherwise...

Ken Kerrigan

While the notion of there being no "multiplier" in the value of PR vs advertising may be of interest in a controlled academic setting it is so far from the real world that it makes one wonder how many thought leaders in our profession actually live in it! Trust me if the NY Times said Zip Chips was among the worst chips they ever tasted sales would plummet - and you can measure that force multiplier like a tropical storm. Conversely if a product really is that good and the media and the advertising are in agreement the multiplier logically would be equal - of course that doesn't take into account the power of word of mouth marketing in today's "brand" new world, which would likely have consumers already buzzing about the chips BEFORE the media or the ads ran. I'd give the study an A for effort but a C+ for reliable findings.

Angie Jeffrey

By way of correction, this study was originally conceptualized and funded by PRtrak/SDI, by Gary Getto and myself. We went to David Michaelson for study design and execution, and he brought in Don Stacks -- which was a great honor and boon to the work.

The initial 'Test' study used a 'new' water product (Ponsef) and was tested on college students at the University of Miami. Initial results did not show much difference between the control cells and others, so we chose not to publicize the initial work very far until more was done. We do congratulate Don and David for taking this the next step, with additional funding from another supplier, and seeing much more concrete results. I tend to agree with David Michaelson's conclusions that proving that editorial is AT LEAST as credible as advertising is not all bad. Since companies are spending enormous budgets on advertising, perhaps more will come our way now that marketing won't be as suspect of our multiplier claims.

Glenn Fannick

Good point. Though I suspect if a company's advertising department saw this they'd be more like to say "ah-ha! there is no 3x multiplier." But the biggest problem here is that this study used an unrealistic best-case and *only* got 1-to-1

Katie Paine

because if we can say definitively that PR is equally effective as Advertising AND PROVE IT, (CW can say anything they want but they can't prove it) and if PR is significantly less costly, shouldn't every good steward of corporate resources shift resources from Advertising to PR?

Glenn Fannick

What I don't get is why this is seen as good news. If CW said that Editorial (sourced by PR) was 3x better than advertising and this study says its 1x. That seems, well, less good. This study also used the best-case scenario of a perfect placement. With the reality of the PR world being far from that.
(BTW, I'm sitting on the other side of the room.)

The comments to this entry are closed.

Measure What Matters

  • “A tremendously good book… it’s a treasure... An absolute doozy of a read.”
    -- reviewer Bob LaDrew, FIR

    Katie Delahaye Paine's great little book Measure What Matters shows organizations of all sizes how to evaluate and improve their public relations and social media efforts. Order Measure What Matters now.

Twitter Updates

    follow me on Twitter

    Tip Jar

    Change is good

    Tip Jar