This is the latest Checklist in our series of tools for public relations measurement and social media measurement. You may also be interested in Katie Delahaye Paine's Social Media Measurement Checklist, and Katie Delahaye Paine’s Product Launch Measurement Checklist.
1. ___ Get consensus on the big picture issues:
a. ___ Ask yourself if you really want the truth, or are you just trying to justify your existence? Are you, and your boss, really interested in reality, or is this just an exercise in budget justification?
b. ___ List the audiences that will see and use the data.
c. ___ List the objectives for the research.
d. ___ Make sure those objectives are in line with corporate and divisional objectives.
2. ___ Inventory existing research:
a. ___ Find out who is already doing what for research in your organization. If it is survey research is it reusable? Is there leverage in keeping questions consistent?
b. ___ Find out if your market research department has a reliable track record with a particular vendor(s). Do they have standard accuracy standards that you can adopt?
3. ___ Do your background homework:
a. ___ Review Dr. Walter K. Lindenmann’s “Guidelines and Standards for Measuring the Effectiveness of PR Programs and Activities” available at the IPR website.
b. ___ Review The CASRO Code of Standards and Ethics for Survey Research.
c. ___ Review The Measurement Guidelines from IAB.
4. ___ Determine the universe upon which you are doing research:
a. ___ Will you investigate a defined media set, or “everything?” (You won’t ever get everything, so realistically, you’ll get about 85%.)
b. ___ Determine if you have a defined universe that matches your target audiences. Will it require sampling?
c. ___ Test to make sure you are getting a representative sample.
d. ___ List the variables that will be included. Get agreement from your boss and your boss’s boss on those variables.
5. ___ Determine who will do the work:
a. ___ If in house, then:
i. ___ Write up your methodology.
ii. ___ Test your methodology.
iii. ___ Refine your methodology until you achieve a minimum of 88% intercoder reliability score. Read about intercoder reliability scores at All Academic Research.
iv. ___ Decide if sampling error limits will be shown (if they can be computed).
v. ___ Determine how projectable the research findings will be to the total universe or population under study.
vi. ___ Analyze your results, using correlations wherever possible.
b. ___ If you’re outsourcing research:
i. ___ Determine who will actually be supervising and/or carrying out the project.
ii. ___ Investigate their backgrounds and experience levels.
iii. ___ Determine who will actually be doing the field work. If the assignment includes media content analysis, who actually will be reading the clips or viewing and/or listening to the broadcast video/audio tapes? If the assignments involve focus groups, who will be moderating the sessions? If the study involves conducting interviews, who will be doing those and how will they be trained, briefed, and monitored?
iv. ___ Determine and confirm that quality control mechanisms have been built into the study to assure that all "readers," "moderators," and "interviewers" adhere to the research design and study parameters.
v. ___ Review the written set of instructions and guidelines for the "readers," the "moderators," and the "interviewers"?
vi. ___ If the data are weighted, insist upon examining the basis for those weights (no black boxes allowed).
vii. ___ Determine if sampling error limits will be shown (if they can be computed).
viii. ___ Determine how projectable the research findings will be to the total universe or population under study.
6. ___ Review the results:
a. ___ Do a “Does this make sense test?” For instance, if you are fourth in the marketplace and the results place you at number one, ask why. If a competitor has a major product launch but its share of conversation declines, what’s up with that?
b. ___ Ask “so what” three times on every chart. Data is only meaningful if it tells you what to do next. Figure out the “So whats?” and the “What are the next steps?”
c. ___ Dig into the negatives first -- what doesn’t make you look good is much more educational than good news that you expect.
“Data will become the new soil in which our ideas will grow, and data whisperers will become the new messiahs.” 
Comments