The Measurement Standard: Blog Edition

Should We Trust Web-Based Studies? (This research says Yes.)

This comparative study of Web-based survey samples and paper-and-pencil surveys (by Samuel D. Gosling, Simine Vazire, Sanjay Srivastava and Oliver P. John and which appeared in American Psychologist) is now four years old, but its conclusions are probably still valid: Yes, Web-based self-report and self-selected samples can be trusted for surveys. From the study's summary:

"Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N
361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and age. Moreover, Internet findings generalize across presentation formats, are not adversely affected by nonserious or repeat responders, and are consistent with findings from traditional methods."

Get the whole study here.

NYTimes Article on Web Stats: Is Web Measurement Technology Stymied by Human Beings Just Being Human?

Don't miss this NYTimes article in today's paper that nicely sums up the on-going Web visitor counting controversy (at least from the public's view). Makes me wonder about the reputation of public relations and of public relations measurement: Here we are congratulating ourselves on being able to measure relationships, and being able to measure trust and transparency and various other esoteric things, but we can't nail down something simple like measurement for Web visits.

What's the problem here? It's not lack of data, there's piles of that. And you'd think that there would be plenty of motivation, what with all the money hanging in the balance. Is it as basic as a battle for business dominance; human beings just being human? "My number's better than your's!"

Maybe the real measurement problem lies someplace other than in the Web traffic reports--like between the ears of the people who use them. Geeze, this relationship measurement stuff might come in handy. (And, on a completely different note, how about those Red Sox, eh?) --Bill Paarlberg