The Measurement Standard: Blog Edition

Can Facebook's Reputation Be Saved? Or: Mark Zuckerberg Should Have Taken My Advice When I Met With Him 5 Years Ago

by Katie Delahaye Paine

May 2012 AP-CNBC Poll

I RARELY WRITE ABOUT CLIENTS or former clients in this column, but this time I really couldn’t resist. I had the privilege of supplying Facebook with some of its first public relations metrics back in 2007 and 2008. As we were presenting results in Palo Alto one day, Mark Zuckerberg joined the meeting. I took the opportunity to ask him how he defined success for his PR team. He paused and then said: “People will trust us more.”

I’ve used this story frequently since then to illustrate the difference between how CEOs define PR success and how PR defines success. At the time, we were measuring the usual volume and tonality and correlating it to new Facebook signups. And, while that was a pretty good outcome measure in our books, it clearly did not reflect the definition of success that Mr. Zuckerberg had in mind.

His interest in trust prompted me to give him our paper on how to measure trust. I also gave him a copy of my book, opened to the How to Measure Trust chapter. (Click here now to buy Measure What Matters. Or click here to download just the chapter on measuring trust at no charge.) He could have -- and should have -- started right away to measure the trust that people had in Facebook. Might have saved some recent grief.

But I tend to doubt that Mr. Zuckerberg heeded my advice, since in the intervening years, Facebook has made little effort to bolster people's trust. Abrupt policy changes, unclear privacy rules, and an appearance of secrecy have resulted in a growing chorus of concern among journalists, legislators and customers. And, since one of the measures of trust is reduced legal fees, I can guess just by reading the papers that their trust index isn’t going up. (For more on Facebook's trust issues, see this article in The Atlantic, where the chart above appears.)

Now that the social networking giant has gone public it has a whole new audience with which it needs to build trust: the millions of individual and institutional investors that purchased its stock last week. And it doesn’t sound like they’re feeling it right now: http://money.cnn.com/2012/05/23/technology/facebook-lawsuit/

Many people liken trust and good public relationships to a rainy day bank account. You build up trust with your stakeholders over the years by being honest and reputable and forthright. Then when you screw up -- and we all do, sooner or later -- you've got enough trust in your account so that they will forgive you.

I find it supremely ironic (but not that surprising) that four or five years ago Mr. Zuckerberg had the insight and wisdom to identify a vital thing his company needed to do to succeed: build up its trust account. Yet since then he appears to have stopped paying attention to that trust account and has paid more attention to his bank account instead, confusing account growth with value growth.

The constant drain on Facebook's trust account coupled with the giant mess that was the IPO has left nothing for a rainy day. And we’re guessing there will be a lot of rainy days in the next few months.

###

(Thanks for the May 2012 poll results image above to AP-CNBC.)

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements. Katie and Beth Kanter are authors of the book “Measuring the Networked Nonprofit,” to be published this year by Wiley.

The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine, CEO of KDPaine & Partners, will be glad to talk with you about measurement for your organization.

 KDPaine & Partners' Facebook Page.

What We Learned by Trying to Automate Relationship Measurement

Summary: The author attempted to develop an automated system for rating media mentions using Grunig's relationship concepts. Traditional media was found to include too little expression of sentiment. A second attempt was made with only social media mentions, and some success was achieved. In general, coding with humans was found to be much faster and more accurate than automated coding.

by Katie Delahaye Paine

With the ubiquity of social media today, the notion of using social media as a way to listen to and gauge public opinion has captured the interest of marketers. At the moment, the hot button is sentiment measurement, which is the notion that you can teach a computer what positive and negative tone is by precoding it with key words. So the computer searches for brand proximity to words like “suck” and “hate” as a proxy for negative sentiment.

However, sentiment analysis has been shown to be notoriously inaccurate, unreliable, and therefore not terribly useful. With the addition of human coding to improve the model, most sentiment analysis tools can achieve 75% accuracy at best.  

Instead of sentiment, why not relationships?

We postulate that marketers are chasing after the wrong emotion. Rather than attempt to gauge sentiment, it would be far more useful to understand how customers, prospects, employees and other constituents feel about their relationship with your brand. So we decided to try to use the core concepts of the Grunig Relationship model to define an algorithm that would rate how stakeholders feel about an organization based on their conversations in social media.

What we thought we were going to do

We defined a list of key words, phrases and concepts that would be tracked, individually and as a cluster, around each of Grunig’s six components of relationships: Control Mutuality, Trust, Commitment, Satisfaction, Exchange, and Communal. We selected three institutions which we thought would be good candidates for this experiment: Harvard University, Stanford University, and MIT. To ensure comparability, we had to eliminate coverage of sports, since Stanford was headed for the NCAA championships and MIT is not known for its athletic achievement.

We then used human coders to content analyze 2,000 items for the relationship concepts. The idea was to take the terms and concepts that showed up and teach the computer to identify relationship components. The plan was to use an algorithm to score the conversations on a scale of 1 to 5, so that each institution would receive a score on each element of the relationship.

What really happened, and lessons learned

During the human coding phase we learned a tremendous amount about using content analysis to measure relationships. 

Lesson #1: You can’t analyze passion if it’s not there.

The first discovery we made was that when you remove sports from any study of the university environment, most of the emotion and passion goes with it. What was left was a great number of discussions about scientific and academic research, very few of which actually contained any sentiment at all. Of the 2,000 total items, only 265 (13%) actually contained any of the relationship concepts we were trying to study. 

Lesson #2: You Need Twitter and Facebook to analyze sentiment.

No matter what liberals and conservatives may say, the vast majority of traditional media is neutral or balanced, i.e., it doesn’t contain much opinion at all. It also doesn’t reflect what people are directly saying, just what a reporter or an editor has allowed in the piece. So, traditional media does not offer much to work with in a relationship study. You can count the comments in the online versions of these stories, but you also should know that only a small percentage of these are actually seen, and by a very small percentage of the audience. 

In reality, if you want to measure sentiment you need to be analyzing Twitter and Facebook, and that’s the rub. Getting full content from either of those sources is not easy. Most services only get about 15% of Twitter, and even those with a full Firehose licence only get about 85%. Facebook poses an even greater challenge since most content providers only provide what is publicly available, which again is small percentage of the universe. 

Personal blogs are another good source of sentiment, and we definitely recommend including them, but you need to exclude the content farms and fake sites to make sure you are getting a true reflection of people’s opinions, not something a bot created. 

Lesson #3: Trust and Communal Relationships were the easiest relationship concepts to detect. 

Communal Relationships (each party sees mutual benefit in the relationship succeeding) are frequently reflected in an organization’s corporate social responsibility, so community events and good deeds were easily identified as promoting a positive communal relationship. 44% of all item contained some expression of communal relationship and 30% contained references to trust. 

Automated coding: Does not compute

Given the relatively low level of sentiment detected and the challenges we had in training humans to code for relationship concepts, our expectations weren’t very high for automated coding. Nonetheless we used what we learned from human coding to create a taxonomy based on the concepts and key words we had discovered.

While it was interesting to discover that the accuracy level for positive coding was better than negative, overall the machine coding data was unusable. There was simply not sufficient data to analyze with any degree of accuracy. Further, we learned that the linquistic modeling tools we were using require an valid/invalid construct rather than the scale model that we wanted to use. Back to the drawing board. 

So we tried focussing on social media

We then decided to analyze an entirely different set of data: 12,000 purely social media items about a highly controversial organization that had been frequently in the news during the past year. This time the results were much more interesting. We found more than half (56%) of the 12,000 contained some expression of a relationship concept. 

Lesson #4: Some concepts are easier than others to identify and score

Once again Communal and Trust were found most frequently, with 17% of mentions conveying trust and 13% suggesting a communal relationship. However, we found that Satisfaction and Trust were most accurately identified. And we learned that some of the other Grunig concepts are virtually impossible to track, such as Control Mutuality and Exchange Relationship.

Taking it to another level, we learned that it’s relatively easy to describe a set of terms to define trust, but teaching a computer takes a lot more time. So, for example, a human knows the difference between “I fell in love with Stanford” and “I fell in love at Stanford.”  But it requires a fairly sophisticated algorithm to teach a computer the difference.

Pulling trust out of a posting gets even more complicated. So, if someone posts to Facebook that “I fell in love at Stanford and felt very comfortable openly expressing my feelings for my gay lover,” that clearly expresses trust in the Stanford campus environment. But teaching a computer that the feelings are for the campus and not the lover takes a whole other level of complexity.

 Lesson #5: It takes too long.

The only restriction on getting a computer to do this accurately is time. If we had nothing to do but perfect our automated relationship taxonomies for the next year or two, we’d eventually get it right. For now, for the most part, humans are faster and more accurate. 

What’s Next?

Clearly this is an important first step – perhaps just a baby step – in the direction of using content analysis to determine relationship health. We feel pretty confident that, given sufficient emotional content, human coders can accurately glean the concepts of Trust, Communal Relationships, and Satisfaction from social media conversations. As we conduct more such analyses, we will continue to define more words and concepts that will help refine the process and improve accuracy still further. In terms of getting a computer to be as accurate, we clearly have a long way to go.

Further complicating the process is the difficulty in creating a generalized taxonomy. Good taxonomies must be specific to the organization studied. So, to get any generalized taxonomy you will need to model which words or phrases are more important. In order to get there you need a greater quantity of human-coded relationship data to build a statistical model that would validate relationship constructs in social media.  

###

Katie Delahaye Paine is CEO of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. Katie Paine is a dynamic and experienced speaker on public relations and social media measurement. Click here for the schedule of Katie’s upcoming speaking engagements.

Measurement of the Corruption of Nations

Measurement meets bribery, embezzlement, and kickbacks. Exciting, eh? It's great to see research put to such grand public benefit. I'm referring to Transparency International’s Corruption Perceptions Index 2010, which ranks the countries of the world on measures of corruption.

The U.S., for my fellow Americans who are interested, ranks #22. At first glance, that seems disappointingly less than squeaky clean. However, given the fairly broad confidence range for the rankings, what it actually means is that we rank somewhere among the 35 least corrupt countries. So one could argue that the U.S. is doing well, or maybe not so well. I guess it's an envelope-half-empty, envelope-half-full kind of thing.

There are those of us who might wish to quibble with the methodology used here, especially since it is based on perception. But T.I. addresses this (“Perceptions are used because corruption – whether frequency or amount – is to a great extent a hidden activity that is difficult to measure. Over time, perceptions have proved to be a reliable estimate of corruption.”) and provides methodology for those interested.

--Bill Paarlberg, Editor, The Measurement Standard
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement.

Flip-Off Frequency Is Best Predictor of Tax Evasion

Here are some amazing tidbits from "Social Capital: Measurement and Consequences," a speech given by Robert Putnam, see source and link below. (Social capital is the notion that networks of connectedness between people--clubs, for instance--have value to individuals and society):

-- Tax Evasion and The Finger:
"...among the thousands of variables in this dataset, by far the best predictor of tax evasion is the number of times in the course of the last year that you gave the ‘finger’ to another driver." (p. 7)

-- Lawyers Per Capita:
"...historically there were about 40 lawyers, plus or minus one, for every 10,000 employees in America. This number was rock-steady over the first seventy years of this [20th] century. And then this number started to increase, just as trust and social capital started to decline, so that by now lawyers' share in the workforce has more than doubled." (p. 8)

-- Where is Social Capital in the US?:
"...the best single predictor of the level of social capital in American states is distance to the Canadian border. Being closer to the Canadian border means more social capital." (p. 10)

-- Social Capital and Immigration:
"...if you rank Americans today by their level of social capital or social trust or social connectedness, and you rank the countries from which their ancestors come, even as long ago as two or three generations, those two rankings are perfectly correlated, even though the connection between those two streams is on average two or three generations old." (p. 11)

-- Social Capital and the Murder Rate:
"...the strongest predictor of the murder rate is a low level of social capital." (p. 12)

-- Your Chance of Dying and Your Social Capital:
"Controlling for [all] risk factors, your chances of dying over the course of the next year are cut in half by joining one [social] group, and cut to a quarter by joining two groups... It is not that people who are healthy become joiners, it is clear from the studies that the arrow runs in the other direction, from joining to health. (p. 12)

 

And by now you have got to be interested enough to want to read the whole speech, so here it is:

Download the 32-page "Social Capital: Measurement and Consequences" by Robert Putnam now from OECD.org. 

(Thanks to Katie Paine for sending this to me: Putnam, R. (2001), Social Capital: Measurement and Consequences, in The Contribution of Human and Social Capital to Sustained Economic Growth and Well-Being, International Symposium Report edited by the OECD and HRDC.)

--Bill Paarlberg, Editor, The Measurement Standard
The Measurement Standard is a publication of KDPaine & Partners, a company that delivers custom research to measure brand image, public relationships, and engagement. 

eMarketer: What Makes Social Media Trustworthy?

From the eMarketer blog:

"A study of frequent social media users by market research firm Invoke Solutions found that the most trusted information was posted by people respondents knew. But blog posts were more likely to be trusted “completely” than posts on Facebook, and trust dropped off sharply when it came to Twitter, even among friends.

Postings by brands or companies were trusted less, but levels were similar whether companies posted to Facebook or blogs. Online community sites did not hold the same trustworthiness as Facebook or blogs... and respondents had an even more skeptical eye for independent bloggers. And across all categories of content creator, Twitter streams were trusted less than other media."

 

-- Bill Paarlberg, Editor

KDPaine & Partners delivers custom research that measures brand image, public relationships, and engagement to help you measure your success.

 

How To Measure Transparency and Open Government

Without transparency you can't have trust.
And without trust, you can't govern.

by Katie Delahaye Paine

This article came about because I was at the Open Government and Innovations Conference last week, giving a talk about measuring social media. Turns out what people really wanted to know was: "How do you measure transparency and open government?"

There is undeniable pressure in Washington these days to be both transparent and deeply immersed in social media. The problem is that there are also 150 years of political culture that says, "Knowledge is power, keep your mouth shut, and whatever you do, stay away from the media."

After eight years of an administration that was cloaked in secrecy and proved itself at every turn to be pretty untrustworthy, it's not surprising that you could find a Twitterer in every corridor at OGI. Most of the conversations revolved around how to be open and transparent when all the systems block you from doing just that. (Note that just last week the White House -- yes, the current White House -- has blocked access to Twitter.)

But humans will be humans. My favorite quote was from one of the few women who can look chic as hell in camouflage:

L to R: Christina Sukach, Chief of Emerging Technology for the U.S. Air Force, and Katie Paine (photo: Debbie Weil)

Christina Sukach, Chief of Emerging Technology for the U.S. Air Force, explained the military's wholehearted embrace of social media this way: "For years we've had to rely on the media to tell the story. Now we can get the message out and tell our own stories."

Trust, transparency and relationships

Let me say first off that I can't give you a comprehensive program to measure open government. But I will show you how to use a survey to measure trust and transparency, which is the essential starting point.

First some background. The basic concept here is that organizational transparency leads to greater trust in the organization, which results in improved organizational efficiency and lower costs of doing business. For more on this chain of causation, read the introduction to the chapter "Measuring Trust and Mistrust" in my book Measuring Pubic Relationships.

In that book chapter, I show how to measure trust using the Grunig relationship survey. Another way to measure trust is by way of its close connection to transparency. And since transparency is a hallmark of open government, measuring transparency is an essential start to measuring open government.

The truth is, without transparency you can't have trust. And without trust, you can't govern.

Measuring trust and transparency: The work of Brad Rawlins.

Research by Dr. Brad Rawlins of Brigham Young University has demonstrated that the more transparent people perceive an organization to be, the more likely they are to trust the organization. (Proceedings of the 11th Annual International Public Relations Research Conference, 2008) And the more the organization provides honest, open, and occasionally vulnerable communications, the more people trust the institution.

"Organizations that encourage and allow public participation, share substantial information so their publics can make informed decisions, give balanced reports that hold them accountable, and open themselves up to public scrutiny, are more likely to be trusted."

Amazingly, the ability to be open and transparent was found to be more influential than competence in terms of willingness to trust. You can download a pdf of this research here. And you can read an article I wrote about the nuts and bolts of that research ("Trust and Transparency Go Hand In Hand") here.

Now you do the survey

So to measure the transparency of the government, or the Department of Defense, or even of your organization, you can conduct the same survey Brad Rawlins designed for his research. His questions are included in the pdf (just above), and also included at the end of this article.

The nuts and bolts of actually doing the survey we covered a couple months ago in an article entitled "Measuring Naked Relationships: Your step-by-step guide to using relationship metrics to evaluate the success of your social media program." Just adapt the nine-step survey directions given there to administer the survey to your employees or target audience -- but use the Rawlins questions instead of the Grunig questions.

Warning: Conducting surveys is a lot of work if you do them yourself. If you don't do it right (asking leading questions, or questions that don’t get you the data you need) you are going to waste a lot of effort. So, do your homework carefully before you start, and if in doubt, seek professional help.

So once you've tested your transparency and determine your score, what do you need to do about it?

Remember, the concept is that greater transparency leads to greater trust, which in turn leads to improved efficiency and lower costs. And, probably, more open governement. So you use the survey to see where you stand on transparency/trust, then you take some steps to improve.

To improve, take a hard look at the organization. Are you only disclosing what you have to disclose? Or what you want to disclose (greenwashing)? Or what you think sounds good (obfuscating through disclosure)? Or are you disclosing those things that maybe you don't want to disclose but it's the right thing to do? It's the latter strategy that will lead to increased trust.

You also need to look at your corporate or organizational culture. To what extent does it encourage participation in the transparency and disclosure process? Does it have a highly restrictive social media strategy or are employees encouraged to tell their stories? Have your stakeholders taken part in the formation of that policy? Do your stakeholders participate? Are they allowed to identify what they need to know, to ensure that the information shared is relevant and useful? Finally, are people held accountable when they are not transparent? And are they held accountable when being transparent reveals flaws?

So make some changes, then survey again after six months or a year and see if there is any improvement.

Good luck, and let me know how things go.

Survey Questions to Measure Transparency and Trust from Brad Rawlins

Statements using 7-point scale between Strongly Disagree and Strongly Agree.

Overall Trust

1. I’m willing to let the organization make decisions for people like me.

2. I think it is important to watch this organization closely so that it does not take advantage of people like me.

3. I trust the organization to take care of people like me.

Organization shows competence

4. I feel very confident about the skills of this organization.

5. This organization has the ability to accomplish what it says it will do.

6. This organization is known to be successful at the things it tries to do.

Organization shows integrity

7. The organization treats people like me fairly and justly.

8. The organization can be relied on to keep its promises.

9. Sound principles seem to guide the behavior of this organization.

10. This organization does not mislead people like me.

Organization shows goodwill

11. Whenever this organization makes a decision I know it will be concerned about people like me.

12. I believe this organization takes the opinions of people like me into account when making decisions.

13. This organization is interested in the well-being of people like me, not just itself.

Overall transparency

14. The organization wants to understand how its decisions affect people like me.

15. The organization provides information that is useful to people like me for making informed decisions.

16. The organization wants to be accountable to people like me for its actions.

17. The organization wants people like me to know what it is doing and why it is doing it.

Communication efforts are participative

18. Asks for feedback from people like me about the quality of its information.

19. Involves people like me to help identify the information I need.

20. Provides detailed information to people like me.

21. Makes it easy to find the information people like me need.

22. Asks the opinions of people like me before making decisions.

23. Takes the time with people like me to understand who we are and what we need.

Communication efforts provide substantial information

24. Provides information in a timely fashion to people like me.

25. Provides information that is relevant to people like me.

26. Provides information that can be compared to previous performance.

27. Provides information that is complete.

28. Provides information that is easy for people like me to understand.

29. Provides accurate information to people like me.

30. Provides information that is reliable

Communication efforts provide accountability

31. Presents more than one side of controversial issues.

32. Is forthcoming with information that might be damaging to the organization.

33. Is open to criticism by people like me.

34. Freely admits when it has made mistakes.

35. Provides information that can be compared to industry standards.

Communication efforts are secretive (reverse item)

36. Provides only part of the story to people like me.

37. Often leaves out important details in the information it provides to people like me.

38. Provides information that is intentionally written in a way to make it difficult to understand.

39. Is slow to provide information to people like me.

40. Only discloses information when it is required.

###

How to Measure Trust and Mistrust

Chapter 6 of Measuring Public Relationships:
The Data-Driven Communicator's Guide to Success

Trust is in the news of late, so we thought we'd exerpt the chapter on measuring trust from Katie Delahaye Paine's book Measuring Public Relationships. The introduction is below. Click here to download the whole chapter as a pdf. (Click here to buy the book.)

Trust, or lack thereof, has a measurable effect on the financial health of an organization. The accounting firm Arthur Anderson was destroyed after the Enron scandal because its clients lost their trust in its results. Whenever news of tainted beef hits the airwaves consumers lose trust in the safety of their favorite burger, and fast food sales take a dive. Conversely, a key to FedEx’s success is that customers trust the company’s pledge to deliver “When it absolutely, positively has to be there” overnight.

When trust helps an organization build relationships with key constituencies, it saves that organization money by reducing the costs of litigation, regulation, legislation, pressure campaigns, boycotts, or lost revenue that result from bad relationships. A high level of trust helps cultivate relationships with donors, consumers, investors, and legislators who are needed to support organizational goals. When employees trust their employer they are more likely to support the mission of the organization and be satisfied with their jobs. Lower employee turnover has a direct impact on the bottom line. Trust from the financial community is critical to an organization’s access to capital and therefore its ability to grow. Good relationships with the media can often avert a crisis...

Click here to download the whole chapter as a pdf.

How Public Relations Measurement Can Win the War on Terror

It's not a new thing to talk about the war on terror as a war in the media for mindshare. But I've never seen it stated as explicitly as in the article "Marketing Osama" which appeared in a recent issue of The Week, and which was reprinted from a story by John Cook originally published in Radar. The point there is that Brand USA and Brand Osama are fighting it out in the media, and this marketing/PR battle is the important front in the war.

Public relations measurement evaluates exactly the sort of metrics that define the strategy that al Qaida has been using to enlarge its support: media impressions, media content, key messages, contributions and new members. Public relations should be not only measuring the war on terror, but, by virtue of tracking the most important metrics, should be defining the war on terror too. (See "Knowing the Enemy: Can social scientists redefine the "war on terror"?")

One of the more interesting aspects of the war on terror is that there is, (so far, that I know of) no way to know if we are winning or losing, or how we will ever know if we do win or lose. And that's where public relations measurement comes in: If we can use public relations metrics to define progress in the war, we can use them to define success and failure. Public relations measurement can win the war on terror by defining when the war will be over.

And here's how we do it: Let's say that al Qaida's media and marketing efforts now generate X impressions of their key messages each year, which in turn result in Y dollars of contributions and Z new members. We gather and track this data for the last ten or twenty years, and correlate it against terrorist activity of all sorts. (Yeah, it's a big job.) We then will understand what level of exposure to key messages (or Osama bin Laden videos) is required to generate what levels of recruits. Or to generate one new suicide bomber.

Then we define success in terms of the metrics. So, we could define "winning the war on terror" as when al-Qaida's impressions, contributions and recruits are dropping. And then we define victory as the point at which these metric fall low enough so that al Qaida can no longer function effectively. If Osama bin Laden doesn't get his message out, or if no one believes it enough to do anything, then we've won. --Bill Paarlberg

Greater Transparency Is the Key to Building Greater Trust

Trust Measurement

Trust and Transparency
Go Hand In Hand
Brad Rawlins' research shows that doing things right isn't nearly as important as doing the right thing.

by Katie Delahaye Paine

About five years ago, Linda Hon, Jim Grunig and I wrote a white paper about measuring trust that set out some pretty clear steps as to how organizations could do it. At the time, we gave some general advice on what to do if you found that trust in your organization was less than what you wanted it to be.

To be honest, most of these recommendations fell somewhere between "duh," and "of course," and mostly had to do with doing things right:

• Articulate a set of ethical principles,
• Create a process for transparency that is appropriate for current and future operations, and
• Establish a formal system of trust measurement.

And they are all perfectly acceptable things to do.

However, new research by Brad Rawlins (he is our Measurement Maven of the Month for this month), shows that doing things right isn't nearly as important as doing the right thing, and that being transparent is a driving factor in the fostering of trust.

In a recent presentation at the Universidad Del Norte in Barranquilla, Colombia, Dr. Rawlins outlined his findings, and we summarize them below. (You can read the whole paper here, excerpted from the IPRRC proceedings of last spring. Read my blog coverage of that presentation here.)

The overall results of the study demonstrate that transparency and trust are highly correlated, and, "one could conclude that as organizations become more transparent they will also become more trusted." Although the study was limited to employees, the results are strong enough to imply that the correlation between trust and transparency will hold for other stakeholder groups as well.

Definitions

For definitions of trust, see our paper above, since Rawlins uses the same terminology. For transparency, Rawlins starts with the 2005 Mirriam-Webster definition:

• Free from pretense or deceit
• Easily detected or seen through
• Readily understood

He then supplements it with one from Anne Florini of the Brookings Institution:

The opposite of secrecy. Secrecy means deliberately hiding your actions; transparency means deliberately revealing them.

According to Rawlins, there are three aspects of transparency:

1. Informational Transparency means openness, making publicly available all legally releasable information -- whether positive or negative in nature -- in a manner which is accurate, timely, balanced, and unequivocal. Information must be substantial to meet stakeholders needs. Disclosure by itself does not equal transparency, in fact some forms of disclosure can defeat the purposes of transparency.
2. Participatory Transparency is what separates transparency from disclosure. Transparency cannot be successful unless you know what stakeholders want and need to know. So, to ensure that the information shared is relevant and useful, stakeholders must be allowed to identify what they need to know.
3. Accountability Transparency. Transparency holds people accountable for their actions, words and decisions. Rawlins cited The Naked Corporation: If you're going to be naked, you'd better be buff. In other words, if you want to shine, you have to clean up your act.

In addition, Rawlins suggests that each organization might experience differing levels of transparency:

• Active transparency, where transparency is simply part of the culture;
• Forced transparency, in response to Sarbanes-Oxley or other legislation; and
• Pseudo transparency, in which an organization obfuscates through disclosure and greenwashing, which is self-promotion disguised as transparency.

Methodology

1200 employees of a large regional healthcare organization were surveyed on issues of trust and transparency. 385 surveys were completed for a 32% response rate. Twenty-four surveys were deleted because they were incomplete, leaving 361 surveys for analysis. The sample demographics matched approximately those of the healthcare organization's population.

Conclusions

1. Trust and transparency are significantly and strongly correlated.

Trust was closely connected with transparency and the two are positively related. According to Rawlins, "As employee perceptions of organizational transparency increased so did trust. Additionally, the three components of trust (competence, integrity, and goodwill) and three components of transparency (participation, substantial information, and accountability) are positively related."

2. Regression analyses indicate that employees found integrity and goodwill more important to overall trust than competency.

Employee participation that leads to an organization sharing information that employees find useful and substantial, and that holds an organization accountable, is the strongest predictor of overall transparency.

3. Employees see sharing information as a sign of integrity.

Sharing substantial information and being accountable was tied to employee perceptions of organizational integrity.

4. Employee participation and willingness to be accountable was tied to perception of goodwill.

Final Thoughts

Dr. Rawlins is the only speaker I've heard of late that closed his presentation with a quote from the bible that I actually found to be entirely relevant to the presentation.   Would that more of our corporations heed these words:

And this is the condemnation. That light is come into the world, and men loved darkness rather than light, because their deeds were evil, for everyone that doest evil hateth the light, neither cometh to the light, lest his deeds should be reproved, but he that doest a Truth cometh to the light that his deeds maybe e manifest, that they are wrought in God.  John 3:19-21

Can the Votes of a Panel Measure Trust, Reputation, and Other Tricky Things?

How do you measure trust? I just read about an innovative social media approach as applied to news media, (thanks to PR Watch.org). NewsTrust, now still in beta, is "...a social network model which uses the intellect of the masses to rate all manner of news content and news sources..." So, as I take it, news sources will be rated by many readers to result in an overall score that roughly translates to "trustworthiness."

And that brings up an interesting thought: If you can derive a useful measure of trustworthiness by having a bunch of people just vote on it, then why couldn't we measure all kinds of tricky things by having people vote on them? Could we compare the trustworthiness of companies or politicians just by combining ratings from enough people?

And if so, then why go to all the media analysis effort of compiling a Reputation Index for big companies when you could just get a bunch of people to rate the companies? Hey, maybe there's already a social media site called YourRep.com or something where everyone can rank companies to provide an overall reputation score.

I guess if we can use Wikipedia to provide accurate information on, say, Total Quality Management (which I happen to have looked up there just a little while ago, and I feel more or less confident that what I read was accurate), then perhaps we can use a similar consensus-of-many approach to defining (or at least getting a handle on) more nebulous concepts as well. --WTP