Twitter Ranking Redux: We road test TweetLevel, Twitalyzer, and MyRealTwitterScore.

 
 by Katie Delahaye Paine

Summary: The author tested three Twitter profile analysis tools by comparing herself to five peer bloggers. Her relative ranking to her peers came out roughly the same for all tools. However, some of the numbers differed so wildly as to make no sense. Recommendations are given for measuring one's own influence and for measuring the relative influence of a blogger.
 

Nary a day goes by when a client doesn’t ask us here at KDPaine & Partners to determine the “Top 50” Twitterers covering their industry. So we spend a lot of time thinking about the best way to do this. Last June, our then Director of Research Chris Near put various Twitter influencer rating systems through their paces, including Twinfluence, TwitterAnalyzer, Twitter Grader, Twitterholic, and  TwitterScore. (Read his results in "Which Twitter Profile Analysis Tool Rules the Nest?")
 

TwitterGrader and TwitterAnalyzer came out on top in Chris’ study. But, as Chris said then, there’s a new tool coming along every week, so we thought it was time to revisit the topic. For this update we added TweetLevel, Twitalyzer, and MyRealTwitterScore. We dropped Twinfluence, TwitterAnalyzer, and TwitterScore from consideration because we couldn’t get them to work. And when we tried MyRealTwitterScore there was insufficient data in their system to rate three of the seven Twitter names in the study.

Even though all these systems crunch the same basic Twitter stats, each uses a different algorithm and each weighs different numbers in different ways. As such, each offers different insights into your program and is good for different things.  I’ll let you pick which one is right for you. Here’s how we conducted the study: 

Methodology:
 
Step 1. Establish a competitive benchmark set  
For the purposes of this analysis I picked 7 Twitterers that I considered my peers: 

• Avinash Kaushik (@avinashkaushik)
• Eric T. Peterson (@erictpeterson)
• Johna Burke (@gojohnab)
• Richard Bagnall (@richardbagnall) 
• Shonali Burke (@shonali)
• Amber Naslund (@ambercadabra)

Step 2: Decide what is important 
Some of these systems (TweetLevel, TwitterGrader, TwitterHolic and MyRealTwitterScore) give you one number to rate a person by, which makes life much easier when you’re in a hurry. Others offer various ranking metrics, including "Influence," "Popularity," "Engagement," "Trust," "Velocity" and "Clout." These metrics have a general but unspecific relationship to the standard meanings of their names. They all combine the various stats that are available in Twitter, including number of tweets, number followed, number of followers, number of retweets, and number of references. Sometimes the metric is expressed as a percentage of all people on Twitter, in other cases it’s a single number. 

For the purposes of ranking the top people, I went with the overall numbers plus the Influence scores that combine the references and likelihood of retweets. Frankly, because of the various ways now available to game the system, I consider the number of followers irrelevant.

The Results: 
 

Conclusions:

Some numbers didn't seem to make sense, so I created a peer average to better understand the landscape. The next thing I realized was that while it was nice to know that I scored high on Engagement, and low on Influence, there really wasn’t much I was going to do about it. (And you can’t help but wonder how I managed to score better on most Twitalyzer grades than that guy who invented the system.)

Tweetlevel and Twitalyzer are definitely the fastest. While Twitter Grader entertains you with cute sayings while you wait for it to generate a report, this is only amusing for the first one or two tries. If you’re trying to judge the relative influence of a half a dozen bloggers it will drive you nuts. Also, if you have a long list of names to check, you may find it easier to use a single-number system, since the multiple criteria may make things too confusing. Or if you’re using Twitalyzer or Tweetlevel decide on one number -- Impact, Clout, or Influence -- and track just that. 

And the winner is...

What I really wanted to know was, based on a basic gut check, which analysis system made the most sense given my knowledge of my own behavior and my peers. To that end, Tweetlevel seemed to make the most sense, no doubt because it uses such a broad combination of numbers and algorithms.

• If you’re measuring your own level of influence, then focus on the increase in your score, not the absolute number. Now that we’ve done this exercise once, I’m planning on going back and rechecking my scores once a month to see if I’m improving and hitting my goals. I’ve got the data in a basic Excel spread sheet, so I can easily generate charts to note the trend. Twitalyzer has a fabulous guide that tells you exactly how to improve your scores on their system. 
• If you’re trying to figure out the relative influence of a blogger that is following you, look at their ranking relative to other bloggers. 

Comments

[Shonali]

I love how you explained this, Katie, though as far as me being your peer, knock my socks off a little more, won't you?! Cor blimey, as I'm willing to wager Richard would say.

Avinash had a really great post evaluating some of the different Twitter "measurement" services, and thank you for adding to it. No matter how funky these services seem, when I look at some of the numbers, I know they just don't make sense (I mean, since when have I had more clout than you, as Twitalyzer would have us believe?!). Which leads me to my next point...

What worries me the most is when I hear people using absolute measures, e.g. number of followers, number of retweets, etc., without any context WHATSOEVER, because, really, what does that mean? If it ain't in context, it don't mean nuttin'. So thank you for setting us straight... again.

[KDPaine]

thanks! and I totally agree. it's what you do with these metrics that count. I think as a very simple way to determine if someone if worth paying attention to or if its safe to ignore.

[Bill Paarlberg]

The article on paid Twitter search (coming soon from TweetUp) in today's NYTimes ( http://tinyurl.com/ycj25go ) says it will use Klout ( http://klout.com/ ) to measure Twitter influence. Any experience with Klout?

[Barb Chamberlain]

This is pretty much what I've been doing with measurement of the @WSUSpokane account I manage. Figuring it's all relative, not absolute, I just look at the trendlines. As long as we're getting better at "it"--whatever "it" is in each system's secret sauce--we're doing something right.

I also measure click-through rate on links we share. The average number of clicks per follower is increasing even as our follower count goes up, which says to me that we're attracting followers who find our content increasingly relevant. Those must be the "right" followers.

I'd like some good ways of doing a quick analysis on our followers, since I have goals for attracting specific types related to the primary focus of our programs and campus overall (people in our region; accounts with a heavy health care/health sciences emphasis; accounts with a design emphasis).

Any analysis of those kinds of tools forthcoming? http://twitterexport.com/ lets you export either friends or followers into a spreadsheet so you can do some manipulation there.

@BarbChamberlain