Someone on Twitter actually thought I was being ironic when I said that I was looking forward to spending the day at Seth Grimes' Sentiment Analysis Symposium (#SAS11) last week. I guess I'm better known for my skepticism about automated sentiment analysis than I thought. In truth it was a great day, full of fascinating conversations and no shortage of spirited debate. Perhaps most surprising was that the overriding takeaway for me was that while sentiment analysis is making great progress, you still needed humans to clean it up and interpret the data. (The other big point of agreement was that all the sentiment in the world really didn't matter unless it was tied to actual business impact. ) Others have done a better job summarizing the day than I ever could so I strongly urge you to read their write ups. My take on it is: automation is producing interesting results, that might be useful if you're marketing movies or want to get a general sense of the conversation, but I woudn't bet my life or my brand on it.
So it was particularly amusing to come back to a friendly notice from my health insurance company Anthem, clearly automatically generated from data gathered from my pharmacy and equally clearly not vetted by any human (although there is a link to a Harvard Medical School website for more information. What Harvard gets out of this association I'm afraid to find out.) The pamphlet was titled: "My Health Note: A confidential health care summary for Katharine Paine, February 2011."
It then proceeded to offer 4 suggestions, color coded for urgency. The first in red said "Call your doctor" regarding one of my blood pressure medications. The second in yellow referred to the exact same medication and suggested I keep taking it. The third told me to get a flu shot and the fourth said "Ask your doctor about a check up" - On the inside of this pamphlet was a list of all my recent medical activity and a few more details including the point that "you prescription claims show you may not be refilling it often enough."
Never mind the creepiness factor, the strategy behind this completely befuddles me. First of all, it indicates to me that Anthem's data is hopelessly out of date, since it is giving me advice based on data from last summer. Since then, Mr. Anthem, I have in fact had a check up and a flu shot, but your systems are so antiquated that you don't know that? Secondly, it is clearly pushing medication not results. The reason I don't take MORE medicine is that the stuff I do take works, and because I take my own blood pressure at least once a week I know that. But you don't, so stop giving me advice. Finally, in a time when health insurance companies are more hated than lawyers, politicians and banks, it seems extraordinary that you would deploy a system like this without considering the implications on something like this to your overall relationships with your members. Do you understand how much more this makes me mistrust and loathe you? Do you know that this makes me want to advocate for a single-payer system?
Podiums and panels these days are filled with talking heads explaining how they've used social media monitoring and sentiment analysis to learn all kinds of interesting things about their brands. But some of the dirty little secret behind those PowerPoints came out last week when one speaker after another acknowledged the difficulties they'd encountered in getting it right. My personal favorite was the difficulty one vendor experienced in measuring sentiment about the city of Milan. Seems that Milano Cookies generate more positive buzz. Now in the interests of transparency, KDPaine & Partners has a staff of dedicated wonderful people who spend most of their days sorting out differences like "The Cleveland Fed said today…" and "Cleveland is FED up with Lebron James, " to ensure that the our clients don't jump to any erroneous conclusions. The good news is that I get to work with the really smart companies who undertand the need for human intervention before you can draw any false conclusions. The bad news is that the vast majority of companies are like Anthem – far too willing to trust an automated system because it appears to make their job easier. They just don't realize that ultimately the bad data it produces may ultimately yield bad decisions that will inevitably make their job a whole lot harder. To paraphrase Ben Franklin -- an ounce of human intervention is worth a pound of cure.