Text Analytics Distraction
Text analytics is a fascinating space that people are eager to understand. Many leading text analytics influencers suggest that we’ve only reached the “tip of the iceberg” on the potential of text analytics. This may be why the “Future of Text Analytics” is such a hot topic. This topic was one of the main themes of the Text Analytics World conference recently held in Boston. The program chair, Tom Reamy, gave a keynote presentation on that theme, Future Directions in Text Analytics. In addition to Reamy’s presentation, leading text analytics expert, Tom H. C. Anderson, was asked for his opinion on the matter. A few of his responses stuck out to me and so I reached out to Tom for clarification. I think his responses go along way into understanding where the Text Analytics industry is currently and the things that are holding the industry back. Anderson’s original conversation with Reamy and my follow up interview are outlined below:
Tom Reamy’s original question to Tom Anderson: What are the problems and issues that are slowing down the field?
Tom Anderson: The infatuation with “Social Media Monitoring” which really mainly is “Twitter Monitoring”. Until walled gardens around Facebook and LinkedIn data come down (I’ve been waiting and waiting), there really is limited usefulness in this area and we may be better off concentrating more of our efforts elsewhere. As clients start realizing they’re just listening to 8% of the population on Twitter or blogs, whom really often are somewhat different than normal customers they begin to question the ROI here.
Why this is intriguing
I think on the surface level people look at social media monitoring as the “future” or potential of text analytics. In my opinion, social media monitoring is viewed as a “shiny new toy”. Tom’s opinion is intriguing because he believes the opposite (right now). He believes social media monitoring is actually hindering the growth of text analytics and that our time is best spent elsewhere.
My follow up to Tom Anderson:
“Tom, this is a bold statement. Can you elaborate on why you think (under today’s marketplace) our time invested in text analytics is best spent on things other than social media monitoring?”
Tom Anderson’s response:
“We’re often talking to large companies (with rather boring products that normal folks would hardly ever mention on social media). For some reason they have a large budget to monitor tweets, yet have no budget/are basically ignoring the comments of half a million actual customers who are emailing and calling them each year. It’s insane! I can give other examples as well.”
Tom Reamy’s original question to Tom Anderson: Do you see any revolutionary changes for text analytics on the horizon?
Tom Anderson: Yes, what I’ve been talking about a lot is domain expertise. OdinText for instance is focused on the use of text analytics for consumer insights. That is a very different thing than using text analytics for engaging with Twitter or detecting terrorists or fraud etc. All these require special knowledge, rule and code modification.
I think there will be less “Enterprise” [claims to do everything] as well as “Twitter Monitoring” firms, and a lot more domain and industry specific text analytics tools/firms.
Also this technology will be incorporated by most of the companies that own sizable amounts of unstructured data. So there will be more licensing and acquisitions going on.
My follow up to Tom Anderson:
It seems like you’re suggesting that at this time there is no “one size fits all” text analytics tool. Can you elaborate further on why domain expertise is important and why you see Twitter Monitoring firms on the decline moving forward?
Tom Anderson’s response:
“Regarding domain expertise, it depends to some degree on your text analytics technology, but for the most part either the technology is designed to handle/answer a specific type of text analytics problem and/or the solution actually requires very domain (actually company and data source) specific training data. The analysis of helicopter test pilot flight logs for instance is very different from voice of customer data which is very different from trying to detect certain illegal activities or fraud. Some claim to do everything equally well, watch out for those firms!
Regarding Twitter, when we’re talking about just 140 character tweets though, does it really matter how good your text analytics approach is? The answer is no, not really.
Don’t get me wrong. I am INCREDIBLY OPTIMISTIC on text analytics! But not all data is created equal, and there is A LOT of data out there other than Tweets!”
My key takeaway from Tom Anderson’s original interview with Reamy and his follow up interview with me:
Although social media may have a unique value in text analytics at some point in time, right now it is acting more as a distraction than an enabler. Perhaps over time, if the walls around meaningful social data come down, this stance may change. Until then, I think we would be wise to focus on the areas right in front of our nose.
If you’re interested, you can check out Tom Reamy’s full interview with Tom Anderson.
Leave a Reply