יום שלישי, 7 ביולי 2009

The next big thing in social media: Sentiment Analysis

(This is a translation of the previous post)

As someone's who's been analyzing the Business Intelligence market for years, I'm very interested in exploring ways to leverage the vast amount of valuable data that is accumulated in social networks sites today, in the same way in which organizations can now leverage business information data that is kept (unlike the web - in a fairly orderly manner) inside their organizational applications and databases. Or perhaps this comparison is not a fair one and we should actually try to look at the content of social networks in a completely different way?

Marketing and PR departments are now trying to figure out "what is being said" about the company / product throughout the web, specifically in websites that are based on user generated content. This information can be extremely valuable, you can understand a lot about how your company / products / services are really perceived, what are the current preferences, trends, and characteristics of cultural consumption relevant to the company.
In order to satisfy exactly this need a new breed of data-mining tools aimed specifically at social networks started to emerge in recent years. These tools have some similarities to "traditional" data-mining tools, and especially to non-structured data mining / Text Mining tools.

The domain of generating insights based on social networks' content is referred to as "Sentiment Analysis": trying to understand what is the writer's attitude towards a particular object.
The purpose of these tools is to allow organizations to "listen" to the conversations taking place of social networks and analyzing these conversations in order to identify some emotion towards the company or product.
This listening is only the first part of the initiative, the second is to actually use this insight as an actionable one and respond in a certain way - act to protect the brand, try to correct, leverage positive blogger review etc.

A new breed of software tools have emerged that try to provide an answer to these needs. These tools provide capabilities that enable text analysis, creating clusters of content that contain positive versus negative sentiments, some of them provide an "overall sentiment score" based on a large group of various content sources about a particular product/campaign etc., and an important capability in these tools is the ability to define specific rules by the marketing department of the organization (for example, a specific word can significantly change the score and this word changed from one organization to the other).
An example of such tools: Umbria (focuses on the social websites - for example, blogs), Biz360, and an example of an Israeli company - Buzzilla.

In this link (Blog of 451 research group) there is a pretty long list of tools for sentiment analysis.

What are the problems currently associated with sentiment analysis of social networks?

  • Content is highly unstructured, there's no control over the way people express themselves, frequent use shortcuts and slang, language problems, and use of different media types - text, audio, video
  • Large number of sites to "listen to", the important sites change from industry to industry (On the other hand, this fact only stresses the need to do it in a computerized way rather than manual)
  • Privacy - currently this is not a problem but in the future it may not be possible to listen to conversations on the network so easily as it is today
  • Another potential problem that I'm assuming will arise at a certain point once companies start to use this tools will be the potential abuse of the media and tools. Imagine that every time you complain about a service you get a tweet from a service rep checking how they can help, or tweeting about a product you're looking for and getting a promotional message as a result… this has a potential of being over-used to the point of creating a negative association with the company.

There are several levels of tools usage, starting from only listening at a pretty basic level of sophistication (being able to identify content that directly mentions the product name); Through monitoring with a little more sophistication (eg, creating clustered content, while taking into consideration other content that might be of interest – for example, content about competitors etc.); Another level would be listening to content and acting as a response (creating an automatic response sent people complaining about the service, or distribution of a link of a positive review to other relevant sites etc.); Another level can be to connect these insights to other systems (for example, CRM packages are now starting to include, as part of their systems "sentiment score" that appears alongside the client's information. This topic is somewhat similar to sentiment analysis on other channels - for example, "emotion detection" in the call center by use of tools from the speech recognition space. Tools in this area recognize voice tone (and associates it with emotion), specific words ("word spotting") - for example, lawsuit/ name of competitor, and can lead to action such as escalation of the call.


But we should be careful when trying to analyze social websites content in the same way in which we analyze content other conventional content types that are far more organized. This is an entirely different media. Any attempt to try and compare the different media types with Twitter, for example, will be problematic. As of now, Twitter provides the quickest means information transfer "from the field" than any other media (A good example is the transfer of information now taking place regarding what is happening in the streets of Iran). And so, this media can be very useful in trying to get a quick picture of the impact of a particular campaign on the conversations "from the field" almost in real time.

In my opinion, organizations now starting to take advantage of the powerful content hidden in social networks will not only get a better "feel" their market in order to protect their brand images, but if applied with correct use, these tools can also help organizations take advantage of these conversations for the purpose of promoting customer centric innovation.

אין תגובות: