The top 3 things that are broken in Sentiment Analysis

With Sentiment Analysis brands and retailers aim to mine the opinions of consumers to be able to understand how they are performing across a number of topics as well as how their competitors are performing along the same topics. Given that this year (2018), an estimated 3.2 billion people will be using social media worldwide there is no shortage of data to mine in social media, and there are additional data sources such as online reviews, call center data, survey results, etc.

The challenge with all this data is that it’s very hard to mine it. It’s a lot of data, and mining it relies on having a team of experts around your Sentiment Analysis software to look for keywords and terms patterns. This is a lot of effort as you need to guess and test quite a bit:

1. Guess all the topics that consumers care about around a product or a service. How can you guess ALL the different topics consumers talk about? The short answer is you can’t, which means you will miss quite a bit. This is why in most cases these teams focus on a short list of high level topics – value for money, loyalty, quality, etc.

2. Guess ALL the different ways that people talk about the same topic: Imagine how kids, teenagers, boys, girls, adults of different ages all speak about the same topic. If we take the example of an electrical appliance battery, you’d need to look for all negative expressions about it such as “doesn’t hold charge” or “weak battery” or “dies on me after 2 hours of use”

3. Guess new hot topic: With new products comes new issues. You’d need to know something has come up in order for you to look for it, and then obviously you need to guess the different variations of terms used by to describe it

So where does this leave us? It leaves us in a world where Sentiment Analysis is relying on lengthy, ongoing substantial manual effort to deliver visibility into a short list of topics that are typically high-level ones.

Where would we want to be if we could imagine an optimal world?

1 – Automation

With so much information available online and in house, solutions relying on humans for pattern identification, even AI driven systems that rely on IT and experts to setup, are too slow. IT and data scientists will just not be able to respond quickly enough to every business data request while in parallel they still need to fine tune and configure a data mining system to respond to competitor and market changes. The key is automation. Find the automated data mining systems that can harvest the insights without delays. Fortunately, they exist now.

2 – Granularity

If you figured out a way to access insights and mine data automatically, you need to keep in mind that generic, one size fits all data (loyalty, quality…) is not usable to all roles in the organization. Specific roles, operational roles, need specific data. A Product Manager needs granular data on the product that he is selling, not on the category or the brand. Therefore you need granular data that each role can slice and dice for their own use and needs. Also keep in mind operation roles needs change ongoing, one day it’s a product competitive analysis and the next day its positioning or roadmap. They all can benefit from granular data, but different compositions of it for the different tasks.

3 – Accessibility

Once we have granular data we can get automatically, you’d want to encourage a wide use of this data across roles in the organization. Unlike the current status where centralized groups maintain the Sentiment Analysis software, the optimal solution needs to be one that everyone can use. It needs to be intuitive, and autonomous. If the solution is complex or if it requires IT or Insights or any other centralized group to change or support or configure. You want to empower the masses to take action and they can’t take it if they don’t have control


Sentiment Analysis is widely used today as a way to understand consumers, the challenge with the current state of the industry is that it is –

• Requiring lots of manual efforts on the journey to value
• Consumer understanding is limited to parts of the topics consumers care about
• The end result is only useable/accessible to a small subset of roles in the organization

The key challenge is to drive meaningful insights from the masses of data that you have, and for this, most existing technologies that rely on humans are not strong enough. Consumer interests are too much of a fast-moving target that is also very eclectic in terms of patterns.

The good news is that the data is there for us to see and mine.

Revuze is an innovative software vendor that addresses just this with the first self training, low touch solution that can mine consumer data automatically. This is why it’s much more granular and typically delivers 5-10X the insights compared to anything else, and it does it without humans helping.

Leave a Reply

Your email address will not be published. Required fields are marked *