How Do Your Customers Really Feel? – Let the Machines Figure it out For You!

In today’s social driven economy, consumers are encouraged to express their satisfaction with a company or commodity through social media, twitter, blogs, and reviews.  More now than ever, it is easy to tap into pre-trained machine learning models that infer sentiment on consumers’ commentary. Referred to as natural language processing (NLP), machine learning of this nature creates impactful analytics and understanding, which enables organizations to make unbiased decisions and quickly react to consumer demands. Not to mention these models can crunch through enormous amounts of text/commentary to draw insights that are simply not efficient or feasible through manual effort.

Real World Application of Leveraging a NLP Machine Learning Model

SC&H recently leveraged Amazon Web Services’ (AWS) cloud infrastructure & machine learning services to accomplish this task, inferring sentiment on restaurant reviews taken from a third party review website. This ‘Big Data’ Pipeline consists of the following steps:

1. Collection – Ingesting source data in varying formats, combining & normalizing data into one central location.
2. Process – Raw data is messy; it first must be transformed to a flat, uniform structure to be used efficiently
3. Analyze – Data on its own is meaningless; we provide its true value by analyzing and visualizing it for consumers to gain insights.

How did we do this? At a high level, we performed the following activities:

1. Ingestion: Collected and stored raw data from 3rd party review websites using object storage. We used S3 buckets in this step.
2. Catalog: Performed a catalog of raw data attributes, taxonomy, & metadata. We used AWS Glue Data Catalog in this step.
3. Analyze Raw Data: Data discovery of all sources, classifying data into objects for analysis. We used a serverless database, known as AWS Athena, in this step.
4. Machine Learning Sentiment Services: Using AWS’ Comprehend Service, triggered command to infer sentiment (a.k.a. natural language processing) on strings of review text. This was triggered using AWS Lambdas using Python as the programming language of choice.
5. ETL to Data Warehouse: Triggered AWS Glue job to extract, transform, and load data into AWS Data Warehouse, called AWS Redshift. This was called using AWS Lambdas using Python as the programming language of choice.
6. Visualize in BI Tool: Prepared advanced visualizations in Tableau.

How Can You Make Decisions With this Data?

Want to see the end visualization we built as a result?  Here’s a sample dashboard SC&H created, demonstrating a time series of total reviews by sentiment category (mixed, negative, neutral, or positive sentiment respectively):

In conclusion, machine learning models are available as a relatively cheap service to add an elevated method of understanding your data, making actionable decisions based on your consumers’ feelings of your company or commodity.  Getting started with these technologies can sometimes be daunting, but organizations that embrace these tools in the correct manner embrace change/growth, eliminate manual and costly processes and ultimately become more proactive to drive success within their organization.