SPSS Statistics

 View Only

Analysis of #CharlieHebdo sentiment with SPSS

By Armand Ruiz posted Fri January 09, 2015 05:44 AM

  
With the horrible attack in Paris  in the Charlie Hebdo office, we are experiencing once more a new way to be informed about last news, this time powered by Twitter. It is amazing how fast people are sharing thoughts, photos, links, and absolutely everything. It thus becomes the data set of the world population's mind in real time.

In this post I am going to show how to query tweets and do some simple analysis using IBM SPSS Modeler and the new SPSS Predictive Extensions based on R. All this analysis...without any coding at all!  We are going to do 3 things:

- Create a Word Cloud with a new WordCloud node based on the R wordcloud package.

- Integration of RCharts with IBM SPSS Modeler. RCharts (developed by Ramnath Vaidyanathan) was born as the initiative to bring powerful JavaScript visualization for R users. So they can now create these interactive charts without having JavaScript skills, only with R. With this integration within SPSS workbench, you don't even need to know R in order to use them. Simply drag and drop the node and start getting powerful results that are easy to share. These are the libraries available now in IBM SPSS Modeler:

-Integration with the new R package HTMLWidgets. This package enable you to add new types of HTML output to R Markdown documents. There are different types of widgets like maps, charts, 3D scatterplots and more.

NewNodesSPSS

This is the stream I built with these new IBM SPSS Predictive Extensions. I selected 10,000 tweets with the hashtag #CharlieHebdo. Then I am doing some text analysis of sentiment of the tweets and generating 4 different outputs. I want to emphasize that this analysis is very simple and I want only to show how fast and easy is to get quite interesting results. Another important point is that the outputs generated with IBM SPSS Modeler and these extensions are easily embeddable into blogs (like this one), social media or any other place you want to share them and...they are interactive! Try to mouse over the charts and you will see nice animations.

SPSSsentimentAnalysis- Output 1:  The word cloud. This node is useful to highlight the most commonly cited words in a text using a quick visualization. The node is cleaning up the data, removing words without meaning and keeping the important ones and counting them.

Rplot

 

-Output 2: Bar Chart of Emotion. After using the new 'Sentiment Analysis' node, the tweets are classified into anger, fear, joy, surprise, disgust and sadness. I am aggregating the sentiment and creating a Bar Chart using the RCharts node.  The sentiment analysis node is based in the sentiment package that you can find in the CRAN network.


-Output 3: Bar Chart of Polarity. Another output of the new 'Sentiment Analysis' node is the polarity. This is classifing the tweets into positive, neutral or negative. Again we are creating a bar chart using the new RChart node.



As you can see, the sentiment about the attack is negative and people are feeling anger and fear.  I am not focusing the post in explaining the algorithms behind, if you want more information about it you can go to the R packages documentation.

Output 4: HTML datatable. Here there is an interactive data frame of the first tweets and the classification after analyzing the sentiment of each of the tweets.



To finish the post, here you have a map created with CartoDB of the geotagged tweets mentioning #JeSuisCharlie using the Paris time zone in January 7, 2014. It is not generated with IBM SPSS Modeler but we are working in the integration and we have already some experimental CartoDB nodes.
cartoDB









#bluemix
#predictiveextensions
#Programmability
#rstats
#SPSS
#SPSSModeler
#SPSSStatistics
#Usecases
11 comments
2 views

Permalink

Comments

Sun July 01, 2018 11:32 AM

Fantastic beat ! I wish to apprentice even as you amend your website, how can i subscribe for a weblog website?
The account aided me a applicable deal. I
had been tiny bit familiar of this your broadcast
provided vibrant transparent idea

Thu November 10, 2016 09:54 PM

Hi Armand

Is the Sentiment Analysis node now available and if so, where can i access it?

Regards

Thu July 28, 2016 07:09 PM

Hi Armand,

I am planning to use sentiment analysis to classify the polarity into positive/negative/neutral as above.. found the extension here https://github.com/IBMPredictiveAnalytics/Sentiment_Analysis_with_Alchemy, which is unfortunately for version 18 modeler. Is there any previous version of text analysis extension that works for Modeler version 17?

Thanks!

Mon April 25, 2016 01:49 AM

Yes we are prototyping with R Markdown and Shiny within SPSS but any contribution is welcomed! ;-)

Sun April 24, 2016 12:33 PM

Hi Armand --
Great use case for SPSS+R. Is it possible to embed R Markdown visualizations, including Shiny widgets, within SPSS stream, possibly in R node. That would be a plus for SPSS to orchestrate the entire end-to-end process.

Sun April 10, 2016 10:39 PM

now it is available: https://github.com/IBMPredictiveAnalytics/Word_Cloud_Visualization

Wed September 16, 2015 07:59 AM

Word Cloud would be awesome. Do you have a date for release onto the Predictive Analytics Extensions community?

I wouldn't mind getting my hands on the Sentiment Analysis node just for demo purposes, it would be a great value add for us.

Thanks a lot

Tue September 15, 2015 09:42 PM

Not sure when, the one we could prioritize is Word Cloud. Sentiment Analysis is not thaaaat great since it works only with english and it is pretty basic. You should check maybe Alchemy APIs for good sentiment analysis. Regarding the twitter one...I don't know at all.

Tue September 15, 2015 09:39 AM

Hi Armand,

Do you know when the Twitter search, Word Cloud and Sentiment Analysis nodes will be available for us to download?

Thanks,
Nabeel

Fri June 19, 2015 05:12 PM

Thanks Adolfo, you can find many extensions here: http://ibmpredictiveanalytics.github.io/

Some of them are not yet available but will be there very soon :)

Fri June 19, 2015 04:24 PM

Hi Armand:

What amazing demonstration of the capabilities of Modeler and R, I'm impressed with your article. Are these SPSS Modeler nodes that you demonstrated available in some IBM web page (analyticszone, developerworks, partnerworld, etc.)?

Best regards!