COMP621U, Spring 2011
Yuanfeng SONG, Jan VOSECKY
Topic: Sentiment Analysis on Twitter
Twitter dataset provided by Z. Cheng, J. Caverlee, and K. Lee and used in CIKM 2010.
The training set: 115,886 Twitter users
and 3,844,612 tweets from the users.
The test set: 5,136 Twitter users and 5,156,047 tweets from the users.
Z. Cheng, J. Caverlee, and K. Lee. You Are Where You Tweet: A Content
locating Twitter Users. In Proceeding of the 19
th ACM Conference on Information
and Knowledge Management (CIKM), Toronto, Oct 2010.
We plan to employ several machine learning techniques to extract users’ sentiment (emotion)
from the content of their twitter messages (‘tweets’). Our g
eneral approach will consist of the
Manually label a set of training and testing instances
Represent tweets in an appropriate format, such as Bag
Indentify any additional features specific to tweets to include in t
he feature vector, in
order to leverage additional information which may increase classification accuracy.
Currently, we are considering the addition of
, such as the timestamp.
Investigate attribute selection and transformation possibilities
Build a machine
learning model from the training data
Evaluate the model on the testing data
We plan to experiment with a number of machine
learning algorithms and compare their
effectiveness. These may include Bayesian classifiers, Maximum
Entropy, Support Vector
Machines, as well as clustering algorithms.
Possible extensions to our work:
sensitive sentiment analysis: public sentiment with respect to specific topics.
Comparison between different locations.
clouds: listing the most
words for a category of tweets.
There has been a large amount of prior research in sentiment analysis, especially in the
domain of product reviews, movie reviews, and blogs. Pang and Lee  is an up
f previous work in sentiment analysis. Sentiment analysis works are mainly focusing
on designing platform or tools to do automatic sentiment analysis using models from machine
learning area such as latent semantic analysis
(LSA), Naive Bayes, support vecto
(SVM) etc.  . Besides model
, another difference between th
se works is different
, such as Twitter,
logs etc. Following this is the difference
feature set and different
feature extraction methodology. For example, Mishne  u
ses many features extracted from
Live Journal web blog service to train
SVM binary classifier for sentiment analysis. Alec
Go  uses
Twitter dataset and extract
features from message
to do semantic analysis.
this problem as a positive and negative emotion classification problem,
to identify more kinds of emotions. Jung et al.  show that there are
some idiosyncratic natures of mood expression in Plurk messages; for example, initial mo
may change as time pass
by (which also known as the fluctuation of moods).
some blogs are so intertwined that
even difficult for human
, not to mention for
machine. All these character
hard to identif
y multiple emotions.
detection can be used for different area
such as recommendation system
mediated communication (CMC).
Table 1 Confusion Matrix
Classification Mean average Precision
Kappa is used to measure the agreement between predicted and observed categorizations of a
dataset, while correcting for agreement
that occurs by chance. The equation is as follows:
Where Pe is the hypothetical probability of chance agreement, using the observed data to
calculate the probabilities of each observer randomly saying each category.
 G. Mishne, “Experiments with Mood Classification in Blog Posts,” in Proceedings of the
1st Workshop on Stylistic Analysis of Text For Information Access, 2005.
 A. Go, R. Bhayani, and L. Huang, “Twitter sentimen
t classification using distant
supervision,” Dec 2009. [Online]. Available:
 L. Terveen, W. Hill, B. Amento, D. McDonald, and J. Creter, “PHOAKS: A system for
ons,” Communications of the Association for Computing Machinery
(CACM), vol. 40, pp. 59
 MY Chen. etc. Classifying Mood in Plurks. The 22nd Conference on Computational
Linguistics and Speech Processing. Chi
Nan University, Taiwan.
 Y. Jun
g, Y. Choi, and S.H. Myaeng, “Determining mood for a blog by combining multiple
sources of evidence,” in Proceedings of IEEE/WIC/ACM International Conference on Web
Intelligence, pp. 271
 J. B. Walther, and K.P. D'addario, “The Impacts of Em
oticons on Message Interpretation
Mediated Communication,” Social Science Review, vol. 19, no. 3, pp. 324
 Confusion matrix. [Online]