---
title: Topic and Opinion Classification based Information Credibility Analysis on Twitter
tags:  #学会  
author: [Yukino Ikegami](https://convert.docswell.com/user/yukinoi)
site: [Docswell](https://www.docswell.com/)
thumbnail: https://bcdn.docswell.com/page/GJWDDW4W72.jpg?width=480
description: My presentation about information credibility analysis at SMC 2013.
published: June 14, 15
canonical: https://convert.docswell.com/s/yukinoi/5XYXNY-twitter-information-credibility
---
# Page. 1

![Page Image](https://bcdn.docswell.com/page/GJWDDW4W72.jpg)

Topic and Opinion Classification
based Information Credibility
Analysis on Twitter
Yukino Ikegami
Kenta Kawai
Yoshimi Namihira
Setsuo Tsuruta
At SMC 2013
2013/10/16
1


# Page. 2

![Page Image](https://bcdn.docswell.com/page/4EZGG522J3.jpg)

Background and Motivation
• False rumors often confuse people
• Confirming reliability of rumors often requires
a domain knowledge about the problem
➢Automatically Information credibility analysis
2013/10/16
2


# Page. 3

![Page Image](https://bcdn.docswell.com/page/Y76YYV66EV.jpg)

Related Work (1)
Using Web-page-dependent features
• [Wassmer et al., 2005]
– Use credentials of the site, advertisements and
Web design
• [Castillo et al. 2011]
– Twitter-dependent features
• E.g. number of followers
– Twitter-independent features
• E.g. number of !/?
2013/10/16
3


# Page. 4

![Page Image](https://bcdn.docswell.com/page/G75GGD6574.jpg)

Related Work (2)
Using textual features
• Rumor information cloud system [Miyabe et al.
2011]
– Confirm a rumor whether is truth or not by alerting
information about a rumor
– Find correcting information by SVM applying word ngrams model
– The word n-gram model consists of words in front and
back of the word “デマ” (“dema” is the abbreviation
of “demagogic” in Japanese-English).
• Dematter [Toriumi et al. 2012]
– Assesse credibility by the percentage of alerting
tweets about a rumor
– Detect alerting tweets by keyword matching
2013/10/16
4


# Page. 5

![Page Image](https://bcdn.docswell.com/page/9J2YY8MGJR.jpg)

Topic and Opinion Classification based
Information Credibility Analysis
Tweet crawler
Topic &amp; opinion
classifier
Tweet opinion
DB
Twitter
Tweet credibility
calculator
2013/10/16
5


# Page. 6

![Page Image](https://bcdn.docswell.com/page/DEYGG6XGEM.jpg)

Topic classification
• Classify tweet by topic model
– Topic model:
Latent Dirichlet Allocation (LDA)
with Gibbs sampling [Griffiths, 2002]
– Feature:
content words (i.e.) noun, verb, adjective, adverb
Topic1
Topic2
Topic3
Vegetable
Measure
Radioactive material
Eat
Amount of radiation
In prefecture
No problem
Result
Governor
Leaf of tea
Pool
Fukushima
2013/10/16
6


# Page. 7

![Page Image](https://bcdn.docswell.com/page/VJNGGZ1878.jpg)

Opinion Classification
• Classify whether a tweet is positive opinion or
negative one by a dictionary
• Takamura’s semantic orientation dictionary
[Takamura et al. 2006]
– Contains word-positivity [-1, 1] pairs
2013/10/16
7


# Page. 8

![Page Image](https://bcdn.docswell.com/page/YE9WWZ5XE3.jpg)

Information Credibility Assessment
• Majority decision
Negative
All tweets
Positive
Tweets
about the interest topic
2013/10/16
8


# Page. 9

![Page Image](https://bcdn.docswell.com/page/GE8NN1Y97D.jpg)

Evaluation
• Dataset: 2960 tweets
– Confirmed whether it is true or not by human
• Criteria: Weighted kappa
– Weight w is designed as follows:
judging certainly false-information as certainly
true or vice versa are critical error
2013/10/16
9


# Page. 10

![Page Image](https://bcdn.docswell.com/page/LELPPQ4M7R.jpg)

Result
TABLE 1: Kappa of each conditions
Fully Random
method
(All tweets)
Our method
(All tweets)
Our method
(Only Topic &amp; Opinion correct)
0.003
0.604
0.616
• Landis’s kappa guideline: κ &gt; 0.61 is substantial
• Our method has the substantial effectiveness
for assessing tweet credibility
2013/10/16
10


# Page. 11

![Page Image](https://bcdn.docswell.com/page/4JM441K67W.jpg)

Conclusion
• Topic and opinion classification based
information analysis on Twitter
– Topic model and sentiment analysis based
majority decision
• Evaluation shows it has substantial effect
2013/10/16
11


# Page. 12

![Page Image](https://bcdn.docswell.com/page/PJRWWLXN79.jpg)

Future works
• Weighting tweets by author’s expertise
– people often determine whether information is
trustworthy or not by author’s expertise
• Applying online topic model
– New topic and usage of existing words are created
one after another
• Excluding neutral tweets
– No-sentiment tweets are useless on our method
2013/10/16
12


# Page. 13

![Page Image](https://bcdn.docswell.com/page/PEXPPW687X.jpg)

References
• [Wassmer et al. 2015] M. Wassmer and C. Eastman,
“Automatic evaluation of credibility on the Web,” ASIS&amp;T
2005, 42(1), 2005.
• [Castillo et al. 2011] Castillo, C., Mendoza, M., and Poblete,
B. “Information credibility on twitter,” WWW 2011, pp. 675684, 2011.
• [Miyabe et al. 2005] M. Miyabe, A. Umejima, A. Nadamoto
and E. Aramaki, “Proposal of Rumor Information Cloud
based on Rumor-Correction Information” (In Japanese),
RRDS4-019, 2011.
• [Toriumi et al. 2006] F. Toriumi, K. Shinoda, G. Kaneyama,
“Accuracy Evaluation of Dema- gogue Detection System
using Social Media” (In Japanese), IPSJ Digital Practice, 3.3,
pp. 201-208, 2012.
2013/10/16
13


