Social media is a platform which provides a kind of virtual life for a people to openly express feeling, opinions and beliefs. Various events are affected by peoples sentiments which provided by the social media. Unfortunately, hateful speech and abusive language target or harassed the many users engaging online, either on social media or forums. So before publication of the any online content there is a need of detecting such hateful and abusive language. The services provided by different online social networks like Twitter, Instagram and Facebook are used by people from different backgrounds, interest and culture.
So the communication between these various background peoples are increased day by day which results into more cyber conflicts between these peoples. According to the most national and international legislations hate speech is referred as expressions that prompts to harm, discrimination, hostility and violence based on identifies social group or demographic group. Hate speech can include any form of expressions such as image, videos, songs as well as written comments.
Written comments are considered in this research. So the hate speech detection is called as a text or content specific analytics task.
In many countries including United Kingdom, Canada and France there are laws prohibiting hate speech social media services such as Facebook, twitter which not having enough provision for hate speech or specific race? These websites are open space for people to discuss and share thoughts and opinions and makes it almost impossible to control their contents. Furthermore many people tend to use aggressive and hateful language.
Ultimately, these all thing are pointed towards the automatic hate speech detection. There are many solutions which depends on the Natural Language Processing are available for automatic hate speech detection. But it having drawback, NLP approaches are totally depends on language used in the text. This gives the motivation to machine learning techniques like neural networks for classification task which mostly uses the pre-trained vectors (e.g. Glove, Word2Vec) as word embedding and achieve better result from classification model. If user uses short slang word for hate speech then these machine learning techniques fail to detect it. To overcome this problem next sentiment polarities are detected in the tweets (Barnaghi et al., 2016) to differentiate the speech. At the same time unsupervised learning models (15) are also used for the hate speech detection. Deep learning approaches (2, 4, and 5) also give the better result in the hate speech detection.
Our approach employs deep learning architecture for the text classification, a recurrent neural networks composed of Long-Short-Term-Memory (LSTM) based classifiers. Then we show experimental evaluation of model with other algorithms.
1.1 Problem Statement
The problem we address in this work can be described as follows:
Let q be an unlabeled sentence (i.e. tweet) contains number of words, posted by user u. Let C, H, O be three classes denoting Clean, Hateful, and Offensive respectively. These classes contain the pasting members with content classified as per the corresponding class. For which O . Further given that user u has another tweets posting , we assume that all contain tweets are classified in classes C, H and O, similarly other tweets by other user. Base on these facts, the problem is to identify the class, which the unlabeled sentence q by user u belongs to. Then gives the prediction of the user u from class H.
1.2 Goals and Objectives
The primary aim of this research is Hate speech detection on Twitter. Long-Short-Term-Memory (LSTM) which is a part of recurrent neural network (RNN) and basics of RNN these two classifications algorithms allowed together on sample date set to find effectiveness of proposed approach for finding hateful speech. Further the research work focuses on those hate speech and their users various features like number of followers, number of followee, date of create account, etc. and predict the hateful user who may gives always the hateful speech.
The scope of this dissertation includes
Increase our understanding of Hate Speech on social media sites like Twitter.
Detection of the Hateful tweets then categories it into three categories: Hateful, Offensive and Clean.
Analyze the effectiveness of the proposed deep learning approach i.e. LSTM.
Giving the prediction about hateful users.
Among variety of online social networking websites, such as Twitter, Facebook, YouTube, Instagram, Linkedin etc. Twitter is the one on which mostly teenagers and adolescents are active. It represents the importance of detection and removal of online hate content. Currently there are quite lot of researches done for automatic detection of online hate speech. The method for detecting the hate speech can be divided into two categories: one manual feature engineering which is consumed by algorithm such as SVM, Na?ve Bayes and Logistic Regression and another represents deep learning paradigm.
In this research for automatic detection of hate speech we use deep learning method i.e. combination of Convolutional Neural Network (CNN) and Long-Short-Term-Memory (LSTM) a type of recurrent neural network. Intuitively, the CNN learns features similar to n-gram sequence and LSTM sequence order that are both useful for classification. The machine learning methods i.e. supervised algorithms and unsupervised algorithms i.e. Na?ve Bayes, SVM, Logistic Regression (18) uses the dictionaries and may choose wrong sentence for hate speech also decrease the accuracy. The deep neural network architecture for hate speech detection (5) outperforms than the machine learning methods. In this research we use ternary classification i.e. whether the tweet is Clean, Offensive or Hateful (1). The offensive means it may be racism. Lastly the features of the user also useful in the hate speech detection. So, the various features of the hateful users observe to predict the always hateful users.
1.4 Structure of Dissertation Report
The organization of this dissertation report is as follows:
Chapter 1: Introduction
Chapter 1 is addressed by introducing hate speech detection problem, goals, objectives and motivation. It also contains the problem statement.
Chapter 2: Literature Survey
This chapter presents the literature review of related work and ends by briefing the research contribution.
Chapter 3: Proposed System
The general research framework that focuses on the detection of the hate speech and classify into three categories i.e. Hateful, Offensive and Clean with the help of deep learning classification algorithms. After classification analyze the hateful speech users and give prediction about them.
Chapter 4: Performance Analysis
This chapter includes the result analysis of the deep learning algorithm used in this research work.
Chapter 5: Conclusion
This chapter includes the dissertation report with a summary of result.
The references used for completion of this research work are mentioned in this section.
In the simple sentence word based approach, fail to identify the hate speech and offensive speech as well as affect on the speech expressions and emotions freedom. Most of words and sentences can have many different meanings in different context which is called Ambiguity Problem. Because of ambiguity problem false positive rate is high i.e. increased. So word based approach is avoided in hate speech detection. Also in Natural language Processing (NLP) approaches are not effective to detect comment by user or unusual spellings. So it is called Spelling Variation Problem which is caused by the replacement of character in a token. Hate speech detection was done by different type. Firstly Lexical Based Approach in which machine uses language pattern, grammar, rules created manually. N.D, Gitary et al (4) presents the model which uses lexicon to find hate speech. Second is machine learning approach. Davidson et al (1) presents multi-classifier model for classification of hate speech, offensive and clean. Hajime Watanabe et al (11) uses n-gram feature to classify tweets in recommended three classes. Third is Hybrid approach in which learning based as well as lexical based approach (8) is used. Lastly Pinkesh Badjatiya et al (5) compare the deep learning methods i.e. Fast Text, CNN and LSTM also task specific embeddings learned using these three methods.
2.2 Previous Work
There has some work done on this topic, specially hate speech detection on social media such as YouTube, Twitter, Facebook etc including online content as well as various data sets.
2018 Hate Speech Detection using Convolution-LSTM Based Deep Neural Network. Zigi Zhang, David Robinson, Jonathan Tepper. Deep neural network is used for classification by combines the convolutional and LSTM networks with drop-out and pooling which improves classification accuracy.
2018 Detecting Offensive Language in Tweets using Deep Learning. Georgios Pitsilis, Heri Ramampioro, Helge Langseth. By using the user behavioral characteristics and features of the user, multiple LSTM based classifiers are used for classification of hate speech in two categories.
2017 Using Convolutional Neural Networks to Classify Hate Speech. Bjorn Gamback, Utpal Kumar Shidar. Here Deep learning method i.e. Convolutional neural network (CNN) is used to classify tweets in four categories by four CNN trained model for each class. The feature set is map in network by max-pooling and softmax function for classification.
2017 Improving Hate Speech Detection with Deep Learning Ensembles. Steven Zimmerman, Chris Fox, Udo Kruschwitz. It creates ensemble model by taking average of softmax result of various models. It shows that in deep learning weight initialization method have important role.
2017 Deep Learning for Hate Speech Detection in Tweets. Pinkesh Badjatiya, Shashank Gupta, Vasudeva Varma. Multiple deep learning architectures are experiment to learn semantic word embeddings. When Random embeddings from deep neural network model and Gradient boosted decision trees are combine, it gives best result.
2017 One Step and Two Step Classification for Abusive Language Detection on Twitter. Ji Ho Park, Pascale Fung. Ti gives two step classification for abusive language detection and classify into racism, sexism and clean. It used Hybrid CNN which takes both word and character features as input. Two step approach gives better result than one step approach.
2016 Abusive Language Detection in Online User Content. Chikashi Nobata, Joel Tetreault, Achint Thomas. Here supervised classification method which uses four NLP features (n-gram, Linguistic, Syntactic and Distributional Semantics) are used. When these features combined with standard NLP features, result increased.
2015 Hate Speech Detection with Comment Embeddings. Nemanja Djuric, Jing Zhou, Robin Morris. In this paper, paragraph2vec for comments and words joint modeling used which is learn by continuous BOW natural language model. Then to distinguish between hateful and clean, embedding is used to train a binary classification. It address issues of scarcity and high dimensionality.
2015 A Lexicon Based Approach for Hate Speech Detection. Njagi Dennis, Zhang Zuping, Jun Long. In this paper, model the classifier which uses sentiment analysis techniques for subjectivity detection. It calculate rate of the polarity of sentiment expressions and remove objective sentences. Then bootstrapping is used for classification.
2004 Classifying Racist Text using Support Vector Machine. Edel Greevy, Alan F. Smeaton. SVMs are used to automatically classify the web pages. It uses bigram representation of a web page within a SVM.
Table 2.1: Summary of Literature Survey
In the paper , sentiment, semantic, pattern and Unigram feature are used to classify the tweets. It provides the binary and ternary classification with 87.4% and 78.4% accuracy respectively. This result is the average of all features i.e. sentiment feature, semantic feature, pattern feature and unigram feature. In the work , Convolutional Neural Network (CNN) models i.e. Random vectors, word2vec and character n-gram is used separately and then calculates the average of their results. CNN models give the 78.3% accuracy. In advanced to prior work, Bjorn Gamback et al , combination of convolution neural network and long short term memory is applied for the classification. It experiments on the seven data set, five of them classified in racism and sexism, one is classified in Refugee and Muslim and one is classified in general (hate and non-hate). There is variation in the result for various data sets by using the CNN and LSTM. While in the work , two step classification is used, one is for classify the abusive language and another is for hate speech classification. It uses the HybridCNN which gives the 95% accuracy. In the paper , convolution neural network is used along with Gated Recurrent Unit (GRU), a part of recurrent neural network i.e. convolutional GRU. Convolutional GRU is work better on small data set.
2.3 Gap in Existing Literature
The existing work have various gaps in it, the work is on psychological, behavioral and personal reasons. There are various deep learning methods used for hate speech detection, which gives better result. We have to analyze the all methods, algorithms and ensemble the deep learning algorithms to improve the detection of hate speech. Also all hate speech detection focuses on the content on social media. But very few focus on user and their behavior. User is the starting point for the hate speech, so their must focus on users feature. This limitation is overcome by shifting focus from tweets to hateful user. So in this research, after classification in hateful, offensive and clean, we analyze the features of user of hateful speech and giving the prediction of hateful user.
The detection of offensive and hate speech on twitter is a crucial functionality for an ensemble approach towards tackling harassment and misdemeanor. Quick and rigorous detection of hate speech can result in a timely reaction of the users and uploader for removal of the tweets or other responses on the twitter. However, one step ahead attempt can be made to impede hate speech on twitter. In this chapter the general research framework is to focuses on the detection of such hate speech and user features for prediction with the help of classification algorithms. The solution proposed is in this type of analysis based on data generated by users on twitter. The chapter includes the insight challenges that are essential to be explore and studied to fulfill development of proposed research work. Along with the hate speech detection the prediction of the users who always gives hate speech is also done for better result.
Figure 3.1 presents the architecture for the proposed solution approach. A framework for the research goal of detecting hate speech uses a 16k annotated data set taken from twitter and verified by experts. This research comprises the problem of hate speech detection in three categories i.e. Hateful, Offensive and Clean.
The proposed approach is a multi-step process consists of 4 phases- training and the testing data collection, Feature extraction and classification of tweet based on classification algorithms.
The intend of this chapter is to present empirical analysis done on the machine learning, deep learning and other methods which contributes and used in hate speech detection on the twitter data. At the end the accuracy results are conferred that represents the effectiveness of the proposed solution.
4.1 Experimental Dataset
The required data for experiment purpose is collected using public Twitter API. The dataset used for the experiment is contains approximately 16k annotate tweets, which is examined by experts and available by Wassem and Hovy (13). This dataset contains 3166 tweets labeled as Hateful, 1943 tweets labeled as Offensive and 10889 tweets labeled as Clean. Remaining are unlabeled which labeled manually. These all tweets are gathered by 613 users.
Table 4.1: Size of Experimental Dataset
The above table 4.1 shows the size of training and test dataset collected for hate speech detection. The dataset as well as training dataset contains all hateful, offensive and clean tweets.
4.2 Evaluation Matrix
To measure the effectiveness as well as the efficiency of proposed solution approach standard performance measures i.e confusion matrix have been used where each rows of the matrix represents the precedent of the actual class and each column represents the precedent of the predicted class. In this case of classifier, each instance can only assigned to either irrelevant class or relevant class.
The precision of the class X is the ratio of number of tweets correctly classified to the total predicted as tweets of class X. And the recall of the class X is the ratio of number of tweets correctly classified to the number of tweets present in class x.
The above table 4.2 shows the standard confusion matrix. Let A represents the number of tweets correctly classified as relevant, B represents the incorrectly classified tweets as irrelevant, C represents the number of irrelevant tweets correctly classified as positive, and D represents the number of irrelevant tweets correctly classified as irrelevant. Then the performance attributes of proposed classifier can be given by:
Above formulas represents how to calculate the precision, recall and accuracy of the algorithms. Accuracy is the proportion of true result with both true and false result.
This research work has presented a multi-perspective study on tweets of twitter which can be considered as hate speech. The first step was to understand the source and nature of problem in order to identify the aspects for which measures could be considered for the reduction of such hate speech on social media website.
5.2 Future Scope
In future stages his work could be extend by considering performance of this system on larger and more diverse training and testing datasets. To achieve the accuracy up to 100% the analysis should be done more on the