Advances in Fuzzy Systems
Volume 2018 (2018), Article ID 2371621, 9 pages
https://doi.org/10.1155/2018/2371621

Research Article

A Hybrid System for Subjectivity Analysis

Samir Rustamov1,2

¹ADA University, Ahmadbey Aghaoglu Street 11, Bakı AZ1008, Baku, Azerbaijan
²Institute of Control Systems of Azerbaijan National Academy of Sciences, Bakhtiyar Vahabzadeh Street 9, AZ1141, Baku, Azerbaijan

Correspondence should be addressed to Samir Rustamov; samir.rustamov@gmail.com

Received 30 June 2017; Accepted 3 May 2018; Published 3 June 2018

Academic Editor: Ferdinando Di Martino

Copyright © 2018 Samir Rustamov. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract.

We suggested different structured hybrid systems for the sentence-level subjectivity analysis based on three supervised machine learning algorithms, namely, Hidden Markov Model, Fuzzy Control System, and Adaptive Neuro-Fuzzy Inference System. The suggested feature extraction algorithm in our experiment computes a feature vector using statistical textual terms frequencies in a training dataset not having the use of any lexical knowledge except tokenization. Taking into consideration this fact, the above-mentioned methods may be employed in other languages as these methods do not utilize the morphological, syntactical, and lexical analysis in the classification problems.

1. Introduction

Identification of subjective data from web documents having opinions within are gaining incrementing interest. Opinions are often views formed by individuals about their sentiments, appraisals, or feelings, etc., not necessarily based on fact or knowledge. Identification of subjectivity attempts to recognize if this written piece of work conveys opinions (personal) or a body of objective facts [1]. This analysis has been utilized in many natural language and text mining solutions. With the aim of generating more instructive data, subjectivity detection has been employed as a primary sifting stage in a lot of natural language processing assignments.

Through our experimentation we are aiming to work out techniques in order to establish classifiers able to identify subjective expressions from objective ones. By means of language independent feature weighting, in the experiment the sentence-level subjectivity classification is attained. A subjectivity database from the opinions about films of “Rotten Tomatoes” [2] was deployed as an experiment.

In the paper, we suggested different structures of hybrid systems based on various supervised machine learning algorithms such as Hidden Markov Model (HMM), Adaptive Neuro-Fuzzy Inference System (ANFIS), and Fuzzy Control System (FCS) which achieved sufficient results. These machine learning methods have been employed for subjectivity analysis individually [3, 4] and our aim is to improve performance of classification by using hybrid systems, which is successfully applied by us in sentiment analysis and natural language call routing problem [5, 6]. Our feature extraction algorithm computes a feature vector using the statistical textual terms frequencies in the corpus not having the use of any lexical knowledge except tokenization. Taking into consideration this fact, the above-mentioned methods may be employed in other languages as these methods do not utilize the lexical, grammatical, and syntactical analysis within the classification process.

2. Related Work

With the aim of recognizing the subjective data in written piece of work or speech, various nonsimilar supervised and unsupervised learning algorithms have been recently analyzed.

A bootstrap technique was employed by Riloff and Wiebe [7] with the aim of studying subjectivity classifiers from a nonannotated texts collection. An identical method was applied by Wiebe and Riloff [8]; however, they studied objective sentences aside from subjective sentences.

With the objective to classify subjective and objective sentences, a MinCut based algorithm was deployed by Pang and Lee [9]. Through this experiment they targeted eliminating objective expressions from each analysis and ameliorating the classification accuracy up to 86.4%.

A Web mining technique was offered by Grefenstette et al. [10] in order to identify subjective adjectives.

Techniques of classifying the strength of opinion within personal sentences were suggested by Wilson et al. [11] and Kim et al. [12].

Subsumption relationships were defined by Riloff et al. [13] within n-grams, unigrams, and lexico-syntactic samples. It was discovered that, in case of subsumption of one feature by another, there is no need for subsumed feature. The subsumption pecking order eliminates a feature set; hence the eliminated feature sets may ameliorate classification functionality.

The utilization of prosodic feature, word -grams, character -grams, and phoneme -grams for subjectivity identification and polarity subsumption of conversation acts in multiparty dialog was researched by Raaijmakers et al. [14]. It was figured out that, for subjectivity detection, an association of phoneme-level information, word-level, prosodic, and character-level provides better performance. The best performance is acquired with an association of phonemes, words, and characters in polarity classification.

By utilizing -gram word successions with fluctuating level of lexical instantiation, the study of patterns of subjectivity available both in labelled and unlabelled information was suggested by Carenini and Murray [15]. It was demonstrated that study of subjective written words with fluctuating instantiation levels from both commented as well as unprocessed information may ameliorate subjectivity recognition and polarity labelling for speech in the meetings and email messages.

Delta TFIDF which is the intuitional general purpose method was suggested by Martineau and Finin [16] to weigh word scores effectively prior to classification. The comparison was made between the results of TFIDFs SVM difference and SVM Term Count Baseline for subjectivity subsumption. The comparison outcomes demonstrated that Delta TFIDF SVM gives low variance with high accuracy.

The subjectivity of postings on Twitter was categorized by Barbosa and Feng [17] according to two feature types such as metainformation about the terms on tweets and specifications of how tweets have been written.

With the aim of recognizing sentence-level subjectivity, Yulan He [18] suggested SubjLDA through adjusting the unused Dirichlet allocation method by including an additional layer to describe subjectivity meaning.

The classification of subjectivity at the segment level was suggested by Benamara et al. [19] for discourse-based sentiment analysis. Each component was classified into four groups such as S, OO, O, and SN wherein S components comprise unambiguously lexicalized subjective as well as appraising phrases, OO components are positive or negative expression embodied in an objective component, O components comprise neither a word with lexicalized subjectivity nor embodied opinion, and SN components are subjective, however nonappraising, which are utilized to express thoughts.

Remus [20] showed that using readability formulae and their combinations as features in addition to already well-known subjectivity clues leads to significant accuracy improvements in sentence-level subjectivity classification.

Bayesian model with pecking order grounded on Latent Dirichlet Allocation, called subjLDA, for sentence-level subjectivity recognition that reflexively determines if the used expression is sentiment or fact was proposed by Lin et al., [1].

Each of these models is based on English information and an English subjectivity dictionary is utilized by most of models. Some models have been recently presented on sentence subjectivity classification in Japanese [21], Chinese [22, 23], Romanian [24, 25], Urdu [26], and Arabic [27] and other models used various machine learning algorithms utilizing universal set of features as well as language specific features.

With the aim of automatically generating resources for a new language through utilizing the resources and instruments for English for subjectivity analysis, various models were researched by Mihalcea et al. [25] and Banea et al. [24]. With the objective of establishing most accurate classifiers for subjectivity classification; another method was employed by Banea et al. [28].

Some methods which aim to discover features which can be employed in other languages have been recently investigated. For instance, the classification of subjectivity which utilizes features which does not depend on language was suggested by Mogadala and Varma [29] and tests were conducted on 5 various languages.

Appel et al. [30] proposed a hybrid approach using SentiWordNet [31] and fuzzy sets to estimate the semantic orientation polarity and intensity of sentiment words, before computing the sentence-level sentiments. Muhammad et al. [32] introduced a lexicon-based sentiment classification system for social media genres, which captures contextual polarity from both local and global context. Fernández-Gavilanes et al. [33] proposed a novel approach to predict sentiment in online texts based on an unsupervised dependency parsing-based text classification method [34].

There are many other types of sentiment analysis that have been developed, such as irony detection [35], multilingual support [36], and differentiating negative emotions [37].

The application of supervised models using language independent features for classification of objective and subjective expressions is our primary target within the present experiment.

3. Feature Extraction

Feature extraction algorithms are an important element of any machine learning model. This algorithm is characterized by us as instinctive and computationally effective which does not demand any supplementary comment by individuals as well as lexical knowledge except tokenization. The mentioned algorithm comprises two parts: preprocessing (information preparation) and computation of the feature vectors.

4. Preprocessing

5000 objective and 5000 subjective labelled sentences from movie reviews are utilized for training process [9]. The classification based on machine learning uses two several sets of documents such as a training set and a test set. The training set is utilized by an automatic classifier with the aim of learning the distinguishing features of documents, and the test set is utilized to authenticate the performance classifier. We currently illustrate the information dispatching in the dataset. With the aim of forming the word list, the following proceedings are implemented:

(i)Gather all documents from the training set and create one whole document;(ii)Transfigure content of the document to term matrix;

Taking into consideration that our goal is to avoid lexical knowledge utilization, every term is considered as a single code term. According to our algorithm characteristics, verbs are not sorted in particular tenses (i.e., “contribute” in present tense and “contributed” in past tense) as well as nouns in singular or plural (i.e., “infant” in singular and “infants” in plural). In lieu, words are considered as different terms.

We parted the data collection stochastically into 2 parts for testing and training and created 10 folds. 90% of the data in each folder is used for training and 10% for testing, respectively.

5. Calculation of Feature Vectors

Frequency statistics analysis is used in majority of language independent feature extraction algorithms. Its periodically weighting functions are utilized. In the present research, we endeavored to scrutinize which features were appropriate for Neuro-Fuzzy models.

Let us describe some of the parameters:

1. is a number of terms in the training set;2. is a number of classes;3. is a number of sentences in the training set;4. are the sentences in the training dataset, where is the length of rth sentence, ;5. describes the association between ith term and the jth class ;6. is the number of times ith term occurred in the jth class;7. denotes the occurrence times of the ith term in the training set;8.frequency formula for the th term in the th class

A new weighting factor having an impact on the system accuracy is applied so that in lieu of the quantity of documents we take the quantity of classes in the prevalent Inverse-Document Frequency (IDF) formula. Like IDF, we distinguish this coefficient as Pruned ICF (Inverse-Class Frequency) [3].

where is a term, is the number of classes containing the term , in which , where

is found empirically for the current problem.

The membership degree of the terms () for relevant classes may be evaluated by specialists or can be computed by means of empirical methods. Taking into consideration that we target not to utilize human comments or lexical knowledge, we estimated the membership degree of every word through an analytical method as mentioned below :

6. Subjectivity Detection Using Fuzzy Control System

Fuzzy inference is the process wherein the mapping is formulated from given data to outcome(s) through utilizing fuzzy logic. By means of a basis provided by this mapping decisions can be made, or samples can be distinguished. Fuzzy inference process draws in logic operations if-then rules and membership functions [38]. In lieu of expert knowledge, in the primary phase with the aim of computing membership function, we apply a statistical method. Afterwards, we employ fuzzy operations and through the backpropagation algorithm we adjust parameters. The general structure of Fuzzy Control System is illustrated in Figure 1.

Figure 1: General description of FCS.

Our algorithm () is applied as follows.

(1)() - membership degree of terms of the th sentence is computed by using (4)-(5).

The Center of Gravity Defuzzification (CoGD) technique is employed for defuzzification. This method prevents defuzzification ambiguities, in case output degree of membership is calculated from several crisp output values.

Objective and subjective expressions classified in accordance with classes are trained by a FCS.

The cost function is expressed as follows [39]:

,   is the desired output.

The function’s partial derivatives are computed as follows:

We applied conjugate gradient decent method for minimization of function (6) and used optimal value in next stage.

Rounding value of gives index of appropriate class, if it satisfies the following condition:

where and is the indicator of the relevant class. In this case, is the primary quantity, which affects to the system reliability.

Inspection of feature vector providing the best outcomes for Fuzzy Control System is obvious. Average accuracies of FCS over 10-fold cross validation are illustrated in Table 1. It must be emphasized that these outcomes hinge on the classification technique and might be dissimilar for every classifier.

Table 1: The results of Fuzzy Control Systems with TF and features.

Features
TF

Accuracy
89.87 %
91.32 %

Delta TFIDF features [16] related to FCS were inspected as well. Due to fact that Delta IDF weighting factors of both classes are identical, the accuracy of the system is not modified in case of employing Delta IDF weighting factor. As it is seen from Table 1, the accuracy of the system increases following the deployment of Pruned ICF weighting coefficient. Recognition results of FCS with different values of are given in Table 2.

Table 2: The results of Fuzzy Control Systems with features.

Δ₁ = 0.3
Δ₁ = 0.4
Δ₁ = 0.5

Correct (%)
76.53
85.17
91.02

Reject (%)
20.81
10.02
0.01

Error (%)
2.66
4.81
7.97

The rejection result is 0.01 for . It means after applying Pruned ICF weighting 0.01% expressions’ terms become 0 in the testing process.

7. Adaptive Neuro-Fuzzy Inference System For Subjectivity Detection

General structure of Adaptive Neuro-Fuzzy Inference System is demonstrated in Figure 2 [40]. Regarding the linguistic notes, the fuzzy interface block gives an input vector to a Multilayer Artificial Neural Network (MANN).

Figure 2: The general structure of Adaptive Neuro-Fuzzy Inference System.

We utilized statistical evaluation of membership degree of words by (5) in lieu of linguistic statements in the primary phase. Afterwards, we employed (3) and (4) fuzzy operations.

MANN is applied to the output of the fuzzification operation. The structure of MANN in ANFIS is illustrated in Figure 3.

Figure 3: The structure of MANN in ANFIS.

The main parameters of MANN are described as follows.

1. is the number of MANNs layers;2. is the number of neurons on layer , ;3. is set of neurons of layer , which connected to the neuron on layer ;4. is bias of neuron on layer ;5. is weighted coefficient (synapse) of connection between neuron on layer and neuron on layer ;6.is the input vector; the output of fuzzification is given to the MANN as input, ;7. and are state and output value of neuron on layer for input signal of the neural networks.

where is a given nonlinear activation function for which we use the hyperbolic tangent

Let the training set , pairs be given, where is desired output for input signal. The training of MANN consists in finding such and , herewith on input signal that MANN has output , which is optimally close to the desired output . MANN’s outputs are taken as indexes of classes appropriate to the sentences. Usually, training quality is defined by the mean square error function [41]:

where is coefficient, which determines the belonging “quality” of input to its “ideal” pattern .

The task of MANN training stipulates to minimize the criterion (14) according to parameters with (10)-(12) requirements. The MANN in the developed system was trained using the conjugate gradient method.

We set two thresholds for the acceptance recognition:

1.,2..

is the output of Multilayer Artificial Neural Network, and are maximum two elements of the vector . It can be described as follows:

Subjectivity detection results of ANFIS with various and are given in Table 3.

Table 3: Cross validation results of ANFIS with feature.

No restriction
;
;

Correct (%)
92.16
85.85
78.81

Reject (%)
0.01
8.58
18.72

Error (%)
7.83
5.57
2.47

The accuracy of FCS (92.02%) is less than the accuracy of ANFIS (92.16%) due to extra variables in the hidden layers of the MANN.

8. Subjectivity Detection Using Hidden Markov Model

HMMs are a substantial statistical method for modeling generative sequences that can be categorized by a fundamental process generating a perceptible sequence. HMMs have been successfully applied in speech recognition. They have also been used successfully in some NLP problems such as part-of-speech tagging, expression yielding, and excerpting necessary data from documents [42].

With the aim of classifying objective and subjective expressions, a discrete ergodic HMM was employed. We parted expressions into a lot of states and yielded terms comprising these states. Gathering such states provides us with better outcomes and does not use knowledge of language.

Let us describe main parameters of HMM as follows [4]:

1. is the initial state distributions: , where N is the number of states.2. is the state transition probability matrix, , .3. are the state-dependent observation probabilities. Here, for every state j, is the probability distribution of words occurring in states.4. are the observation sequences, where R is the number of observed sequences, is the length of r-th observed sequence, and , T is the given quantity, .

We applied Baum-Welch algorithm for the training of HMMs for all classes. The scaled-forward algorithm is applied for the estimating probability of classes for each observed sentence in testing process. The maximum probability among probabilities of classes is found as a decision (Figure 4).

Figure 4: Subjectivity detection using HMM.

The HMM outcomes with various states are illustrated in Table 4. Experiments were carried out for the ergodic and left-right Hidden Markov Models. According to experiments ergodic HMMs with 2 states show best performance.

Table 4: The results of ergodic and left-right HMMs for detecting sentence level subjectivity.

1 state
2 states
3 states
5 states

Left-right HMM (%)
89,37
88,26
87,28
86,85

Ergodic HMM (%)
89,38
89,63
89,33
89,21

9. Hybrid Systems

With the aim of classifying the subjectivity and objectivity of sentences, we suggest employing a combined system utilizing the HMM, FCS, and ANFIS methods. Within the experiment process we analyze each sentence by all methods. The outcomes of HMMs, FCS, and ANFIS are directed to the decision-making block and compared therein.

Two types of hybrid systems are suggested.

9.1. Hybrid-I

The outcomes verified by the HMMs, FCS, and ANFIS methods are confirmed by this system. In case of decline of some of these models’ classification, no decision is accepted by the system. As the system forestalls the error in the testing process, it is considered more reliable. The results of Hybrid-I are shown in Table 5.

Table 5: The results of Hybrid-I.

FCS(%)
ANFIS (%)
EHMM-2 (%)
Hybrid –I

Correct ()
Rejection ()
Error ()

Average
92.02
92.16
89.63
88.21
6.67
5.12

9.2. Hybrid-II

We suggest a sequential method in this system. In accordance with the procedure, in case of failure by one classifier to classify a sentence, the unclassified sentence will be passed to the next classifier up to classification of the given sentence or no availability of another classifier. By means of this method, the number of rejected sentences is reduced.

At first ANFIS is applied to rejected sentences by FCS and then consequently ergodic HMMs with 2 states are applied to the rejected sentences by ANFIS in Hybrid-II. By this way we increase classification rate up to 92.24%. The results of Hybrid-II are given in Table 6.

Table 6: Average 10-fold CV results of Hybrid-II.

FCS
()
ANFİS ()
Applied rejected samples of FCS
EHMM-2 
()
Hybrid-II 
Correct ()

Correct 
()
Rejection 
()
Error 
()
Correct 
()
Rejection 
()
Error 
()

76.53
20.81
2.66
78.52
18.76
2.72
73.13
92.24

10. Conclusion

Three various classification system methods such as FCS, ANFIS, and HMM were depicted and employed by us for detecting the sentence-level subjectivity in a film analysis database. Training and testing mechanisms of these methods have been particularly demonstrated by us to classify sentences as objective and subjective. Through this analysis we targeted developing methods which would be capable of being applied in various languages since they do not depend on linguistic knowledge. The feature extraction proceeding is a significant element of these methods. We concentrated on study of informative features enhancing the precision of the systems which do not have language specific restrictions. Thus “Pruned ICF Weighting Function” novel was conceived with a parameter specially calculated for the subjectivity data set. While bringing the already created system in comparison with other different systems, we should foreground that, in fact, the utilization of linguistic knowledge increases precision. Due to fact that linguistic knowledge is not used in our method, our output may solely be compared with methods having same restrictions which utilize components based on bag of terms that are investigated on the same data collection. Researches of Pang and Lee [9] and Martineau and Finin [16] are included in those examples. By means of Naıve Bayes classifiers Pang and Lee achieve 92% accuracy on sentence-level subjectivity classification and through utilization of SVMs they report 90% on the same set of data. 91.26% accuracy was achieved by Martineau and Finin [16] utilizing SVM Difference of TFIDFs. The presently achieved outputs are similar as follows: FCS (92.02%), ANFIS (92.16%), and HMM (89.63%). Notwithstanding these indications, the methods we suggested have certain superiorities. Since function (6) is reduced only regarding (in the identified problem N=2), FCS has low complexity compared to other sophisticated supervised machine learning methods. Through extra fluctuations added in the central layer of the neural network, ANFIS may slightly increase accuracy. In case of applying IF-THEN rules in ANFIS and FCS, it is assumed that the accuracy of the system will increase to the level where it gets abreast with human discernment. FCS and ANFIS categorize sentences by dint of occurrence of the words in the output; however the HMM categorizes sentences according to the arrangement of the sentences. The given interpretation can be utilized to establish two various types of hybrid systems. Through combining multiple classifiers, a better performance than the result of any classifier can be achieved. The Hybrid-I system forestalls particular errors in the classification process, and the Hybrid-II system improved the accuracy to 92.24%.

Conflicts of Interest

The author declares that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work has been carried out in Center for Data Analytics Research at ADA University.

References

Lin. Chenghua, He. Yulan, and Richard. Everson, “Sentence subjectivity detection with weakly-supervised learning,” in Proceedings of the 5th International Joint Conference on Natural Language Processing, pp. 1153–1161, 2011.

http://www.cs.cornell.edu/people/pabo/movie-review-data. Subjectivity DataSet.

S. S. Rustamov and M. A. Clements, “Sentence-level subjectivity detection using neuro-fuzzy and hidden markov models,” in Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis in (NAACL-HLT '13), pp. 108–114, Atlanta, Ga, USA, 2013.

S. Rustamov, E. Mustafayev, and M. A. Clements, “An application of hidden Markov models in subjectivity analysis,” in Proceedings of the IEEE 7th International Conference on Application of Information and Communication Technologies (AICT '13), pp. 64–67, Baku, Azerbaijan, 2013.

K. Aida-Zade and S. Rustamov, “Learning user intentions in natural language call routing systems,” Studies in Fuzziness and Soft Computing, vol. 342, pp. 37–46, 2016.

S. Rustamov, E. Mustafayev, and M. A. Clements, “Sentiment analysis using Neuro-Fuzzy and Hidden Markov models of text,” in Proceedings of the IEEE Southeastcon 2013, Jacksonvilla, Fl, USA, 2013.

E. Riloff and J. Wiebe, “Learning extraction patterns for subjective expressions,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 105–112, 2003.

E. Riloff, J. Wiebe, and W. Phillips, “Exploiting subjectivity classification to improve information extraction,” in Proceedings of the 20th National Conference on Artificial Intelligence, pp. 1106–1111, 2005.

B. Pang and L. Lee, “A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts,” in Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics (ACL), 278, 271 pages, 2004.

J. G. Shanahan, Y. Qu, D. A. Evans et al., “Validating the coverage of lexical resources for affect analysis and automatically classifying new words along semantic axes,” in Proceedings of the AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications, pp. 93–107.

T. Wilson, J. Wiebe, and R. Hwa, “Just how mad are you? finding strong and weak opinion clauses,” in Proceedings of the National Conference on Artificial Intelligence, pp. 761–769, 2004.

K. Soo-Min and H. Eduard, “Automatic detection of opinion bearing words and sentences,” in Proceedings of the International Joint Conference on Natural Language Processing, pp. 61–66, 2005.

E. Riloff, S. Patwardhan, and J. Wiebe, “Feature subsumption for opinion analysis,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP '06), 2006.

S. Raaijmakers, K. Truong, and T. Wilson, “Multimodal subjectivity analysis of multiparty conversation,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 466–474, 2008.

G. Murray and G. Carenini, “Predicting subjectivity in multimodal conversations,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1348–1357, 2009.

J. Martineau and T. Finin, “Delta TFIDF: An Improved Feature Space for Sentiment Analysis,” in Proceedings of the In Proceedings of the 3rd AAAI International Conference on Weblogs and Social Media, 2009.

L. Barbosa and J. Feng, “Robust sentiment detection on twitter from biased and noisy data,” in Proceedings of the International Conference on Computational Linguistics (Coling '10), 2010.

Y. He, “Bayesian Models for Sentence-Level Subjectivity Detection,” Tech. Rep. KMI-10-02, June 2010.

F. Benamara, C. Baptiste, Y. Mathieu, and V. Popescu, “Towards context-based subjectivity analysis,” in In Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP '11), 2011.

Robert. Remus, “Improving sentence-level subjectivity classification through readability measurement,” in Proceedings of the NODALIDA-2011 Conference Proceedings, pp. 168–174, 2011.

H. Kanayama and T. Nasukawa, “Fully automatic lexicon expansion for domain-oriented sentiment analysis,” in Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pp. 355–363, 2006.

T. Zagibalov and J. Carroll, “Unsupervised classification of sentiment and objectivity in chinese text,” in Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP '08), pp. 304–311, 2008.

Z. Zhang, Q. Ye, R. Law, and Y. Li, “Automatic detection of subjective sentences based on chinese subjective patterns,” Communications in Computer and Information Science, vol. 35, pp. 29–36, 2009.

C. Banea, R. Mihalcea, J. Wiebe, and S. Hassan, “Multilingual subjectivity analysis using machine translation,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 127–135, 2008.

R. Mihalcea, C. Banea, and J. Wiebe, “Learning multilingual subjective language via cross-lingual projections,” in Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL '07), pp. 976–983, 2007.

S. Mukund and R. K. Srihari, “A vector space model for subjectivity classification in urdu aided by co-training,” in Proceedings of the 23rd International Conference on Computational Linguistics (Coling '10), pp. 860–868, 2010.

M. Abdul-Mageed, M. T. Diab, and M. Korayem, “Subjectivity and sentiment analysis of modern standard Arabic,” in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, ACL-HLT 2011, pp. 587–591, 2011.

C. Banea, R. Mihalcea, and J. Wiebe, “Multilingual subjectivity: Are more languages better?” in Proceedings of the 23rd International Conference on Computational Linguistics, Coling 2010, pp. 28–36, 2010.

A. Mogadala and V. Varma, “Language independent sentence-level subjectivity analysis with feature selection,” in Proceedings of the 26th Pacific Asia Conference on Language, Information and Computation, pp. 171–180, 2012.

O. Appel, F. Chiclana, J. Carter, and H. Fujita, “A hybrid approach to the sentiment analysis problem at the sentence level,” Knowledge-Based Systems, vol. 108, pp. 110–124, 2016.

S. Baccianella, A. Esuli, and F. Sebastiani, “SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining,” in Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010, pp. 2200–2204, 2010.

A. Muhammad, N. Wiratunga, and R. Lothian, “Contextual sentiment analysis for social media genres,” Knowledge-Based Systems, vol. 108, pp. 92–101, 2016.

M. Fernández-Gavilanes, T. Álvarez-López, J. Juncal-Martínez, E. Costa-Montenegro, and F. Javier González-Castaño, “Unsupervised method for sentiment analysis in online texts,” Expert Systems with Applications, vol. 58, pp. 57–75, 2016.

T. Chen, R. Xu, Y. He, and X. Wang, “Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN,” Expert Systems with Applications, vol. 72, pp. 221–230, 2017.

A. Reyes and P. Rosso, “On the difficulty of automatically detecting irony: beyond a simple case of negation,” Knowledge and Information Systems, vol. 40, no. 3, pp. 595–614, 2014.

A. Hogenboom, B. Heerschop, F. Frasincar, U. Kaymak, and F. De Jong, “Multi-lingual support for lexicon-based sentiment analysis guided by semantics,” Decision Support Systems, vol. 62, pp. 43–53, 2014.

E. Cambria, P. Gastaldo, F. Bisio, and R. Zunino, “An ELM-based model for affective analogical reasoning,” Neurocomputing, pp. 443–455, 2015.

M. Helmi and S. M. T. AlModarresi, “Human activity recognition using a fuzzy inference system,” in Proceedings of the 2009 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE '09), pp. 1897–1902, Jeju Island, Republic of Korea, August 2009.

K. Aida-Zade, S. Rustamov, E. Mustafayev, and N. Aliyeva, “Human-computer dialogue understanding hybrid system,” in Proceedings of the International Symposium on Innovations in Intelligent Systems and Applications (INISTA '12), Trabzon, Turkey, July 2012.

R. Fuller, Neural Fuzzy Systems, 1995.

K. R. Aida-Zade, C. Ardil, and S. S. Rustamov, “Investigation of combined use of MFCC and LPC features in speech recognition systems,” International Journal of Signal Processing, vol. 3, no. ISSN 1304-4478, pp. 74–80, 2006.

P. Blunsom, Hidden Markov Models, Lecture notes, 2004.