A Rule Extraction Study from SVM on Sentiment Analysis
AbstractA natural way to determine the knowledge embedded within connectionist models is to generate symbolic rules. Nevertheless, extracting rules from Multi Layer Perceptrons (MLPs) is NP-hard. With the advent of social networks, techniques applied to Sentiment Analysis show a growing interest, but rule extraction from connectionist models in this context has been rarely performed because of the very high dimensionality of the input space. To fill the gap we present a case study on rule extraction from ensembles of Neural Networks and Support Vector Machines (SVMs), the purpose being the characterization of the complexity of the rules on two particular Sentiment Analysis problems. Our rule extraction method is based on a special Multi Layer Perceptron architecture for which axis-parallel hyperplanes are precisely located. Two datasets representing movie reviews are transformed into Bag-of-Words vectors and learned by ensembles of neural networks and SVMs. Generated rules from ensembles of MLPs are less accurate and less complex than those extracted from SVMs. Moreover, a clear trade-off appears between rules’ accuracy, complexity and covering. For instance, if rules are too complex, less complex rules can be re-extracted by sacrificing to some extent their accuracy. Finally, rules can be viewed as feature detectors in which very often only one word must be present and a longer list of words must be absent. View Full-Text
Share & Cite This Article
Bologna, G.; Hayashi, Y. A Rule Extraction Study from SVM on Sentiment Analysis. Big Data Cogn. Comput. 2018, 2, 6.
Bologna G, Hayashi Y. A Rule Extraction Study from SVM on Sentiment Analysis. Big Data and Cognitive Computing. 2018; 2(1):6.Chicago/Turabian Style
Bologna, Guido; Hayashi, Yoichi. 2018. "A Rule Extraction Study from SVM on Sentiment Analysis." Big Data Cogn. Comput. 2, no. 1: 6.
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.