Future Work

In future, a new loss function that is more appropriate to both ATE and APC tasks should be investigated. In addition, handling subjective sentences and negation that cause most of the errors in the current model should be explored to improve the performance of the APC task. Moreover, using convolutional neural network for the ATEPC task is also a promising direction because it performs well in the ATE task in the previous work.

Finally, because the contextual attention visualization showed the relation between aspect and opinion terms, we believe that it is totally suitable for the task of aspect and opinion terms co-extraction (AOTE). An AOTE model with the contextual attention mechanism should be implemented and empirically evaluated.

Bibliography

[1] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine transla-tion by jointly learning to align and translate. CoRR, abs/1409.0473, 2014.

[2] Y. Bengio, P. Simard, and P. Frasconi. Learning long-term dependencies with gradi-ent descgradi-ent is difficult. Trans. Neur. Netw., 5(2):157–166, March 1994.

[3] Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 452–461. Association for Computational Linguistics, 2017.

[4] Kyunghyun Cho, Bart van Merrienboer, C¸ aglar G¨ul¸cehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. CoRR, abs/1406.1078, 2014.

[5] Ronan Collobert, Jason Weston, L´eon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural language processing (almost) from scratch. J. Mach.

Learn. Res., 12:2493–2537, November 2011.

[6] Franck Dernoncourt, Ji Young Lee, and Peter Szolovits. NeuroNER: an easy-to-use program for named-entity recognition based on neural networks. Conference on Empirical Methods on Natural Language Processing (EMNLP), 2017.

[7] Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Lin-guistics (Volume 2: Short Papers), pages 49–54, Baltimore, Maryland, June 2014.

Association for Computational Linguistics.

[8] Alex Graves and Jrgen Schmidhuber. Framewise phoneme classification with bidirec-tional lstm and other neural network architectures. NEURAL NETWORKS, pages 5–6, 2005.

[9] Ruidan He, Wee Sun Lee, Hwee Tou Ng, and Daniel Dahlmeier. An unsupervised neural attention model for aspect extraction. InProceedings of the 55th Annual Meet-ing of the Association for Computational LMeet-inguistics (Volume 1: Long Papers), pages 388–397, Vancouver, Canada, July 2017. Association for Computational Linguistics.

[10] Sepp Hochreiter and J¨urgen Schmidhuber. Long short-term memory. Neural Com-put., 9(8):1735–1780, November 1997.

[11] Minqing Hu and Bing Liu. Mining and summarizing customer reviews. InProceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, pages 168–177, New York, NY, USA, 2004. ACM.

[12] Zhiheng Huang, Wei Xu, and Kai Yu. Bidirectional LSTM-CRF models for sequence tagging. CoRR, abs/1508.01991, 2015.

[13] Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT ’11, pages 151–160, Stroudsburg, PA, USA, 2011. Association for Computa-tional Linguistics.

[14] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization.

CoRR, abs/1412.6980, 2014.

[15] John F. Kolen and Stefan C. Kremer. Gradient Flow in Recurrent Nets: The Diffi-culty of Learning LongTerm Dependencies. IEEE, 2001.

[16] Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Com-putational Linguistics: Human Language Technologies, pages 260–270, San Diego, California, June 2016. Association for Computational Linguistics.

[17] Chenghua Lin and Yulan He. Joint sentiment/topic model for sentiment analysis. In Proceedings of the 18th ACM Conference on Information and Knowledge Manage-ment, CIKM ’09, pages 375–384, New York, NY, USA, 2009. ACM.

[18] Bing Liu. Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers, 2012.

[19] Bing Liu and Lei Zhang. A Survey of Opinion Mining and Sentiment Analysis, pages 415–463. Springer US, Boston, MA, 2012.

[20] Julian McAuley and Alex Yang. Addressing complex and subjective product-related queries with customer reviews. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, pages 625–635, Republic and Canton of Geneva, Switzerland, 2016. International World Wide Web Conferences Steering Committee.

[21] Qiaozhu Mei, Xu Ling, Matthew Wondra, Hang Su, and ChengXiang Zhai. Topic sentiment mixture: Modeling facets and opinions in weblogs. In Proceedings of the 16th International Conference on World Wide Web, WWW ’07, pages 171–180, New York, NY, USA, 2007. ACM.

[22] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781, 2013.

[23] Volodymyr Mnih, Nicolas Heess, Alex Graves, and Koray Kavukcuoglu. Recurrent models of visual attention. CoRR, abs/1406.6247, 2014.

[24] Thien Hai Nguyen and Kiyoaki Shirai. PhraseRNN: Phrase recursive neural network for aspect-based sentiment analysis. In Proceedings of the 2015 Conference on Em-pirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pages 2509–2514, 2015.

[25] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532–1543. ACL, 2014.

[26] Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphee De Clercq, Veronique Hoste, Marianna Apidianaki, Xavier Tannier, Na-talia Loukachevitch, Evgeniy Kotelnikov, N´uria Bel, Salud Mar´ıa Jim´enez-Zafra, and G¨ul¸sen Eryi˘git. Semeval-2016 task 5: Aspect based sentiment analysis. InProceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 19–30, San Diego, California, June 2016. Association for Computational Linguistics.

[27] Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. Semeval-2015 task 12: Aspect based sentiment analysis. In Pro-ceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 486–495, Denver, Colorado, June 2015. Association for Computational Lin-guistics.

[28] Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion An-droutsopoulos, and Suresh Manandhar. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35, Dublin, Ireland, August 2014. Association for Com-putational Linguistics and Dublin City University.

[29] Ana-Maria Popescu and Oren Etzioni. Extracting product features and opinions from reviews. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT ’05, pages 339–346, Stroudsburg, PA, USA, 2005. Association for Computational Linguistics.

[30] Soujanya Poria, Erik Cambria, and Alexander Gelbukh. Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems, 108:42 – 49, 2016. New Avenues in Knowledge Bases for Natural Language Processing.

[31] Alexander M. Rush, Sumit Chopra, and Jason Weston. A neural attention model for abstractive sentence summarization. CoRR, abs/1509.00685, 2015.

[32] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. Recursive deep models for semantic com-positionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–1642, Stroudsburg, PA, October 2013. Association for Computational Linguistics.

[33] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting.

J. Mach. Learn. Res., 15(1):1929–1958, January 2014.

[34] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, pages 3104–3112, Cambridge, MA, USA, 2014. MIT Press.

[35] Charles Sutton and Andrew McCallum. An introduction to conditional random fields.

Found. Trends Mach. Learn., 4(4):267–373, April 2012.

[36] Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. Effective lstms for target-dependent sentiment classification. InCOLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, De-cember 11-16, 2016, Osaka, Japan, pages 3298–3307, 2016.

[37] Duyu Tang, Bing Qin, and Ting Liu. Aspect level sentiment classification with deep memory network. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 214–224, 2016.

[38] Ivan Titov and Ryan McDonald. Modeling online reviews with multi-grain topic models. In Proceedings of the 17th International Conference on World Wide Web, WWW ’08, pages 111–120, New York, NY, USA, 2008. ACM.

[39] Joachim Wagner, Piyush Arora, Santiago Cortes, Utsab Barman, Dasha Bogdanova, Jennifer Foster, and Lamia Tounsi. Dcu: Aspect-based polarity classification for semeval task 4. In Proceedings of the 8th International Workshop on Semantic Eval-uation (SemEval 2014), pages 223–229, Dublin, Ireland, August 2014. Association for Computational Linguistics and Dublin City University.

[40] Wenya Wang, Sinno Jialin Pan, Daniel Dahlmeie, and Xiaokui Xiao. Coupled multi-layer attentions for co-extraction of aspect and opinion terms. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA., pages 3316–3322, 2017.

[41] Wenya Wang, Sinno Jialin Pan, Daniel Dahlmeier, and Xiaokui Xiao. Recursive neu-ral conditional random fields for aspect-based sentiment analysis. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 616–626, 2016.