2017 AutomaticQuestionAnsweringUsing

(Minaee & Liu, 2017) ⇒ Shervin Minaee, and Zhu Liu. (2017). “Automatic Question-Answering Using A Deep Similarity Neural Network.” In: Proceedings of 2017_IEEE Global Conference on Signal and Information Processing (GlobalSIP). doi:10.1109/GlobalSIP.2017.8309095

Subject Headings: Deep Semantic Similarity Neural Network, Deep Similarity Neural Network QA System, QA System.

Notes

Cited By

Quotes

Abstract

Automatic question-answering is a classical problem in natural language processing, which aims at designing systems that can automatically answer a question, in the same way as human does. In this work, we propose a deep learning based model for automatic question-answering. First the questions and answers are embedded using neural probabilistic modeling. Then a deep similarity neural network is trained to find the similarity score of a pair of answer and question. Then for each question, the best answer is found as the one with the highest similarity score. We first train this model on a large-scale public question-answering database, and then fine-tune it to transfer to the customer-care chat data. We have also tested our framework on a public question-answering database and achieved very good performance

1. Introduction

2. Question And Answer Embedding

3. Deep Similarity Network

After extracting features we need to train a model which takes a pair of question and answer, and outputs a score that shows the properness of that answer for the given question. There are different ways to achieve this goal. In a very simple way one could concatenate the doc2vec features of question and answer and train a classifier on top of that which predicts the probability of matching. In this work, inspired by Siamese network by Lecun and colleagues 22-23, we propose a deep similarity network that takes the features for a pair of question and answer and feed them into two parallel neural networks, and combines them after a few layers of transformation to make decision. The block diagram of this model is shown in Figure 4.

**Figure 4:** The block-diagram of the proposed similarity network.

(...)

4. Experimental Results

5. Conclusion

Acknowledgments

We would like to thank Kyunghyun Cho at NYU for his valuable comments and suggestions regarding this work. We also thank Minwei Feng from IBM Watson for providing InsuranceQA corpus (https://github.com/shuzi/insuranceQA).

Figures

.

References

1. B.F. Green, A.K. Wolf, C. Chomsky, K. Laughery, "Base-ball: an automatic question-answerer", western joint IRE-AIEE-ACM computer conference, 1961.
2. W.A. Woods, R. Kaplan, "Lunar rocks in natural English: Explorations in natural language question answering", Linguistic structures processing, vol. 5, no. 5, pp. 521569, 1977.
3. K. Bollacker, C. Evans, P. Paritosh, T. Sturge, J. Taylor, "Freebase: a collaboratively created graph database for structuring human knowledge", Proceedings of the ACM SIGMOD International Conference on Management of data, 2008.
4. C.D. Manning, H. Schutze, Foundations of statistical natural language processing, Cambridge:MIT press, vol. 999, 1999.
5. M. Iyyer, J.L. Boyd-Graber, LMB Claudino, R. Socher, I.H. Daume, "A Neural Network for Factoid Question Answering over Paragraphs", Conference on Empirical Methods on Natural Language Processing, 2014.
6. J. Weston, S. Chopra, A. Bordes, Memory networks, 2014.
7. A. Bordes, J. Weston, N. Usunier, "Open question answering with weakly supervised embedding models", Proceedings of ECML, 2014.
8. N. Kalchbrenner, E. Grefenstette, P. Blunsom, "A convolutional neural network for modelling sentences", Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, June, 2014.
9. M. Feng, B. Xiang, M.R. Glass, L. Wang, B. Zhou, "Applying deep learning to answer selection: A study and an open task", IEEE Workshop on Automatic Speech Recognition and Understanding, 2015.
10. S. Hochreiter, J. Schmidhuber, "Long short-term memory", Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
11. M. Tan, B. Xiang, B. Zhou, LSTM-based Deep Learning Models for non-factoid answer selection, 2015.
12. G. Hinton, J.L. McClelland, D.E. Rumelhart, "Distributed representations" in Parallel distributed processing: Explorations in the microstructure of cognition Foundations, MIT Press, 1986.
13. Y. Bengio, R. Ducharme, P. Vincent, C. Jauvin, "A neural probabilistic language model", journal of machine learning research, pp. 1137-1155, 2003.
14. T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean, "Distributed representations of words and phrases and their compositionality", Advances in Neural Information Processing Systems, 2013.
15. T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of word representations in vector space, 2013.
16. Q.V. Le, T. Mikolov, "Distributed Representations of Sentences and Documents", ICML, pp. 1188-1196, 2014.
17. B. Hu, Z. Lu, H. Li, Q. Chen, "Convolutional neural network architectures for matching natural language sentences", Advances in Neural Information Processing Systems, pp. 2042-2050, 2014.
18. A. Severyn, A. Moschitti, "Learning to rank short text pairs with convolutional deep neural networks", International ACM SIGIR Conference on Research and Development in Information Retrieval, 2015.
19. H. He, K. Gimpel, J. Lin, "Multi-perspective sentence similarity modeling with convolutional neural networks", Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1576-1586, 2015.
20. H. Zhao, Z. Lu, P. Poupart, "Self-adaptive hierarchical sentence model", arXiv preprint arXiv, 2015.
21. R. Kiros, Y. Zhu, R.R. Salakhutdinov, R. Zemel, R. Urtasun, A. Torralba, S. Fidler, "Skip-thought vectors", Advances in Neural Information Processing Systems, pp. 3294-3302, 2015.
22. S Chopra, R Hadsell, Y LeCun, “Learning a similarity metric discriminatively, with application to face verification”, Conference on Computer Vision and Pattern Recognition, IEEE, 2005.
23. [[1993_SignatureVerificationUsingaSiam|J. Bromley, J.W. Bentz, L. Bottou, I. Guyon, Y. LeCun, C. Moore, E. Sackinger, R. Shah, "Signature verification using a Siamese time delay neural network", International Journal of Pattern Recognition and Artificial Intelligence, 1993.]].
24. R. Rehurek, P. Sojka, "Software framework for topic modelling with large corpora", Proceedings of the LREC Workshop on New Challenges for NLP Frameworks, 2010.
25. Abadi Martn et al., Tensorflow: Large-scale machine learning on heterogeneous distributed systems, 2016.
26. M. Feng, B. Xiang, M.R. Glass, L. Wang, B. Zhou, "Applying deep learning to answer selection: A study and an open task", IEEE Workshop on Automatic Speech Recognition and Understanding, 2015.
27. https://github.com/shuzi/insuranceQA.

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2017 AutomaticQuestionAnsweringUsing	Shervin Minaee Zhu Liu			Automatic Question Answering Using A Deep Similarity Neural Network				10.1109/GlobalSIP.2017.8309095		2017