广西师范大学学报（自然科学版） ›› 2019, Vol. 37 ›› Issue (1): 32-41.doi: 10.16088/j.issn.1001-6600.2019.01.004

• 第二十四届全国信息检索学术会议专栏 • 上一篇    下一篇

基于高层语义注意力机制的中文实体关系抽取

武文雅, 陈钰枫*, 徐金安, 张玉洁   

  1. 北京交通大学计算机与信息技术学院,北京100044
  • 收稿日期:2018-09-27 出版日期:2019-01-20 发布日期:2019-01-08
  • 通讯作者: 陈钰枫(1981—),女,福建南平人,北京交通大学副教授,博士。E-mail:chenyf@bjtu.edu.cn
  • 基金资助:
    国家自然科学基金(61473294,61370130);北京市自然基金(4172047);中央高校基本科研业务费专项资金(2015JBM033)

High-level Semantic Attention-based Convolutional Neural Networks for Chinese Relation Extraction

WU Wenya,CHEN Yufeng*,XU Jin’an,ZHANG Yujie   

  1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China
  • Received:2018-09-27 Online:2019-01-20 Published:2019-01-08

PDF (PC)

276

摘要： 实体关系抽取在挖掘结构化事实的信息抽取系统中扮演着重要的角色。近年来,深度学习在关系抽取任务中取得了显著的成果,同时,注意力机制也逐步地融入到神经网络中,进一步提高了关系抽取的性能。但是,目前的注意力机制主要关注一些低层次的特征,比如词汇等。本文提出一种基于高层语义注意力机制的分段卷积神经网络模型(PCNN_HSATT,high-level semantic attention-based piecewise convolutional neural networks),该模型将注意力机制设置在分段最大池化层后,动态地关注了高层次的语义信息。除此之外,由于中文实体关系语料稀疏性较大,本文利用同义词词林对COAE2016语料进行增强以扩大数据规模。最后在COAE2016和ACE2005的中文语料上进行实验,F1值分别达到了78.41%和73.94%,与效果最好的SVM方法相比分别提高了10.45%和0.67%,这充分证明了PCNN_HSATT模型在中文关系抽取上的有效性。

关键词: 关系抽取, 卷积神经网络, 注意力机制, 数据增广, 依存句法约束

Abstract: Relation extraction is an important part of many information extraction systems that mines structured facts from texts. Recently, deep learning has achieved good results in relation extraction. Attention mechanism is also gradually applied to networks, which improves the performance of the task. However, the current attention mechanism is mainly applied to the basic features on the lexical level rather than the higher overall features. In order to obtain more information of high-level features for relation predicting, this paper proposes high-level semantic attention-based piecewise convolutional neural networks (PCNN_HSATT), which adds an attention layer after the piecewise max pooling layer in order to get significant information of sentence global features. Furthermore, this paper puts forward a data augmentation method by utilizing an external dictionary HIT IR-Lab Tongyici Cilin (Extended) to reply the sparse challenge in Chinese entity relation extraction corpus. Experimental results on COAE2016 and ACE2005 Chinese datasets are 78.41% and 73.94% respectively. Compared with the best existing method SVM, the results improve 10.45% and 0.67% respectively, which demonstrates that this approach is effective in Chinese entity relation extraction task.

Key words: relation extraction, convolutional neural networks, attention mechanism, data augmentation, dependency syntax constraint

中图分类号: 

  • TP391
[1] ZENG D, LIU K, LAI S, et al. Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics. Stroudsburg, PA: ACM, 2014: 2335-2344.
[2] XU K, FENG Y S, HUANG S F, et al. Semantic relation classification via convolutional neural networks with simple negative sampling[J]. Computer Science, 2015, 71(7):941-949.
[3] DOS SANTOS C N, XIANG B, ZHOU B. Classifying relations by ranking with convolutional neural networks[C]//Proceeding of Annual Meeting of the Association for computational Linguistics. Stroudsburg, PA:ACL, 2015:626-634.
[4] HASHIMOTO K, MIWA M, TSURUOKA Y, et al. Simple customization of recursive neural networks for semantic relation classification[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2013:1372-1376.
[5] ZHOU P, SHI W, TIAN J, et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016:207-212.
[6] LIU Y, WEI F, LI S, et al. A dependency-based neural network for relation classification[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2015: 285-290.
[7] CAI R, ZHANG X, WANG H, et al. Bidirectional recurrent convolutional neural network for relation classification[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016:756-765.
[8] WANG L, CAO Z, MELO G D, et al. Relation classification via multi-level attention CNNs[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016:1298-1307.
[9] ZENG D, LIU K, CHEN Y, et al. Distant supervision for relation extraction via piecewise convolutional neural[C]//Proceedings of Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2015:17-21.
[10] SUN J, GU X, LI Y, et al. Chinese entity relation extraction algorithms based on COAE2016 datasets[J]. Journal of Shandong University(Natural Science Edition), 2017, 52(9): 7-12.
[11] 车万翔,刘挺,李生. 实体关系自动抽取[J]. 中文信息学报,2004,19(2):1-6.
[12] 黄鑫,朱巧明,钱龙华,等.基于特征组合的中文实体关系抽取[J].微电子学与计算机,2010,27(4):198-200.
[13] ZHAO S, GRISHMAN R. Extracting relations with integrated information using kernel methods[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2005: 419-426.
[14] BUNESCU R C, MOONEY R J. A shortest path dependency kernel for relation extraction[C]//Proceedings of Human Language Technology Conference/Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2005:724-731.
[15] SOCHER R, HUVAL B, MANNING C D,et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Stroudsburg, PA: ACL, 2012:1201-1211.
[16] CHENG F, MIYAO Y. Classifying temporal relations by bidirectional LSTM over dependency paths[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2017:1-6.
[17] RINK B, HARABAGIU S. UTD: Classifying semantic relations by combining lexical and semantic resources[C]//Proceedings of the 5th International Workshop on Semantic Evaluation. Stroudsburg, PA: ACL, 2010:256-259.
[18] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493-2537.
[19] LI S, XU J A, ZHANG Y, et al. A method of unknown words processing for neural machine translation using HowNet[C]//Proceedings of the 13th China Workshop on Machine Translation. Berlin: Springer, 2017:22-29.
[20] LIU Q, LI S. Word similarity computing based on HowNet[J]. Computational Linguistics and Chinese Language Processing, 2002, 7(2):59-76.
[1] 李维勇, 柳斌, 张伟, 陈云芳. 一种基于深度学习的中文生成式自动摘要方法[J]. 广西师范大学学报（自然科学版）, 2020, 38(2): 51-63.
[2] 严浩, 许洪波, 沈英汉, 程学旗. 开放式中文事件检测研究[J]. 广西师范大学学报（自然科学版）, 2020, 38(2): 64-71.
[3] 王健, 郑七凡, 李超, 石晶. 基于ENCODER_ATT机制的远程监督关系抽取[J]. 广西师范大学学报（自然科学版）, 2019, 37(4): 53-60.
[4] 范瑞,蒋品群,曾上游,夏海英,廖志贤,李鹏. 多尺度并行融合的轻量级卷积神经网络设计[J]. 广西师范大学学报（自然科学版）, 2019, 37(3): 50-59.
[5] 岳天驰, 张绍武, 杨亮, 林鸿飞, 于凯. 基于两阶段注意力机制的立场检测方法[J]. 广西师范大学学报（自然科学版）, 2019, 37(1): 42-49.
[6] 薛洋,曾庆科,夏海英,王文涛. 基于卷积神经网络超分辨率重建的遥感图像融合[J]. 广西师范大学学报（自然科学版）, 2018, 36(2): 33-41.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发