广西师范大学学报（自然科学版） ›› 2019, Vol. 37 ›› Issue (1): 80-88.doi: 10.16088/j.issn.1001-6600.2019.01.009

• 第二十四届全国信息检索学术会议专栏 • 上一篇    下一篇

基于循环胶囊网络的临床语义关系识别研究

王祺1, 邱家辉1, 阮彤1, 高大启1*, 高炬2   

  1. 1.华东理工大学信息科学与工程学院,上海200237;
    2.上海中医药大学附属曙光医院,上海200021
  • 收稿日期:2018-09-27 出版日期:2019-01-20 发布日期:2019-01-08
  • 通讯作者: 高大启(1957—),男,湖北宜昌人,华东理工大学教授,博士。E-mail: gaodaqi@ecust.edu.cn
  • 基金资助:
    国家自然科学基金(61772201);“精准医学研究”重大专项(2018YFC0910500);国家重大新药创制项目(2018ZX09201008)

Recurrent Capsule Network for Clinical Relation Extraction

WANG Qi1,QIU Jiahui1,RUAN Tong1,GAO Daqi1*,GAO Ju2   

  1. 1.School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China;
    2.Shanghai Shuguang Hospital, Shanghai 200021, China
  • Received:2018-09-27 Online:2019-01-20 Published:2019-01-08

PDF (PC)

183

摘要： 得益于医疗信息化的不断推进,医院已积累了大量的电子病历记录。然而,这些病历记录大多以自然语言的形式存在,无法为计算机所“理解”,也就无法对其做进一步的处理与挖掘。由此,对病历文本进行结构化研究,识别出病历实体间的语义关系,便显得尤为重要。本文针对临床语义关系识别任务,提出循环胶囊网络模型,使用分段循环神经网络来捕捉两实体及其上下文信息,并使用胶囊网络来进行最终的关系分类。实验表明,本文提出的方法较现有监督学习方法取得了更好的识别效果(F1-score为96.51%),证明了本文方法的优越性。

关键词: 电子病历记录, 关系识别, 循环神经网络, 胶囊网络, 深度学习

Abstract: A large number of electronic health records (EHRs) have been accumulated since the wide adoption of medical information systems in China. However, most of these records are written in natural language, which cannot be processed by computers directly. Thus, it is important to transform unstructured EHRs into structured ones. In this paper, a recurrent capsule network is proposed for clinical relation extraction in EHRs, where entity pairs and their contexts are captured by piece-wise recurrent neural network layers, and capsule layers are finally employed for relation classification. Experimental results show that this model performs better than the existing supervised methods, achieving a F1-score of 96.51%.

Key words: electronic health record, relation extraction, recurrent neural network, capsule network, deep learning

中图分类号: 

  • TP391
[1] GUNTER T D,TERRY N P.The emergence of national electronic health record architectures in the United States and Australia:models,costs,and questions[J].Journal of Medical Internet Research,2005,7(1):e3.DOI:10.2196/jmir.7.1.e3.
[2] ZELENKO D,AONE C,RICHARDELLA A.Kernel methods for relation extraction[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing:Volume 10.Stroudsburg,PA:Association for Computational Linguistics, 2002:71-78.DOI:10.3115/1118693.1118703.
[3] ZHAO Shubin,GRISHMAN R.Extracting relations with integrated information using kernel methods[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2005:419-426.DOI:10.3115/1219840.1219892.
[4] SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:1201-1211.
[5] ZENG Daojian,LIU Kang,LAI Siwei,et al.Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. Stroudsburg,PA:Association for Computational Linguistics,2014: 2335-2344.
[6] NGUYEN T H,GRISHMAN R.Relation extraction: perspective from convolutional neural networks[C]//Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:39-48.DOI:10.3115/v1/W15-1506.
[7] ZHANG Dongxu,WANG Dong.Relation classification via recurrent neural network[EB/OL].(2015-08-05)[2018-09-27]. https://arxiv.org/abs/1508.01006.
[8] ZHOU Peng,SHI Wei,TIAN Jun,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.2 Short Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:207-212.DOI:10.18653/v1/P16-2034.
[9] GUPTA S,MANNING C D.Improved pattern learning for bootstrapped entity extraction[C]//Proceedings of the Eighteenth Conference on Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2014:98-108.DOI:10.3115/v1/W14-1611.
[10] LIN Yankai,SHEN Shiqi,LIU Zhiyuan,et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.1 Long Papers. Stroudsburg,PA:Association for Computational Linguistics,2016:2124-2133.DOI:10.18653/v1/P16-1200.
[11] JI Guoliang,LIU Kang,HE Shizhu,et al.Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2017:3060-3066.
[12] SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems 30.Red Hook,NY:Curran Associates Inc,2017:3856-3866.
[13] MILLER S,FOX H,RAMSHAW L,et al.A novel use of statistical parsing to extract information from text[C]//Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference. Stroudsburg,PA:Association for Computational Linguistics,2000:226-233.
[14] KAMBHATLA N.Combining lexical,syntactic,and semantic features with maximum entropy models for extracting relations[C]//Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions.Stroudsburg,PA: Association for Computational Linguistics,2004:22.DOI: 10.3115/1219044.1219066.
[15] CULOTTA A,McCALLUM A,BETZ J.Integrating probabilistic extraction models and data mining to discover relations and patterns in text[C]//Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2006:296-303.DOI:10.3115/1220835.1220873.
[16] XU Yan,MOU Lili,LI Ge,et al.Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA: Association for Computational Linguistics,2015:1785-1794.DOI:10.18653/v1/D15-1206.
[17] CARLSON A,BETTERIDGE J,KISIEL B,et al.Toward an architecture for never-ending language learning[C]//Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2010:1306- 1313.
[18] RIEDEL S,YAO Limin,McCALLUM A.Modeling relations and their mentions without labeled text[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases:LNCS Volume 6323.Berlin:Springer,2010: 148-163.DOI:10.1007/978-3-642-15939-8_10.
[19] HOFFMANNR,ZHANG Congle,LING Xiao,et al.Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies:Volume 1.Stroudsburg,PA:Association for Computational Linguistics,2011:541-550.
[20] SURDEANU M,TIBSHIRANI J,NALLAPATI R,et al.Multi-instance multi-label learning for relation extraction[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:455-465.
[21] ZENG Daojian,LIU Kang,CHEN Yubo,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2015:1753-1762.DOI:10.18653/v1/D15-1203.
[22] SINGH S,RIEDEL S,MARTIN B,et al.Joint inference of entities, relations, and coreference[C]//Proceedings of the 2013 Workshop on Automated Knowledge Base Construction.New York,NY:ACM Press,2013:1-6.DOI:10.1145/ 2509558.2509559.
[23] MIWA M,SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2014:1858-1869.DOI:10.3115/v1/D14-1200.
[24] LI Qi,JI Heng.Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics:Volume 1 Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2014:402-412.DOI:10.3115/v1/P14-1038.
[25] MIWA M,BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Volume 1 Long Papers.Stroudsburg,PA: Association for Computational Linguistics,2016:1105-1116.
[26] ZHENG Suncong,WANG Feng,BAO Hongyun,et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics:Volume 1 Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2017:1227-1236.
[27] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.DOI: 10.1162/neco.1997.9.8.1735.
[28] MIKOLOV T,CHEN Kai,CORRADO G,et al.Efficient estimation of word representations in vector space[EB/OL]. (2013-01-16)[2018-09-27].https://arxiv.org/abs/1301.3781.
[29] KINGMA D P,BA J L.ADAM: a method for stochastic optimization[EB/OL].(2014-12-22)[2018-09-27]. https://arxiv.org/abs/1412.6980.
[1] 张明宇, 赵猛, 蔡夫鸿, 梁钰, 王鑫红. 基于深度学习的波浪能发电功率预测[J]. 广西师范大学学报（自然科学版）, 2020, 38(3): 25-32.
[2] 李维勇, 柳斌, 张伟, 陈云芳. 一种基于深度学习的中文生成式自动摘要方法[J]. 广西师范大学学报（自然科学版）, 2020, 38(2): 51-63.
[3] 刘英璇, 伍锡如, 雪刚刚. 基于深度学习的道路交通标志多目标实时检测[J]. 广西师范大学学报（自然科学版）, 2020, 38(2): 96-106.
[4] 张金磊, 罗玉玲, 付强. 基于门控循环单元神经网络的金融时间序列预测[J]. 广西师范大学学报（自然科学版）, 2019, 37(2): 82-89.
[5] 黄丽明, 陈维政, 闫宏飞, 陈翀. 基于循环神经网络和深度学习的股票预测方法[J]. 广西师范大学学报（自然科学版）, 2019, 37(1): 13-22.
[6] 岳天驰, 张绍武, 杨亮, 林鸿飞, 于凯. 基于两阶段注意力机制的立场检测方法[J]. 广西师范大学学报（自然科学版）, 2019, 37(1): 42-49.
[7] 余传明, 李浩男, 安璐. 基于多任务深度学习的文本情感原因分析[J]. 广西师范大学学报（自然科学版）, 2019, 37(1): 50-61.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发