基于循环胶囊网络的临床语义关系识别研究

doi:10.16088/j.issn.1001-6600.2019.01.009

Abstract

Abstract: A large number of electronic health records (EHRs) have been accumulated since the wide adoption of medical information systems in China. However, most of these records are written in natural language, which cannot be processed by computers directly. Thus, it is important to transform unstructured EHRs into structured ones. In this paper, a recurrent capsule network is proposed for clinical relation extraction in EHRs, where entity pairs and their contexts are captured by piece-wise recurrent neural network layers, and capsule layers are finally employed for relation classification. Experimental results show that this model performs better than the existing supervised methods, achieving a F1-score of 96.51%.

Key words: electronic health record, relation extraction, recurrent neural network, capsule network, deep learning

CLC Number:

TP391

WANG Qi,QIU Jiahui,RUAN Tong,GAO Daqi,GAO Ju. Recurrent Capsule Network for Clinical Relation Extraction[J].Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 80-88.

References

[1] GUNTER T D,TERRY N P.The emergence of national electronic health record architectures in the United States and Australia:models,costs,and questions[J].Journal of Medical Internet Research,2005,7(1):e3.DOI:10.2196/jmir.7.1.e3.
[2] ZELENKO D,AONE C,RICHARDELLA A.Kernel methods for relation extraction[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing:Volume 10.Stroudsburg,PA:Association for Computational Linguistics, 2002:71-78.DOI:10.3115/1118693.1118703.
[3] ZHAO Shubin,GRISHMAN R.Extracting relations with integrated information using kernel methods[C]//Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2005:419-426.DOI:10.3115/1219840.1219892.
[4] SOCHER R,HUVAL B,MANNING C D,et al.Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:1201-1211.
[5] ZENG Daojian,LIU Kang,LAI Siwei,et al.Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. Stroudsburg,PA:Association for Computational Linguistics,2014: 2335-2344.
[6] NGUYEN T H,GRISHMAN R.Relation extraction: perspective from convolutional neural networks[C]//Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2015:39-48.DOI:10.3115/v1/W15-1506.
[7] ZHANG Dongxu,WANG Dong.Relation classification via recurrent neural network[EB/OL].(2015-08-05)[2018-09-27]. https://arxiv.org/abs/1508.01006.
[8] ZHOU Peng,SHI Wei,TIAN Jun,et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.2 Short Papers.Stroudsburg,PA:Association for Computational Linguistics,2016:207-212.DOI:10.18653/v1/P16-2034.
[9] GUPTA S,MANNING C D.Improved pattern learning for bootstrapped entity extraction[C]//Proceedings of the Eighteenth Conference on Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2014:98-108.DOI:10.3115/v1/W14-1611.
[10] LIN Yankai,SHEN Shiqi,LIU Zhiyuan,et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Vol.1 Long Papers. Stroudsburg,PA:Association for Computational Linguistics,2016:2124-2133.DOI:10.18653/v1/P16-1200.
[11] JI Guoliang,LIU Kang,HE Shizhu,et al.Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2017:3060-3066.
[12] SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]//Advances in Neural Information Processing Systems 30.Red Hook,NY:Curran Associates Inc,2017:3856-3866.
[13] MILLER S,FOX H,RAMSHAW L,et al.A novel use of statistical parsing to extract information from text[C]//Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference. Stroudsburg,PA:Association for Computational Linguistics,2000:226-233.
[14] KAMBHATLA N.Combining lexical,syntactic,and semantic features with maximum entropy models for extracting relations[C]//Proceedings of the ACL 2004 on Interactive Poster and Demonstration Sessions.Stroudsburg,PA: Association for Computational Linguistics,2004:22.DOI: 10.3115/1219044.1219066.
[15] CULOTTA A,McCALLUM A,BETZ J.Integrating probabilistic extraction models and data mining to discover relations and patterns in text[C]//Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics.Stroudsburg,PA:Association for Computational Linguistics,2006:296-303.DOI:10.3115/1220835.1220873.
[16] XU Yan,MOU Lili,LI Ge,et al.Classifying relations via long short term memory networks along shortest dependency paths[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA: Association for Computational Linguistics,2015:1785-1794.DOI:10.18653/v1/D15-1206.
[17] CARLSON A,BETTERIDGE J,KISIEL B,et al.Toward an architecture for never-ending language learning[C]//Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence.Palo Alto,CA:AAAI Press,2010:1306- 1313.
[18] RIEDEL S,YAO Limin,McCALLUM A.Modeling relations and their mentions without labeled text[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases:LNCS Volume 6323.Berlin:Springer,2010: 148-163.DOI:10.1007/978-3-642-15939-8_10.
[19] HOFFMANNR,ZHANG Congle,LING Xiao,et al.Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies:Volume 1.Stroudsburg,PA:Association for Computational Linguistics,2011:541-550.
[20] SURDEANU M,TIBSHIRANI J,NALLAPATI R,et al.Multi-instance multi-label learning for relation extraction[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.Stroudsburg,PA:Association for Computational Linguistics,2012:455-465.
[21] ZENG Daojian,LIU Kang,CHEN Yubo,et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2015:1753-1762.DOI:10.18653/v1/D15-1203.
[22] SINGH S,RIEDEL S,MARTIN B,et al.Joint inference of entities, relations, and coreference[C]//Proceedings of the 2013 Workshop on Automated Knowledge Base Construction.New York,NY:ACM Press,2013:1-6.DOI:10.1145/ 2509558.2509559.
[23] MIWA M,SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2014:1858-1869.DOI:10.3115/v1/D14-1200.
[24] LI Qi,JI Heng.Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics:Volume 1 Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2014:402-412.DOI:10.3115/v1/P14-1038.
[25] MIWA M,BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics:Volume 1 Long Papers.Stroudsburg,PA: Association for Computational Linguistics,2016:1105-1116.
[26] ZHENG Suncong,WANG Feng,BAO Hongyun,et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics:Volume 1 Long Papers.Stroudsburg,PA:Association for Computational Linguistics,2017:1227-1236.
[27] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.DOI: 10.1162/neco.1997.9.8.1735.
[28] MIKOLOV T,CHEN Kai,CORRADO G,et al.Efficient estimation of word representations in vector space[EB/OL]. (2013-01-16)[2018-09-27].https://arxiv.org/abs/1301.3781.
[29] KINGMA D P,BA J L.ADAM: a method for stochastic optimization[EB/OL].(2014-12-22)[2018-09-27]. https://arxiv.org/abs/1412.6980.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Recurrent Capsule Network for Clinical Relation Extraction

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 9

Metrics

Comments

Recommended 0

[1]	ZHANG Mingyu,ZHAO Meng,CAI Fuhong,LIANG Yu,WANG Xinhong. Wave Power Prediction Based on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(3): 25-32.
[2]	LI Weiyong, LIU Bin, ZHANG Wei, CHEN Yunfang. An Automatic Summarization Model Based on Deep Learning for Chinese [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 51-63.
[3]	LIU Yingxuan, WU Xiru, XUE Ganggang. Multi-target Real-time Detection for Road Traffic SignsBased on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 96-106.
[4]	ZHANG Jinlei, LUO Yuling, FU Qiang. Predicting Financial Time Series Based on Gated Recurrent Unit Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(2): 82-89.
[5]	HUANG Liming,CHEN Weizheng,YAN Hongfei,CHEN Chong. A Stock Prediction Method Based on Recurrent Neural Network and Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 13-22.
[6]	WU Wenya,CHEN Yufeng,XU Jin’an,ZHANG Yujie. High-level Semantic Attention-based Convolutional Neural Networks for Chinese Relation Extraction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 32-41.
[7]	YUE Tianchi, ZHANG Shaowu, YANG Liang, LIN Hongfei, YU Kai. Stance Detection Method Based on Two-Stage Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 42-49.
[8]	YU Chuanming,LI Haonan,AN Lu. Analysis of Text Emotion Cause Based on Multi-task Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 50-61.
[9]	ZHAO Hui-wei, LI Wen-hua, FENG Chun-hua, LUO Xiao-shu. Periodic Oscillation Analysis for a Recurrent Neural NetworksModel with Time Delays [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(1): 29-34.