Journal of Guangxi Normal University(Natural Science Edition) ›› 2022, Vol. 40 ›› Issue (3): 40-48.doi: 10.16088/j.issn.1001-6600.2021091503

Previous Articles     Next Articles

Identification of Adverse Drug Reaction on Social Media Using Bi-directional Language Model

LI Zhengguang1, CHEN Heng1*, LIN Hongfei2   

  1. 1. Research Center for Language Intelligence, Dalian University of Foreign Language, Dalian Liaoning 116044, China;
    2. School of Computer Science and Technology, Dalian University of Technology, Dalian Liaoning 116024, China
  • Received:2021-09-15 Revised:2021-12-20 Online:2022-05-25 Published:2022-05-27

PDF (PC)

367

Abstract: More time-effective and wider adverse drug reactions are concealed in tweets related to feelings of taking medication. However, it is difficult to extract adverse drug reaction (ADR) from these tweets due to relatively shortness and sparseness of tweets. Therefore, a neural network model is proposed in this paper, which employes the pretrained bidirectional language model and attention mechanism to identify ADR. Firstly, specific character-level features are extracted via a pretrained bidirectional character-level neural language model. Secondly, the attention mechanism is used to capture local and global semantic contexts while extracting ADRs. Thirdly, to improve the efficiency of the proposed method, Character-level features are combined with word-level features. Finally, co-training is replaced with the pretrained of the whole-word level and fine-tuned pretrained character embeddings. These optimizations contribute to improving the performance of identification. The proposed model achieves better performance on the PSB 2016 Social Media Mining Sharing Task Workshop-Task 2: ADR Extraction, obtaining the F1-scores of 82.2% on official datasets. Character features are useful for distinguishing ADR and non-ADR in morphology. In addition, attention mechanism improves the performance of identifying ADR due to capturing local and global semantic contexts.

Key words: adverse drug reaction, social media, bi-directional language model, attention mechanism, pretrained model

CLC Number: 

  • TP391.1
[1]SARKER A, BELOUSOV M, FRIEDRICHS J, et al. Data and systems for medication-related text classification and concept normalization from Twitter: insights from the Social Media Mining for Health(SMM4H)-2017 shared task[J]. Journal of the American Medical Informatics Association, 2018, 25(10): 1274-1283. DOI: 10.1093/jamia/ocy114.
[2]朱晓旭,林鸿飞,曾泽渊. 基于社交媒体的药物不良反应检测[J]. 山西大学学报(自然科学版), 2020, 43(1): 14-21.
[3]BENTON A, UNGAR L, HILL S, et al. Identifying potential adverse effects using the web: a new approach to medical hypothesis generation[J]. Journal of Biomedical Informatics, 2011, 44(6): 989-996.
[4]张亚飞, 于琦, 王于心, 等. 基于药物论坛中潜在不良反应与适应症的知识发现体系构建[J]. 中华医学图书情报杂志, 2020, 29(7): 38-43.
[5]ZHANG Y, CUI S, GAO H. Adverse drug reaction detection on social media with deep linguistic features[J]. Journal of Biomedical Informatics, 2020, 106: 103437.
[6]许力, 李建华. 基于BERT和BiLSTM-CRF的生物医学命名实体识别[J]. 计算机工程与科学, 2021, 43(10): 1873-1879.
[7]LUO L, YANG Z, YANG P, et al. An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition[J]. Bioinformatics. 2018, 34(8): 1381-1388.
[8]ZHANG T, LIN H, REN Y, et al. Identifying adverse drug reaction entities from social media with adversarial transfer learning model[J]. Neurocomputing, 2021, 45: 254-262.
[9]LI Z, YANG Z, WANG L, et al. Lexicon knowledge boosted interaction graph network for adverse drug reaction recognition from social media[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 25(7): 2777-2786.
[10]佘朝阳, 严馨, 徐广义, 等. 基于数据增强和半监督学习的药物不良反应检测[J/OL]. 计算机工程[2021-10-15]. https://doi.org/10.19678/j.issn.1000-3428.0062170.
[11]SAHU S K, ANAND A. Recurrent neural network models for disease name recognition using domain invariant features[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2016: 2216-2225. DOI: 10.18653/v1/P16-1209.
[12]JAGANNATHA A, YU H. Structured prediction models for RNN based sequence labeling in clinical text[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2016: 856-865. DOI: 10.18653/v1/D16-1082.
[13]PANDEY C, IBRAHIM Z, WU H H, et al. Improving RNN with attention and embedding for adverse drug reactions[C]// Proceedings of the 2017 International Conference on Digital Health. New York, NY: Association for Computing Machinery, 2017: 67-71. DOI: 10.1145/3079452.3079501.
[14]PENG Y F, YAN S K, LU Z Y. Transfer learning in biomedical natural language processing: an evaluation of BERT and ELMo on ten benchmarking datasets[C]// Proceedings of the 18th BioNLP Workshop and Shared Task. Stroudsburg, PA: Association for Computational Linguistics, 2019: 58-65. DOI: 10.18653/v1/W19-5006.
[15]申晨, 林鸿飞. 基于图嵌入的社交媒体药物不良反应事件检测方法[J]. 大连理工大学学报, 2020, 60(5): 547-554.
[16]宋雅文, 杨志豪, 罗凌, 等. 基于字符卷积神经网络的生物医学变异实体识别方法[J]. 中文信息学报, 2021, 35(5): 63-69.
[17]SARABADANI S. Detection of adverse drug reaction mentions in tweets using ELMo[C]// Proceedings of the 4th Social Media Mining for Health Applications(#SMM4H) Workshop & Shared Task. Stroudsburg, PA: Association for Computational Linguistics, 2019: 120-122. DOI: 10.18653/v1/W19-3221.
[18]SRIVASTAVA R K, GREFF K, SCHMIDHUBER J. Highway networks[EB/OL].(2015-11-03)[2021-09-15].http:// arxiv.org/abs/1505.00387. DOI: 10.48550/arXiv.1505.00387.
[19]DEWI I N, 蔡晓玲, 刘晓锋, 等. 结合类别关键词与注意力机制的药物相互关系抽取模型[J]. 华南理工大学学报(自然科学版), 2021, 49(1): 10-17.
[20]YANG Z C, YANG D Y, DYER C, et al. Hierarchical attention networks for document classification[C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2016: 1480-1489. DOI: 10.18653/v1/N16-1174.
[21]魏巍, 傅维刚. 面向社交媒体的细粒度ADR本体的半自动构建方法研究[J]. 图书情报工作, 2019, 63(3): 108-114.
[22]COCOS A, FIKS A G, MASINO A J. Deep learning for pharmacovigilance: Recurrent neural network architectures for labeling adverse drug reactions in Twitter posts[J]. Journal of the American Medical Informatics Association, 2017, 24(4): 813-821. DOI: 10.1093/jamia/ocw180.
[23]NIKFARJAM A, SARKER A, O’CONNOR K, et al. Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features[J]. Journal of the American Medical Informatics Association, 2015, 22(3): 671-681. DOI: 10.1093/jamia/ocu041.
[24]NIKFARJAM A, GONZALEZ G H. Pattern mining for extraction of mentions of adverse drug reactions from user comments[J]. AMIA Annual Symposium Proceedings, 2011, 2011: 1019-1026.
[25]LAI S, LIU K, XU L, et al. How to generate a good word embedding?[J]. IEEE Intelligent Systems, 2016, 31(6): 5-14.
[26]DUCHI J, HAZAN E, SINGER Y. Adaptive Subgradient methods for online learning and stochastic optimization[J]. Journal of Machine Learning Research, 2011, 12: 2121-2159.
[27]CHOWDHURY S, ZHANG C W, YU P S. Multi-Task Pharmacovigilance Mining from Social Media Posts[C]// Proceedings of the 2018 World Wide Web Conference. Geneva, Switzerland: International World Wide Web Conferences Steering Committee, 2018: 117-126. DOI: 10.1145/3178876.3186053.
[1] WAN Liming, ZHANG Xiaoqian, LIU Zhigui, SONG Lin, ZHOU Ying, LI Li. CT Image Segmentation of UNet Pulmonary Nodules Based on Efficient Channel Attention [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 66-75.
[2] ZHANG Ping, XU Qiaozhi. Segmentation of Lung Nodules Based on Multi-receptive Field and Grouping Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 76-87.
[3] WU Jun, OUYANG Aijia, ZHANG Lin. Phosphorylation Site Prediction Model Based on Multi-head Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 161-171.
[4] LI Weiyong, LIU Bin, ZHANG Wei, CHEN Yunfang. An Automatic Summarization Model Based on Deep Learning for Chinese [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 51-63.
[5] WANG Jian, ZHENG Qifan, LI Chao, SHI Jing. Remote Supervision Relationship Extraction Based on Encoder and Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(4): 53-60.
[6] WU Wenya,CHEN Yufeng,XU Jin’an,ZHANG Yujie. High-level Semantic Attention-based Convolutional Neural Networks for Chinese Relation Extraction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 32-41.
[7] YUE Tianchi, ZHANG Shaowu, YANG Liang, LIN Hongfei, YU Kai. Stance Detection Method Based on Two-Stage Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 42-49.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] AI Yan, JIA Nan, WANG Yuan, GUO Jing, PAN Dongdong. Review of Statistical Methods and Applications of Genetic Association Analysis for Multiple Traits and Multiple Locus[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 1 -14 .
[2] BAI Defa, XU Xin, WANG Guochang. Review of Generalized Linear Models and Classification for Functional Data[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 15 -29 .
[3] ZENG Qingfan, QIN Yongsong, LI Yufang. Empirical Likelihood Inference for a Class of Spatial Panel Data Models[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 30 -42 .
[4] ZHANG Zhifei, DUAN Qian, LIU Naijia, HUANG Lei. High-dimensional Nonlinear Regression Model Based on JMI[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 43 -56 .
[5] YANG Di, FANG Yangxin, ZHOU Yan. New Category Classification Research Based on MEB and SVM Methods[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 57 -67 .
[6] CHEN Zhongxiu, ZHANG Xingfa, XIONG Qiang, SONG Zefang. Estimation and Test for Asymmetric DAR Model[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(1): 68 -81 .
[7] DU Jinfeng, WANG Hairong, LIANG Huan, WANG Dong. Progress of Cross-modal Retrieval Methods Based on Representation Learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 1 -12 .
[8] LI Muhang, HAN Meng, CHEN Zhiqiang, WU Hongxin, ZHANG Xilong. Survey of Algorithms Oriented to Complex High Utility Pattern Mining[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 13 -30 .
[9] CHAO Rui, ZHANG Kunli, WANG Jiajia, HU Bin, ZHANG Weicong, HAN Yingjie, ZAN Hongying. Construction of Chinese Multimodal Knowledge Base[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 31 -39 .
[10] ZHOU Shengkai, FU Lizhen, SONG Wen’ai. Semantic Similarity Computing Model for Short Text Based on Deep Learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 49 -56 .
Copyright © Journal of Guangxi Normal University(Natural Science Edition), All Rights Reserved.
Tel: 0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
Powered by Beijing Magtech Co. Ltd