Journal of Guangxi Normal University(Natural Science Edition) ›› 2024, Vol. 42 ›› Issue (1): 79-90.doi: 10.16088/j.issn.1001-6600.2023050808

Previous Articles     Next Articles

Multi-type Knowledge-Enhanced Microblog Stance Detection Model

WANG Tianyu, YUAN Jiawei, QI Rui, LI Yang*   

  1. College of Computer and Control Engineering, Northeast Forestry University, Harbin Heilongjiang 150040, China
  • Received:2023-05-08 Revised:2023-08-25 Online:2024-01-25 Published:2024-01-19

Abstract: In response to the two core issues of implicit appearance of target topics in Weibo texts and implicit expression of text semantics in text position detection, a new position detection method, KE-BERT, is proposed in this paper, based on the combination of multi-type knowledge enhancement and pre-trained language models. Therefore, KE-BERT can overcome semantic deficiencies. The model incorporates relevant commonsense knowledge from knowledge graphs and Baidu encyclopedia, utilizes the enhanced pre-trained language model BERT as the encoder, and employs convolution attention mechanisms to combine and focus commonsense knowledge. Finally, the stance is determined through softmax classification. Experimental results on the NLPCC-2016 corpus demonstrate the effectiveness of the model, achieving a macro average F1-score of 0.803. KE-BERT outperforms the other existing methods in classification performance.

Key words: stance detection, knowledge enhancement, BERT, CNN, attention

CLC Number:  TP391.1
[1] ALDAYEL A, MAGDY W. Stance detection on social media: state of the art and trends[J]. Information Processing & Management, 2021, 58(4): 102597. DOI: 10.1016/j.ipm.2021.102597.
[2] SUN Y Q, LI Y. Stance detection with knowledge enhanced BERT[C]// Artificial Intelligence. Cham: Springer, 2021: 239-250. DOI: 10.1007/978-3-030-93049-3_20.
[3] HE Z H, MOKHBERIAN N, LERMAN K. Infusing knowledge from Wikipedia to enhance stance detection[C]// Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis. Stroudsburg, PA: Association for Computational Linguistics, 2022: 71-77. DOI: 10.18653/v1/2022.wassa-1.7.
[4] 李洋, 孙宇晴, 景维鹏. 文本立场检测综述[J]. 计算机研究与发展, 2021, 58(11): 2538-2557. DOI: 10.7544/issn1000-1239.2021.20200518.
[5] SUN Q Y, WANG Z Q, ZHU Q M, et al. Exploring various linguistic features for stance detection[C]// Natural Language Understanding and Intelligent Applications. Cham: Springer, 2016: 840-847. DOI: 10.1007/978-3-319-50496-4_76.
[6] 奠雨洁, 金琴, 吴慧敏. 基于多文本特征融合的中文微博的立场检测[J]. 计算机工程与应用, 2017, 53(21): 77-84. DOI: 10.3778/j.issn.1002-8331.1702-0292.
[7] VIJAYARAGHAVAN P, SYSOEV I, VOSOUGHI S, et al. DeepStance at SemEval-2016 task 6: detecting stance in tweets using character and word-level CNNs[C]// Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). Stroudsburg, PA: Association for Computational Linguistics, 2016: 413-419. DOI: 10.18653/v1/S16-1067.
[8] TAULÉ M, RANGEL F, MARTÍ M A, et al. Overview of the task on multimodal stance detection in tweets on Catalan #1Oct Referendum[C]// Proceedings of the Third Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval 2018). Sevilla: CEUR-WS, 2018: 149-166.
[9] ZARRELLA G, MARSH A. MITRE at SemEval-2016 task 6: transfer learning for stance detection[C]// Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). Stroudsburg, PA: Association for Computational Linguistics, 2016: 458-463. DOI: 10.18653/v1/S16-1074.
[10] AUGENSTEIN I, ROCKTÄSCHEL T, VLACHOS A, et al. Stance detection with bidirectional conditional encoding[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2016: 876-885. DOI: 10.18653/v1/D16-1084.
[11] 胡慧君, 易洋, 施琦, 等. 基于细粒度信息交互注意力的情绪分类方法[J]. 武汉大学学报(理学版), 2023, 69(3): 400-408. DOI: 10.14188/j.1671-8836.2021.0360.
[12] DU J C, XU R F, HE Y L, et al. Stance classification with target-specific neural attention networks[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17). Melbourne: International Joint Conferences on Artificial Intelligence Organization, 2017: 3988-3994. DOI: 10.24963/ijcai.2017/557.
[13] 岳天驰, 张绍武, 杨亮, 等. 基于两阶段注意力机制的立场检测方法[J]. 广西师范大学学报(自然科学版), 2019, 37(1): 42-49. DOI: 10.16088/j.issn.1001-6600.2019.01.005.
[14] 颜瑶. 结合主题目标信息的社交媒体文本立场分析[D]. 哈尔滨: 哈尔滨工业大学, 2017. DOI: 10.7666/d.D01590498.
[15] SUN Q Y, WANG Z Q, ZHU Q M, et al. Stance detection with hierarchical attention network[C]// Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 2399-2409.
[16] YU N, PAN D, ZHANG M S, et al. Stance detection in Chinese MicroBlogs with neural networks[C]// Natural Language Understanding and Intelligent Applications. Cham: Springer, 2016: 893-900. DOI: 10.1007/978-3-319-50496-4_83.
[17] 白静. 基于注意力机制的中文微博立场检测模型[D]. 武汉: 武汉大学, 2017.
[18] 杨顺成, 李彦, 赵其峰. 基于GCN和Bi-LSTM的微博立场检测方法[J]. 重庆理工大学学报(自然科学), 2020, 34(6): 167-173. DOI: 10.3969/j.issn.1674-8425(z).2020.06.024.
[19] ILIĆ S, MARRESE-TAYLOR E, BALAZS J, et al. Deep contextualized word representations for detecting sarcasm and irony[C]// Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. Stroudsburg, PA: Association for Computational Linguistics, 2018: 2-7. DOI: 10.18653/v1/W18-6202.
[20] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. (2018-06-11)[2023-05-08]. https://cdn.openai.com/research-covers/language-unsupervised/language_under-standing_paper.pdf.
[21] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186. DOI: 10.18653/v1/N19-1423.
[22] LIU Y H, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2023-05-08]. https://arxiv.org/abs/1907.11692. DOI: 10.48550/arXiv.1907.11692.
[23] 胡慧君, 杨雨烟, 易洋, 等. 基于细粒度信息感知BERT-EEP的情绪分类方法[J]. 计算机工程与科学, 2023, 45(4): 751-760. DOI: 10.3969/j.issn.1007-130X.2023.04.023.
[24] 王安君, 黄凯凯, 陆黎明. 基于Bert-Condition-CNN的中文微博立场检测[J]. 计算机系统应用, 2019, 28(11): 45-53. DOI: 10.15888/j.cnki.csa.007152.
[25] VRANDEIĆ D, KRÖTZSCH M. Wikidata: a free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85. DOI: 10.1145/2629489.
[26] LIU H, SINGH P. ConceptNet:a practical commonsense reasoning tool-kit[J]. BT Technology Journal, 2004, 22(4): 211-226. DOI: 10.1023/B:BTTJ.0000047600.45421.6d.
[27] XU B, XU Y, LIANG J Q, et al. CN-DBpedia: a never-ending Chinese knowledge extraction system[C]// Advances in Artificial Intelligence: from Theory to Practice. Cham: Springer, 2017: 428-438. DOI: 10.1007/978-3-319-60045-1_44.
[28] ZHANG Z Y, HAN X, LIU Z Y, et al. ERNIE: enhanced language representation with informative entities[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 1441-1451. DOI: 10.18653/v1/P19-1139.
[29] WANG X Z, GAO T Y, ZHU Z C, et al. KEPLER: a unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021, 9: 176-194. DOI: 10.1162/tacl_a_00360.
[30] SUN T X, SHAO Y F, QIU X P, et al. CoLAKE: contextualized language and knowledge embedding[C]// Proceedings of the 28th International Conference on Computational Linguistics. Barcelona: International Committee on Computational Linguistics, 2020: 3660-3670. DOI: 10.18653/v1/2020.coling-main.327.
[31] LIU W J, ZHOU P, ZHAO Z, et al. K-BERT: enabling language representation with knowledge graph[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(3): 2901-2908. DOI: 10.1609/aaai.v34i03.5681.
[32] XU R F, ZHOU Y, WU D Y, et al. Overview of NLPCC shared task 4: stance detection in Chinese microblogs[C]// Natural Language Understanding and Intelligent Applications. Cham: Springer, 2016: 907-916. DOI: 10.1007/978-3-319-50496-4_85.
[33] 张翠肖, 郝杰辉, 刘星宇, 等. 基于CNN-BiLSTM的中文微博立场分析研究[J]. 计算机技术与发展, 2020, 30(7): 154-159. DOI: 10.3969/j.issn.1673-629X.2020.07.033.
[34] 耿源羚, 张绍武, 张益嘉, 等. 基于卷积注意力的情感增强微博立场检测[J]. 山西大学学报(自然科学版), 2022, 45(2): 302-312. DOI: 10.13451/j.sxu.ns.2021105.
[1] XIAO Yuting, LÜ Xiaoqi, GU Yu, LIU Chuanqiang. Classification of Diabetic Retinopathy Based on Split Residual Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 91-101.
[2] XI Lingfei, Yilihamu Yaermaimaiti, LIU Yajie. Surface Defect Detection Method for Aluminum Profile Based on Improved YOLOv5 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 111-119.
[3] WANG Yongjie, GAO Xinghui, FANG Mengkai. Strong Convergence Theorem for Pseudomonotonic Variational Inequality Solution Sets and Quasi-non-expansionary Mapping Fixed Point Sets Common Elements [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 128-138.
[4] LIN Wancong, HAN Mingjie, JIN Ting. Multi-level Argument Position Classification Method via Data Augmentation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 62-69.
[5] SONG Guanwu, CHEN Zhiming, LI Jianjun. Remote Sensing Image Classification with Cascade Attention Based on ResNet-50 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 80-91.
[6] GUO Jialiang, JIN Ting. Semantic Enhancement-Based Multimodal Sentiment Analysis [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 14-25.
[7] WU Zhengqing, CAO Hui, LIU Baokai. Chinese Fake Review Detection Based on Attention Convolutional Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 26-36.
[8] LIANG Zhengyou, CAI Junmin, SUN Yu, CHEN Lei. Point Cloud Classification Based on Residual Dynamic Graph Convolution and Feature Enhancement [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 37-48.
[9] TANG Houqing, XIN Binbin, ZHU Hongyu, YI Jiawei, ZHANG Dongdong, WU Xinzhang, SHUANG Feng. Rolling Bearing Fault Diagnosis Based on Multi-scale Attentional Inverted Residual Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(4): 109-122.
[10] HUANG Yeqi, WANG Mingwei, YAN Rui, LEI Tao. Surface Quality Detection of Diamond Wire Based on Improved YOLOv5 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(4): 123-134.
[11] DENG Xizhen, JIANG Ming, CEN Mingcan, LUO Yuling. Ransomware Classification Based on Entropy Image Static Analysis Technology [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(3): 91-104.
[12] WEI Mingjun, ZHOU Taiyu, JI Zhanlin, ZHANG Xinnan. Detection Method of Mask Wearing in Public Places Based on YOLOv3 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(1): 76-86.
[13] WANG Li’e, WANG Yihui, LI Xianxian. A Multi-source Data Fusion and Privacy Protection Method of POI Recommendation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(1): 87-101.
[14] LI Zhixin, SU Qiang. Knowledge-aided Image Captioning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(5): 418-432.
[15] CAI Likun, WU Yunbing, CHEN Ganlin, LIU Chongling, LIAO Xiangwen. Category Text Generation Based on Generative Adversarial Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(4): 79-90.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LUO Yun-yan, LI Rong-zheng, LI Bing, DING Chen-xu. Optimization of Extraction Process of Alkaloids from Meconopsis horridula Hook. f. & Thomson by Response Surface Methodology[J]. Journal of Guangxi Normal University(Natural Science Edition), 2014, 32(4): 84 -90 .
[2] DONG Shulong, MA Jiangming, XIN Wenjie. Research Progress and Trend of Landscape Visual Evaluation —Knowledge Atlas Analysis Based on CiteSpace[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 1 -13 .
[3] GUO Jialiang, JIN Ting. Semantic Enhancement-Based Multimodal Sentiment Analysis[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 14 -25 .
[4] WU Zhengqing, CAO Hui, LIU Baokai. Chinese Fake Review Detection Based on Attention Convolutional Neural Network[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 26 -36 .
[5] LIANG Zhengyou, CAI Junmin, SUN Yu, CHEN Lei. Point Cloud Classification Based on Residual Dynamic Graph Convolution and Feature Enhancement[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 37 -48 .
[6] OUYANG Shuxin, WANG Mingjun, RONG Chuitian, SUN Huabo. Anomaly Detection of Multidimensional QAR Data Based on Improved LSTM[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 49 -60 .
[7] LI Yiyang, ZENG Caibin, HUANG Zaitang. Random Attractors for Chemostat Model with Wall Attachment Driven by Fractional Brownian Motion[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 61 -68 .
[8] LI Pengbo, LI Yongxiang. Radial Symmetric Solutions of p-Laplace Equations on Exterior Domains[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 69 -75 .
[9] WU Zixian, CHENG Jun, FU Jianling, ZHOU Xinwen, XIE Jialong, NING Quan. Analysis of PI-based Event-Triggered Control Design for Semi-Markovian Power Systems[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 76 -85 .
[10] CHENG Lei, YAN Puxuan, DU Bohao, YE Si, ZOU Huahong. Thermal Stability and Dielectric Relaxation of MOF-2 Synthesized in Aqueous Phase[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 86 -95 .