多类型知识增强的微博立场检测模型

doi:10.16088/j.issn.1001-6600.2023050808

摘要/Abstract

摘要： 针对文本立场检测中目标话题在微博文本中隐式出现以及文本语义隐含表达这2个核心问题,本文提出一种基于多类型知识增强与预训练语言模型相结合的立场检测新方法KE-BERT。该模型同时从知识图谱和百度百科中引入多类型的相关常识知识来弥补语义缺失,使用改进的预训练语言模型BERT作为编码器,然后通过卷积注意力机制对常识知识进行融合与聚焦,最后通过Softmax分类获得立场。该模型在NLPCC-2016语料库上实验的宏平均F₁值达到0.803,分类性能超越现有主流模型,验证了模型的有效性。

关键词: 立场检测, 知识增强, BERT, 卷积神经网络, 注意力机制

Abstract: In response to the two core issues of implicit appearance of target topics in Weibo texts and implicit expression of text semantics in text position detection, a new position detection method, KE-BERT, is proposed in this paper, based on the combination of multi-type knowledge enhancement and pre-trained language models. Therefore, KE-BERT can overcome semantic deficiencies. The model incorporates relevant commonsense knowledge from knowledge graphs and Baidu encyclopedia, utilizes the enhanced pre-trained language model BERT as the encoder, and employs convolution attention mechanisms to combine and focus commonsense knowledge. Finally, the stance is determined through softmax classification. Experimental results on the NLPCC-2016 corpus demonstrate the effectiveness of the model, achieving a macro average F₁-score of 0.803. KE-BERT outperforms the other existing methods in classification performance.

Key words: stance detection, knowledge enhancement, BERT, CNN, attention

中图分类号: TP391.1

王天雨, 袁嘉伟, 齐芮, 李洋. 多类型知识增强的微博立场检测模型[J]. 广西师范大学学报（自然科学版）, 2024, 42(1): 79-90.

WANG Tianyu, YUAN Jiawei, QI Rui, LI Yang. Multi-type Knowledge-Enhanced Microblog Stance Detection Model[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 79-90.

参考文献

[1] ALDAYEL A, MAGDY W. Stance detection on social media: state of the art and trends[J]. Information Processing & Management, 2021, 58(4): 102597. DOI: 10.1016/j.ipm.2021.102597.
[2] SUN Y Q, LI Y. Stance detection with knowledge enhanced BERT[C]// Artificial Intelligence. Cham: Springer, 2021: 239-250. DOI: 10.1007/978-3-030-93049-3_20.
[3] HE Z H, MOKHBERIAN N, LERMAN K. Infusing knowledge from Wikipedia to enhance stance detection[C]// Proceedings of the 12th Workshop on Computational Approaches to Subjectivity, Sentiment & Social Media Analysis. Stroudsburg, PA: Association for Computational Linguistics, 2022: 71-77. DOI: 10.18653/v1/2022.wassa-1.7.
[4] 李洋, 孙宇晴, 景维鹏. 文本立场检测综述[J]. 计算机研究与发展, 2021, 58(11): 2538-2557. DOI: 10.7544/issn1000-1239.2021.20200518.
[5] SUN Q Y, WANG Z Q, ZHU Q M, et al. Exploring various linguistic features for stance detection[C]// Natural Language Understanding and Intelligent Applications. Cham: Springer, 2016: 840-847. DOI: 10.1007/978-3-319-50496-4_76.
[6] 奠雨洁, 金琴, 吴慧敏. 基于多文本特征融合的中文微博的立场检测[J]. 计算机工程与应用, 2017, 53(21): 77-84. DOI: 10.3778/j.issn.1002-8331.1702-0292.
[7] VIJAYARAGHAVAN P, SYSOEV I, VOSOUGHI S, et al. DeepStance at SemEval-2016 task 6: detecting stance in tweets using character and word-level CNNs[C]// Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). Stroudsburg, PA: Association for Computational Linguistics, 2016: 413-419. DOI: 10.18653/v1/S16-1067.
[8] TAULÉ M, RANGEL F, MARTÍ M A, et al. Overview of the task on multimodal stance detection in tweets on Catalan #1Oct Referendum[C]// Proceedings of the Third Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval 2018). Sevilla: CEUR-WS, 2018: 149-166.
[9] ZARRELLA G, MARSH A. MITRE at SemEval-2016 task 6: transfer learning for stance detection[C]// Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016). Stroudsburg, PA: Association for Computational Linguistics, 2016: 458-463. DOI: 10.18653/v1/S16-1074.
[10] AUGENSTEIN I, ROCKTÄSCHEL T, VLACHOS A, et al. Stance detection with bidirectional conditional encoding[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2016: 876-885. DOI: 10.18653/v1/D16-1084.
[11] 胡慧君, 易洋, 施琦, 等. 基于细粒度信息交互注意力的情绪分类方法[J]. 武汉大学学报(理学版), 2023, 69(3): 400-408. DOI: 10.14188/j.1671-8836.2021.0360.
[12] DU J C, XU R F, HE Y L, et al. Stance classification with target-specific neural attention networks[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17). Melbourne: International Joint Conferences on Artificial Intelligence Organization, 2017: 3988-3994. DOI: 10.24963/ijcai.2017/557.
[13] 岳天驰, 张绍武, 杨亮, 等. 基于两阶段注意力机制的立场检测方法[J]. 广西师范大学学报(自然科学版), 2019, 37(1): 42-49. DOI: 10.16088/j.issn.1001-6600.2019.01.005.
[14] 颜瑶. 结合主题目标信息的社交媒体文本立场分析[D]. 哈尔滨: 哈尔滨工业大学, 2017. DOI: 10.7666/d.D01590498.
[15] SUN Q Y, WANG Z Q, ZHU Q M, et al. Stance detection with hierarchical attention network[C]// Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 2399-2409.
[16] YU N, PAN D, ZHANG M S, et al. Stance detection in Chinese MicroBlogs with neural networks[C]// Natural Language Understanding and Intelligent Applications. Cham: Springer, 2016: 893-900. DOI: 10.1007/978-3-319-50496-4_83.
[17] 白静. 基于注意力机制的中文微博立场检测模型[D]. 武汉: 武汉大学, 2017.
[18] 杨顺成, 李彦, 赵其峰. 基于GCN和Bi-LSTM的微博立场检测方法[J]. 重庆理工大学学报(自然科学), 2020, 34(6): 167-173. DOI: 10.3969/j.issn.1674-8425(z).2020.06.024.
[19] ILIĆ S, MARRESE-TAYLOR E, BALAZS J, et al. Deep contextualized word representations for detecting sarcasm and irony[C]// Proceedings of the 9th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. Stroudsburg, PA: Association for Computational Linguistics, 2018: 2-7. DOI: 10.18653/v1/W18-6202.
[20] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. (2018-06-11)[2023-05-08]. https://cdn.openai.com/research-covers/language-unsupervised/language_under-standing_paper.pdf.
[21] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186. DOI: 10.18653/v1/N19-1423.
[22] LIU Y H, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2023-05-08]. https://arxiv.org/abs/1907.11692. DOI: 10.48550/arXiv.1907.11692.
[23] 胡慧君, 杨雨烟, 易洋, 等. 基于细粒度信息感知BERT-EEP的情绪分类方法[J]. 计算机工程与科学, 2023, 45(4): 751-760. DOI: 10.3969/j.issn.1007-130X.2023.04.023.
[24] 王安君, 黄凯凯, 陆黎明. 基于Bert-Condition-CNN的中文微博立场检测[J]. 计算机系统应用, 2019, 28(11): 45-53. DOI: 10.15888/j.cnki.csa.007152.
[25] VRANDEIĆ D, KRÖTZSCH M. Wikidata: a free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85. DOI: 10.1145/2629489.
[26] LIU H, SINGH P. ConceptNet:a practical commonsense reasoning tool-kit[J]. BT Technology Journal, 2004, 22(4): 211-226. DOI: 10.1023/B:BTTJ.0000047600.45421.6d.
[27] XU B, XU Y, LIANG J Q, et al. CN-DBpedia: a never-ending Chinese knowledge extraction system[C]// Advances in Artificial Intelligence: from Theory to Practice. Cham: Springer, 2017: 428-438. DOI: 10.1007/978-3-319-60045-1_44.
[28] ZHANG Z Y, HAN X, LIU Z Y, et al. ERNIE: enhanced language representation with informative entities[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 1441-1451. DOI: 10.18653/v1/P19-1139.
[29] WANG X Z, GAO T Y, ZHU Z C, et al. KEPLER: a unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021, 9: 176-194. DOI: 10.1162/tacl_a_00360.
[30] SUN T X, SHAO Y F, QIU X P, et al. CoLAKE: contextualized language and knowledge embedding[C]// Proceedings of the 28th International Conference on Computational Linguistics. Barcelona: International Committee on Computational Linguistics, 2020: 3660-3670. DOI: 10.18653/v1/2020.coling-main.327.
[31] LIU W J, ZHOU P, ZHAO Z, et al. K-BERT: enabling language representation with knowledge graph[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(3): 2901-2908. DOI: 10.1609/aaai.v34i03.5681.
[32] XU R F, ZHOU Y, WU D Y, et al. Overview of NLPCC shared task 4: stance detection in Chinese microblogs[C]// Natural Language Understanding and Intelligent Applications. Cham: Springer, 2016: 907-916. DOI: 10.1007/978-3-319-50496-4_85.
[33] 张翠肖, 郝杰辉, 刘星宇, 等. 基于CNN-BiLSTM的中文微博立场分析研究[J]. 计算机技术与发展, 2020, 30(7): 154-159. DOI: 10.3969/j.issn.1673-629X.2020.07.033.
[34] 耿源羚, 张绍武, 张益嘉, 等. 基于卷积注意力的情感增强微博立场检测[J]. 山西大学学报(自然科学版), 2022, 45(2): 302-312. DOI: 10.13451/j.sxu.ns.2021105.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed