广西师范大学学报（自然科学版） ›› 2025, Vol. 43 ›› Issue (3): 12-22.doi: 10.16088/j.issn.1001-6600.2024092804

• CCIR2024 • 上一篇    下一篇

基于掩码注意力与多特征卷积网络的关系抽取方法

卢展跃1,2,3, 陈艳平1,2,3*, 杨卫哲1,2,3, 黄瑞章1,2,3, 秦永彬1,2,3   

  1. 1.贵州大学文本计算与认知智能教育部工程研究中心, 贵州贵阳 550025;
    2.公共大数据国家重点实验室(贵州大学), 贵州贵阳 550025;
    3.贵州大学计算机科学与技术学院, 贵州贵阳 550025
  • 收稿日期:2024-09-28 修回日期:2024-12-20 出版日期:2025-05-05 发布日期:2025-05-14
  • 通讯作者: 陈艳平(1980—), 男, 贵州长顺人, 贵州大学教授, 博士。E-mail: ypench@gmail.com
  • 基金资助:
    贵州省科学技术基金重点项目(〔2024〕003); 国家重点研发计划(2023YFC3304500); 国家自然科学基金(62166007)

Relational Extraction Method Based on Mask Attention and Multi-feature Convolutional Networks

LU Zhanyue1,2,3, CHEN Yanping1,2,3*, YANG Weizhe1,2,3, HUANG Ruizhang1,2,3, QIN Yongbin1,2,3   

  1. 1. Text Computing and Cognitive Intelligence Engineering Research Center of the Ministry of Education, Guizhou University, Guiyang Guizhou 550025, China;
    2. State Key Laboratory of Public Big Data (Guizhou University), Guiyang Guizhou 550025, China;
    3. College of Computer Science and Technology, Guizhou University, Guiyang Guizhou 550025, China
  • Received:2024-09-28 Revised:2024-12-20 Online:2025-05-05 Published:2025-05-14

PDF (PC)

236

摘要： 关系抽取旨在抽取2个命名实体之间的语义关系。近年来,提示学习通过拼接提示模板并进行掩码预测的方式,统一了预训练语言模型训练和微调过程的优化目标,在关系抽取领域取得优异的性能。然而,固定的提示模板与关系实例间的弱语义关联导致模型对复杂关系的语义感知能力差。针对这一问题,本文提出一种基于掩码注意力与多特征卷积网络的关系抽取方法。该方法采用三仿射注意力机制将提示模板中的掩码与关系实例的语义空间进行交互映射,形成二维掩码语义,并利用多特征卷积网络和多层感知机提取二维掩码语义中的关系信息。该方法通过建立掩码与关系实例间的显式语义依赖,增强提示模型对复杂关系的语义感知能力。该方法在数据集SemEval、SciERC和CLTC上的F1值分别达到91.4%、 91.2%和82.6%,验证了该方法的有效性。

关键词: 自然语言处理, 关系抽取, 提示学习, 三仿射注意力机制, 卷积神经网络

Abstract: Relation extraction aims to extract the semantic relationship between two named entities. Recently, prompt learning has unified the optimization objectives of pre-trained language models and fine-tuning by concatenating prompt templates and performing mask prediction, achieving excellent performance in the field of relationship extraction. However, weak semantic associations between fixed prompt templates and relation instances are observed, which limits the model’s ability to perceive complex relationships. To address this issue, a relation extraction method based on mask attention and multi-feature convolution networks is proposed. The tri-affine attention mechanism is adopted to interactively map the mask in the prompt template with the semantic space of the original text. Two-dimensional mask semantics are then formed through this process. Multi-feature convolutional networks and multi-layer perceptron are employed to extract relational information from the two-dimensional mask semantics. Explicit semantic dependencies between the mask and the relation instance are established. This approach enhances the semantic perception of complex relationships in prompt models. Performances of 91.4%, 91.2%, and 82.6% are achieved on the SemEval, SciERC, and CLTC datasets, respectively. The effectiveness of the proposed method is demonstrated by experimental results.

Key words: natural language processing, relation extraction, prompt learning, tri-affine attention mechanism, convolutional neural network

中图分类号:  TP391.1

[1] 谢德鹏, 常青. 关系抽取综述[J]. 计算机应用研究, 2020, 37(7): 1921-1924, 1930. DOI: 10.19734/j.issn.1001-3695.2018.12.0923.
[2]黄勋, 游宏梁, 于洋. 关系抽取技术研究综述[J]. 现代图书情报技术, 2013(11): 30-39. DOI: 10.11925/infotech.1003-3513.2013.11.05.
[3]DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186. DOI: 10.18653/v1/N19-1423.
[4]LIU Y H, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2024-09-28]. https://arxiv.org/abs/1907.11692. DOI: 10.48550/arXiv.1907.11692.
[5]车万翔, 刘挺, 李生. 实体关系自动抽取[J]. 中文信息学报, 2005, 19(2): 1-6. DOI: 10.3969/j.issn.1003-0077.2005.02.001.
[6]SOARES L B, FITZGERALD N, LING J, et al. Matching the blanks: distributional similarity for relation learning[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 2895-2905. DOI: 10.18653/v1/P19-1279.
[7]ZHOU W X, CHEN M H. An improved baseline for sentence-level relation extraction[C]// Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 161-168. DOI: 10.18653/v1/2022.aacl-short.21.
[8]WU S C, HE Y F. Enriching pre-trained language model with entity information for relation classification[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: Association for Computing Machinery, 2019: 2361-2364. DOI: 10.1145/3357384.3358119.
[9]BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook, NY: Curran Associates Inc., 2020: 1877-1901.
[10]HAN X, ZHAO W L, DING N, et al. PTR: prompt tuning with rules for text classification[J]. AI Open, 2022, 3: 182-192. DOI: 10.1016/j.aiopen.2022.11.003.
[11]SCHICK T, SCHÜTZE H. It’s not just size that matters: small language models are also few-shot learners[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2021: 2339-2352. DOI: 10.18653/v1/2021.naacl-main.185.
[12]CHEN X, ZHANG N Y, XIE X, et al. KnowPrompt: knowledge-aware prompt-tuning with synergistic optimization for relation extraction[C]// Proceedings of the ACM Web Conference 2022. New York: Association for Computing Machinery, 2022: 2778-2788. DOI: 10.1145/3485447.3511998.
[13]闫雄, 段跃兴, 张泽华. 采用自注意力机制和CNN融合的实体关系抽取[J]. 计算机工程与科学, 2020, 42(11): 2059-2066. DOI: 10.3969/j.issn.1007-130X.2020.11.019.
[14]宋睿, 陈鑫, 洪宇, 等. 基于卷积循环神经网络的关系抽取[J]. 中文信息学报, 2019, 33(10): 64-72. DOI: 10.3969/j.issn.1003-0077.2019.10.008.
[15]MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2016:1105-1116. DOI: 10.18653/v1/P16-1105.
[16]冯建周, 宋沙沙, 王元卓, 等. 基于改进注意力机制的实体关系抽取方法[J]. 电子学报, 2019, 47(8): 1692-1700. DOI: 10.3969/j.issn.1003-0077.2019.10.008.
[17]VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. (2023-08-02)[2024-09-28]. https://arxiv.org/abs/1706.03762. DOI: 10.48550/arXiv.1706.03762.
[18]阳小华, 张硕望, 欧阳纯萍. 中文关系抽取技术研究[J]. 南华大学学报(自然科学版), 2018, 32(1): 66-72. DOI: 10.3969/j.issn.1673-0062.2018.01.013.
[19]武小平, 张强, 赵芳, 等. 基于BERT的心血管医疗指南实体关系抽取方法[J]. 计算机应用, 2021, 41(1): 145-149. DOI: 10.11772/j.issn.1001-9081.2020061008.
[20]CHEN Y P, WANG K, YANG W Z, et al. A multi-channel deep neural network for relation extraction[J]. IEEE Access, 2020, 8: 13195-13203. DOI: 10.1109/ACCESS.2020.2966303.
[21]ZHAO K, XU H, CHENG Y, et al. Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction[J]. Knowledge-Based Systems, 2021, 219: 106888. DOI: 10.1016/j.knosys.2021.106888.
[22]LI J C, KATSIS Y, BALDWIN T, et al. SPOT: knowledge-enhanced language representations for information extraction[C]// Proceedings of the 31st ACM International Conference on Information & Knowledge Management. Stroudsburg, PA: Association for Computational Linguistics, 2022: 1124-1134. DOI: 10.1145/3511808.3557459.
[23]PETERS M E, NEUMANN M, LOGAN R, et al. Knowledge enhanced contextual Word representations[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Stroudsburg, PA: Association for Computational Linguistics, 2019: 43-54. DOI: 10.18653/v1/D19-1005.
[24]文坤建, 陈艳平, 黄瑞章, 等. 基于提示学习的生物医学关系抽取方法[J]. 计算机科学, 2023, 50(10): 223-229. DOI: 10.11896/jsjkx.220900108.
[25]魏超, 陈艳平, 王凯, 等. 基于掩码提示与门控记忆网络校准的关系抽取方法[J]. 计算机应用, 2024, 44(6): 1713-1719. DOI: 10.11772/j.issn.1001-9081.2023060818.
[26]HENDRICKX I, KIM S N, KOZAREVA Z, et al. SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals[C]// Proceedings of the 5th International Workshop on Semantic Evaluation. Stroudsburg, PA: Association for Computational Linguistics, 2010: 33-38.
[27]LUAN Y, HE L H, OSTENDORF M, et al. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2018: 3219-3232. DOI: 10.18653/v1/D18-1360.
[28]XU J J, WEN J, SUN X, et al. A discourse-level named entity recognition and relation extraction dataset for Chinese literature text[EB/OL]. (2019-06-11)[2024-09-28]. https://arxiv.org/abs/1711.07010. DOI: 10.48550/arXiv.1711.07010.
[29]TIAN Y H, CHEN G M, SONG Y, et al. Dependency-driven relation extraction with attentive graph convolutional networks[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2021:4458-4471. DOI: 10.18653/v1/2021.acl-long.344.
[30]QIN Y B, YANG W Z, WANG K, et al. Entity relation extraction based on entity indicators[J]. Symmetry, 2021, 13(4): 539. DOI: 10.3390/sym13040539.
[31]WANG K, CHEN Y P, WEN K J, et al. Cue prompt adapting model for relation extraction[J]. Connection Science, 2023, 35(1): 2161478. DOI: 10.1080/09540091.2022.2161478.
[32]TOUVRON H, LAVRIL T, IZACARD G, et al. LLaMA: open and efficient foundation language models[EB/OL]. (2023-02-27)[2024-09-28]. https://arxiv.org/abs/2302.13971. DOI: 10.48550/arXiv.2302.13971.
[33]LI B, YU D Y, YE W, et al. Sequence generation with label augmentation for relation extraction[C]// Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. Menlo Park, CA: AAAI Press, 2023: 13043-13050. DOI: 10.1609/aaai.v37i11.26532.
[34]ZHAO Q H, GAO T H, GUO N. A novel Chinese relation extraction method using polysemy rethinking mechanism[J]. Applied Intelligence, 2023, 53(7): 7665-7676. DOI: 10.1007/s10489-022-03817-5.
[1] 何安康, 陈艳平, 扈应, 黄瑞章, 秦永彬. 融合边界交互信息的命名实体识别方法[J]. 广西师范大学学报（自然科学版）, 2025, 43(3): 1-11.
[2] 黄仁慧, 张锐锋, 文晓浩, 闭金杰, 黄守麟, 李廷会. 基于复数协方差卷积神经网络的运动想象脑电信号解码方法[J]. 广西师范大学学报（自然科学版）, 2025, 43(3): 43-56.
[3] 王天雨, 袁嘉伟, 齐芮, 李洋. 多类型知识增强的微博立场检测模型[J]. 广西师范大学学报（自然科学版）, 2024, 42(1): 79-90.
[4] 吴正清, 曹晖, 刘宝锴. 基于注意力卷积神经网络的中文虚假评论检测[J]. 广西师范大学学报（自然科学版）, 2023, 41(5): 26-36.
[5] 唐侯清, 辛斌斌, 朱虹谕, 乙加伟, 张冬冬, 武新章, 双丰. 基于多尺度注意力倒残差网络的轴承故障诊断[J]. 广西师范大学学报（自然科学版）, 2023, 41(4): 109-122.
[6] 潘海明, 陈庆锋, 邱杰, 何乃旭, 刘春雨, 杜晓敬. 基于卷积推理的多跳知识图谱问答算法[J]. 广西师范大学学报（自然科学版）, 2023, 41(1): 102-112.
[7] 张涛, 杜建民. 基于无人机遥感的荒漠草原微斑块识别研究[J]. 广西师范大学学报（自然科学版）, 2022, 40(6): 50-58.
[8] 郝雅茹, 董力, 许可, 李先贤. 预训练语言模型的可解释性研究进展[J]. 广西师范大学学报（自然科学版）, 2022, 40(5): 59-71.
[9] 田晟, 宋霖. 基于CNN和Bagging集成的交通标志识别[J]. 广西师范大学学报（自然科学版）, 2022, 40(4): 35-46.
[10] 周圣凯, 富丽贞, 宋文爱. 基于深度学习的短文本语义相似度计算模型[J]. 广西师范大学学报（自然科学版）, 2022, 40(3): 49-56.
[11] 彭涛, 唐经, 何凯, 胡新荣, 刘军平, 何儒汉. 基于多步态特征融合的情感识别[J]. 广西师范大学学报（自然科学版）, 2022, 40(3): 104-111.
[12] 马铖旭, 曾上游, 赵俊博, 陈红阳. 基于卷积神经网络的逆光图像增强研究[J]. 广西师范大学学报（自然科学版）, 2022, 40(2): 81-90.
[13] 陈文康, 陆声链, 刘冰浩, 李帼, 刘晓宇, 陈明. 基于改进YOLOv4的果园柑橘检测方法研究[J]. 广西师范大学学报（自然科学版）, 2021, 39(5): 134-146.
[14] 杨州, 范意兴, 朱小飞, 郭嘉丰, 王越. 神经信息检索模型建模因素综述[J]. 广西师范大学学报（自然科学版）, 2021, 39(2): 1-12.
[15] 邓文轩, 杨航, 靳婷. 基于注意力机制的图像分类降维方法[J]. 广西师范大学学报（自然科学版）, 2021, 39(2): 32-40.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发