Journal of Guangxi Normal University(Natural Science Edition) ›› 2025, Vol. 43 ›› Issue (3): 12-22.doi: 10.16088/j.issn.1001-6600.2024092804

• CCIR2024 • Previous Articles     Next Articles

Relational Extraction Method Based on Mask Attention and Multi-feature Convolutional Networks

LU Zhanyue1,2,3, CHEN Yanping1,2,3*, YANG Weizhe1,2,3, HUANG Ruizhang1,2,3, QIN Yongbin1,2,3   

  1. 1. Text Computing and Cognitive Intelligence Engineering Research Center of the Ministry of Education, Guizhou University, Guiyang Guizhou 550025, China;
    2. State Key Laboratory of Public Big Data (Guizhou University), Guiyang Guizhou 550025, China;
    3. College of Computer Science and Technology, Guizhou University, Guiyang Guizhou 550025, China
  • Received:2024-09-28 Revised:2024-12-20 Online:2025-05-05 Published:2025-05-14

PDF (PC)

236

Abstract: Relation extraction aims to extract the semantic relationship between two named entities. Recently, prompt learning has unified the optimization objectives of pre-trained language models and fine-tuning by concatenating prompt templates and performing mask prediction, achieving excellent performance in the field of relationship extraction. However, weak semantic associations between fixed prompt templates and relation instances are observed, which limits the model’s ability to perceive complex relationships. To address this issue, a relation extraction method based on mask attention and multi-feature convolution networks is proposed. The tri-affine attention mechanism is adopted to interactively map the mask in the prompt template with the semantic space of the original text. Two-dimensional mask semantics are then formed through this process. Multi-feature convolutional networks and multi-layer perceptron are employed to extract relational information from the two-dimensional mask semantics. Explicit semantic dependencies between the mask and the relation instance are established. This approach enhances the semantic perception of complex relationships in prompt models. Performances of 91.4%, 91.2%, and 82.6% are achieved on the SemEval, SciERC, and CLTC datasets, respectively. The effectiveness of the proposed method is demonstrated by experimental results.

Key words: natural language processing, relation extraction, prompt learning, tri-affine attention mechanism, convolutional neural network

CLC Number:  TP391.1
[1] 谢德鹏, 常青. 关系抽取综述[J]. 计算机应用研究, 2020, 37(7): 1921-1924, 1930. DOI: 10.19734/j.issn.1001-3695.2018.12.0923.
[2]黄勋, 游宏梁, 于洋. 关系抽取技术研究综述[J]. 现代图书情报技术, 2013(11): 30-39. DOI: 10.11925/infotech.1003-3513.2013.11.05.
[3]DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186. DOI: 10.18653/v1/N19-1423.
[4]LIU Y H, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[EB/OL]. (2019-07-26)[2024-09-28]. https://arxiv.org/abs/1907.11692. DOI: 10.48550/arXiv.1907.11692.
[5]车万翔, 刘挺, 李生. 实体关系自动抽取[J]. 中文信息学报, 2005, 19(2): 1-6. DOI: 10.3969/j.issn.1003-0077.2005.02.001.
[6]SOARES L B, FITZGERALD N, LING J, et al. Matching the blanks: distributional similarity for relation learning[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 2895-2905. DOI: 10.18653/v1/P19-1279.
[7]ZHOU W X, CHEN M H. An improved baseline for sentence-level relation extraction[C]// Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 161-168. DOI: 10.18653/v1/2022.aacl-short.21.
[8]WU S C, HE Y F. Enriching pre-trained language model with entity information for relation classification[C]// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: Association for Computing Machinery, 2019: 2361-2364. DOI: 10.1145/3357384.3358119.
[9]BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook, NY: Curran Associates Inc., 2020: 1877-1901.
[10]HAN X, ZHAO W L, DING N, et al. PTR: prompt tuning with rules for text classification[J]. AI Open, 2022, 3: 182-192. DOI: 10.1016/j.aiopen.2022.11.003.
[11]SCHICK T, SCHÜTZE H. It’s not just size that matters: small language models are also few-shot learners[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2021: 2339-2352. DOI: 10.18653/v1/2021.naacl-main.185.
[12]CHEN X, ZHANG N Y, XIE X, et al. KnowPrompt: knowledge-aware prompt-tuning with synergistic optimization for relation extraction[C]// Proceedings of the ACM Web Conference 2022. New York: Association for Computing Machinery, 2022: 2778-2788. DOI: 10.1145/3485447.3511998.
[13]闫雄, 段跃兴, 张泽华. 采用自注意力机制和CNN融合的实体关系抽取[J]. 计算机工程与科学, 2020, 42(11): 2059-2066. DOI: 10.3969/j.issn.1007-130X.2020.11.019.
[14]宋睿, 陈鑫, 洪宇, 等. 基于卷积循环神经网络的关系抽取[J]. 中文信息学报, 2019, 33(10): 64-72. DOI: 10.3969/j.issn.1003-0077.2019.10.008.
[15]MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2016:1105-1116. DOI: 10.18653/v1/P16-1105.
[16]冯建周, 宋沙沙, 王元卓, 等. 基于改进注意力机制的实体关系抽取方法[J]. 电子学报, 2019, 47(8): 1692-1700. DOI: 10.3969/j.issn.1003-0077.2019.10.008.
[17]VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL]. (2023-08-02)[2024-09-28]. https://arxiv.org/abs/1706.03762. DOI: 10.48550/arXiv.1706.03762.
[18]阳小华, 张硕望, 欧阳纯萍. 中文关系抽取技术研究[J]. 南华大学学报(自然科学版), 2018, 32(1): 66-72. DOI: 10.3969/j.issn.1673-0062.2018.01.013.
[19]武小平, 张强, 赵芳, 等. 基于BERT的心血管医疗指南实体关系抽取方法[J]. 计算机应用, 2021, 41(1): 145-149. DOI: 10.11772/j.issn.1001-9081.2020061008.
[20]CHEN Y P, WANG K, YANG W Z, et al. A multi-channel deep neural network for relation extraction[J]. IEEE Access, 2020, 8: 13195-13203. DOI: 10.1109/ACCESS.2020.2966303.
[21]ZHAO K, XU H, CHENG Y, et al. Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction[J]. Knowledge-Based Systems, 2021, 219: 106888. DOI: 10.1016/j.knosys.2021.106888.
[22]LI J C, KATSIS Y, BALDWIN T, et al. SPOT: knowledge-enhanced language representations for information extraction[C]// Proceedings of the 31st ACM International Conference on Information & Knowledge Management. Stroudsburg, PA: Association for Computational Linguistics, 2022: 1124-1134. DOI: 10.1145/3511808.3557459.
[23]PETERS M E, NEUMANN M, LOGAN R, et al. Knowledge enhanced contextual Word representations[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Stroudsburg, PA: Association for Computational Linguistics, 2019: 43-54. DOI: 10.18653/v1/D19-1005.
[24]文坤建, 陈艳平, 黄瑞章, 等. 基于提示学习的生物医学关系抽取方法[J]. 计算机科学, 2023, 50(10): 223-229. DOI: 10.11896/jsjkx.220900108.
[25]魏超, 陈艳平, 王凯, 等. 基于掩码提示与门控记忆网络校准的关系抽取方法[J]. 计算机应用, 2024, 44(6): 1713-1719. DOI: 10.11772/j.issn.1001-9081.2023060818.
[26]HENDRICKX I, KIM S N, KOZAREVA Z, et al. SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals[C]// Proceedings of the 5th International Workshop on Semantic Evaluation. Stroudsburg, PA: Association for Computational Linguistics, 2010: 33-38.
[27]LUAN Y, HE L H, OSTENDORF M, et al. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2018: 3219-3232. DOI: 10.18653/v1/D18-1360.
[28]XU J J, WEN J, SUN X, et al. A discourse-level named entity recognition and relation extraction dataset for Chinese literature text[EB/OL]. (2019-06-11)[2024-09-28]. https://arxiv.org/abs/1711.07010. DOI: 10.48550/arXiv.1711.07010.
[29]TIAN Y H, CHEN G M, SONG Y, et al. Dependency-driven relation extraction with attentive graph convolutional networks[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2021:4458-4471. DOI: 10.18653/v1/2021.acl-long.344.
[30]QIN Y B, YANG W Z, WANG K, et al. Entity relation extraction based on entity indicators[J]. Symmetry, 2021, 13(4): 539. DOI: 10.3390/sym13040539.
[31]WANG K, CHEN Y P, WEN K J, et al. Cue prompt adapting model for relation extraction[J]. Connection Science, 2023, 35(1): 2161478. DOI: 10.1080/09540091.2022.2161478.
[32]TOUVRON H, LAVRIL T, IZACARD G, et al. LLaMA: open and efficient foundation language models[EB/OL]. (2023-02-27)[2024-09-28]. https://arxiv.org/abs/2302.13971. DOI: 10.48550/arXiv.2302.13971.
[33]LI B, YU D Y, YE W, et al. Sequence generation with label augmentation for relation extraction[C]// Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. Menlo Park, CA: AAAI Press, 2023: 13043-13050. DOI: 10.1609/aaai.v37i11.26532.
[34]ZHAO Q H, GAO T H, GUO N. A novel Chinese relation extraction method using polysemy rethinking mechanism[J]. Applied Intelligence, 2023, 53(7): 7665-7676. DOI: 10.1007/s10489-022-03817-5.
[1] HE Ankang, CHEN Yanping, HU Ying, HUANG Ruizhang, QIN Yongbin. Fusing Boundary Interaction Information for Named Entity Recognition [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 1-11.
[2] HUANG Renhui, ZHANG Ruifeng, WEN Xiaohao, BI Jinjie, HUANG Shoulin, LI Tinghui. Complex-value Covariance-based Convolutional Neural Network for Decoding Motor Imagery-based EEG Signals [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 43-56.
[3] WU Zhengqing, CAO Hui, LIU Baokai. Chinese Fake Review Detection Based on Attention Convolutional Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 26-36.
[4] PAN Haiming, CHEN Qingfeng, QIU Jie, HE Naixu, LIU Chunyu, DU Xiaojing. Multi-hop Knowledge Graph Question Answering Based on Convolution Reasoning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(1): 102-112.
[5] HAO Yaru, DONG Li, XU Ke, LI Xianxian. Interpretability of Pre-trained Language Models: A Survey [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(5): 59-71.
[6] TIAN Sheng, SONG Lin. Traffic Sign Recognition Based on CNN and Bagging Integration [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(4): 35-46.
[7] ZHOU Shengkai, FU Lizhen, SONG Wen’ai. Semantic Similarity Computing Model for Short Text Based on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 49-56.
[8] MA Chengxu, ZENG Shangyou, ZHAO Junbo, CHEN Hongyang. Research on Backlight Image Enhancement Based on Convolutional Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(2): 81-90.
[9] CHEN Wenkang, LU Shenglian, LIU Binghao, LI Guo, LIU Xiaoyu, CHEN Ming. Real-time Citrus Recognition under Orchard Environment by Improved YOLOv4 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 134-146.
[10] YANG Zhou, FAN Yixing, ZHU Xiaofei, GUO Jiafeng, WANG Yue. Survey on Modeling Factors of Neural Information Retrieval Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 1-12.
[11] DENG Wenxuan, YANG Hang, JIN Ting. A Dimensionality-reduction Method Based on Attention Mechanismon Image Classification [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 32-40.
[12] YAN Hao, XU Hongbo, SHEN Yinghan, CHENG Xueqi. Research on Open Chinese Event Detection [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 64-71.
[13] FAN Rui, JIANG Pinqun, ZENG Shangyou, XIA Haiying, LIAO Zhixian, LI Peng. Design of Lightweight Convolution Neural Network Based on Multi-scale Parallel Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(3): 50-59.
[14] WU Wenya,CHEN Yufeng,XU Jin’an,ZHANG Yujie. High-level Semantic Attention-based Convolutional Neural Networks for Chinese Relation Extraction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 32-41.
[15] WANG Qi,QIU Jiahui,RUAN Tong,GAO Daqi,GAO Ju. Recurrent Capsule Network for Clinical Relation Extraction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 80-88.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Guangxi Normal University(Natural Science Edition), All Rights Reserved.
Tel: 0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
Powered by Beijing Magtech Co. Ltd