广西师范大学学报（自然科学版） ›› 2025, Vol. 43 ›› Issue (3): 23-34.doi: 10.16088/j.issn.1001-6600.2024092807

• CCIR2024 • 上一篇    下一篇

基于主题多视图表示的零样本实体检索方法

齐丹丹1, 王长征2, 郭少茹1, 闫智超1, 胡志伟1, 苏雪峰1,3, 马博翔1, 李时钊1, 李茹1,4*   

  1. 1.山西大学计算机与信息技术学院, 山西太原 030006;
    2.山西同方知网数字出版技术有限公司, 山西太原 030006;
    3.山西工程科技职业大学现代物流学院, 山西晋中 030609;
    4.计算智能与中文信息处理教育部重点实验室(山西大学), 山西太原 030006
  • 收稿日期:2024-09-28 修回日期:2024-12-22 出版日期:2025-05-05 发布日期:2025-05-14
  • 通讯作者: 李茹(1963—), 女, 山西大同人, 山西大学教授, 博士。E-mail:liru@sxu.edu.cn
  • 基金资助:
    山西省重点研发计划(202102020101008); 山西省科技合作交流专项(202204041101016); 山西省基础研究计划(202203021211286, 202403021211092)

Topic-based Multi-view Entity Representation for Zero-Shot Entity Retrieval

QI Dandan1 , WANG Changzheng2 , GUO Shaoru1, YAN Zhichao1, HU Zhiwei1, SU Xuefeng1,3, MA Boxiang1, LI Shizhao1, LI Ru1,4*   

  1. 1. School of Computer and Information Technology, Shanxi University, Taiyuan Shanxi 030006, China;
    2. Shanxi Tongfang Knowledge Network Digital Publishing Technology Co., Ltd., Taiyuan Shanxi 030006, China;
    3. School of Modern Logistics, Shanxi Vocational University of Engineering Science and Technology, Jinzhong Shanxi 030609, China;
    4. Key Laboratory Computational Intelligence and Chinese Information Processing of Ministry of Education (Shanxi University), Taiyuan Shanxi 030006, China
  • Received:2024-09-28 Revised:2024-12-22 Online:2025-05-05 Published:2025-05-14

PDF (PC)

183

摘要： 零样本实体检索旨在将实体提及(mention)链接到训练阶段未见过的实体,在多种自然语言处理任务中起关键作用。然而现有方法依然存在2个问题:1)仅使用实体描述的前k个句子来构建实体的多视图表示,导致实体多视图语义冗余与缺失,很难充分学习提及与实体之间的匹配关系;2)仅以提及为中心构造正负例,对提及与实体之间的对比关系覆盖度较低,导致其匹配错误。 针对以上2个问题,本文提出基于主题的多视图实体表示(Topic-MVER)方法。该方法基于主题构建实体的多视图表示,并使用对比学习建模提及与实体之间的3种关系,提升提及和实体对表示的匹配性。该方法在ZESHEL和MedMentions数据集上的Recall@1分别达到48.13%和73.86%,较基线模型分别提升2.73和1.21个百分点,验证了本文方法的有效性。

关键词: 实体检索, 零样本, 长文本, 主题多视图, 对比学习

Abstract: Zero-shot entity retrieval, which aims to link mentions to entities unseen during training, plays a vital role in many natural language processing tasks. However, previous methods suffer from two main limitations: (1) The use of only the first k sentences of entity descriptions to construct multi-view representations leads to redundancy and loss of semantic information in these views, making it difficult to fully learn the matching relationship between mentions and entities; (2) The focus solely on mentions to construct positive and negative examples, with inadequate consideration of the comparative relationships between mentions and entities, results in incorrect matchings. To address these issues, a topic-based multi-view entity representations (Topic-MVER) method is proposed in this paper. This method constructs multi-view representations for entities based on topics and employs contrastive learning to model three types of relationships between mentions and entities, enhancing the matching degree between them. Finally, the method achieves Recall@1 scores of 48.13% and 73.86% on the ZESHEL and MedMentions datasets, respectively, presenting improvements of 2.73% and 1.21% over the baseline models. This validates the effectiveness of the proposed method.

Key words: entity retrieval, zero-shot, long document, topic-based multi-view, contrastive learning

中图分类号:  TP391.1

[1] PERSHINA M, HE Y F, GRISHMAN R. Personalized page rank for named entity disambiguation[C]// Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2015: 238-243. DOI: 10.3115/v1/N15-1026.
[2]ZWICKLBAUER S, SEIFERT C, GRANITZER M. Robust and collective entity disambiguation through semantic embeddings[C]// Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. New York: Association for Computing Machinery, 2016: 425-434. DOI: 10.1145/2911451.2911535.
[3]CECCARELLI D, LUCCHESE C, ORLANDO S, et al. Learning relatedness measures for entity linking[C]// Proceedings of the 22nd ACM international conference on Information & Knowledge Management. New York: Association for Computing Machinery, 2013: 139-148. DOI: 10.1145/2505515.2505711.
[4]HAN X P, SUN L, ZHAO J. Collective entity linking in web text: a graph-based method[C]// Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. New York: Association for Computing Machinery, 2011: 765-774. DOI: 10.1145/2009916.2010019.
[5]徐正斐, 辛欣. 基于大语言模型的中文实体链接实证研究[J]. 自动化学报, 2025, 51(2): 327-342. DOI: 10.16383/j.aas.c240069.
[6]龙斌. 基于预训练模型的中文短文本实体链接研究与实现[D]. 无锡: 江南大学, 2023. DOI: 10.27169/d.cnki.gwqgu.2023.001591.
[7]TANG H Y, SUN X W, JIN B H, et al. A bidirectional multi-paragraph reading model for zero-shot entity linking[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(15): 13889-13897. DOI: 10.1609/aaai.v35i15.17636.
[8]XU Z R, CHEN Y L, HU B T, et al. A read-and-select framework for zero-shot entity linking[C]// Findings of the Association for Computational Linguistics: EMNLP 2023. Stroudsburg, PA: Association for Computational Linguistics, 2023: 13657-13666. DOI: 10.18653/v1/2023.findings-emnlp.912.
[9]李明扬, 姜嘉伟, 孔芳. 融入丰富信息的高性能神经实体链接[J]. 中文信息学报, 2020, 34(1): 87-96. DOI: 10.3969/j.issn.1003-0077.2020.01.012.
[10]张玥, 李韧, 杨建喜, 等. 基于深度神经网络的实体链接研究综述[J]. 中文信息学报, 2024, 38(8): 1-14. DOI: 10.3969/j.issn.1003-0077.2024.08.001.
[11]LOGESWARAN L, CHANG M W, LEE K, et al. Zero-shot entity linking by reading entity descriptions[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 3449-3460. DOI: 10.18653/v1/P19-1335.
[12]GILLICK D, KULKARNI S, LANSING L, et al. Learning dense representations for entity retrieval[C]// Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). Stroudsburg, PA: Association for Computational Linguistics, 2019: 528-537. DOI: 10.18653/v1/K19-1049.
[13]WU L, PETRONI F, JOSIFOSKI M, et al. Scalable zero-shot entity linking with dense entity retrieval[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: Association for Computational Linguistics, 2020: 6397-6407. DOI: 10.18653/v1/2020.emnlp-main.519.
[14]MA X Y, JIANG Y, BACH N, et al. MuVER: improving first-stage entity retrieval with multi-view entity representations[C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2021: 2617-2624.
[15]ZHANG W Z, STRATOS K. Understanding hard negatives in noise contrastive estimation[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2021: 1090-1101. DOI: 10.18653/v1/2021.naacl-main.86.
[16]MOHAN S, LI D H. MedMentions: a large biomedical corpus annotated with UMLS concepts[EB/OL]. (2019-02-25)[2024-09-28]. https://arxiv.org/abs/1902.09476. DOI: 10.48550/arXiv.1902.09476.
[17]冯冲, 石戈, 郭宇航, 等. 基于词向量语义分类的微博实体链接方法[J]. 自动化学报, 2016, 42(6): 915-922. DOI: 10.16383/j.aas.2016.c150715.
[18]汪沛, 线岩团, 郭剑毅, 等. 一种结合词向量和图模型的特定领域实体消歧方法[J]. 智能系统学报, 2016, 11(3): 366-375. DOI: 10.11992/tis.201603048.
[19]陈贵龙. 基于深度学习的实体链接方法与系统[D].北京: 中国科学院大学, 2020. DOI: 10.27824/d.cnki.gzkdx.2020.000028.
[20]ROBERTSON S, ZARAGOZA H. The probabilistic relevance framework: BM25 and beyond[J]. Foundations and Trends© in Information Retrieval, 2009, 3(4): 333-389. DOI: 10.1561/1500000019.
[21]王晨旭. 面向新闻文本的实体抽取和链接技术研究[D]. 武汉: 华中科技大学, 2021. DOI: 10.27157/d.cnki.ghzku.2021.006189.
[22]AGARWAL D, ANGELL R, MONATH N, et al. Entity linking via explicit mention-mention coreference modeling[C]// Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2022: 4644-4658. DOI: 10.18653/v1/2022.naacl-main.343.
[23]WU T Q, BAI X Y, GUO W G, et al. Modeling fine-grained information via knowledge-aware hierarchical graph for zero-shot entity retrieval[C]// Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. New York: Association for Computing Machinery, 2023: 1021-1029. DOI: 10.1145/3539597.3570415.
[24]邵子洋. 基于语义强化嵌入的实体链接研究及应用[D]. 成都: 电子科技大学, 2023. DOI: 10.27005/d.cnki.gdzku.2023.002158.
[25]PARTALIDOU E, CHRISTOU D, TSOUMAKAS G. Improving zero-shot entity retrieval through effective dense representations[C]// Proceedings of the 12th Hellenic Conference on Artificial Intelligence. New York: Association for Computing Machinery, 2022: 1-5. DOI: 10.1145/3549737.3549771.
[26]SUN K, ZHANG R C, MENSAH S, et al. A transformational biencoder with in-domain negative sampling for zero-shot entity linking[C]// Findings of the Association for Computational Linguistics: ACL 2022. Stroudsburg, PA: Association for Computational Linguistics, 2022: 1449-1458. DOI: 10.18653/v1/2022.findings-acl.114.
[27]XIONG L, XIONG C Y, LI Y, et al. Approximate nearest neighbor negative contrastive learning for dense text retrieval[EB/OL]. (2020-10-20)[2024-09-28]. https://arxiv.org/abs/2007.00808. DOI: 10.48550/arXiv.2007.00808.
[28]ZHAN J T, MAO J X, LIU Y Q, et al. Optimizing dense retrieval model training with hard negatives[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: Association for Computing Machinery, 2021: 1503-1512. DOI: 10.1145/3404835.3462880.
[29]范鹏程, 沈英汉, 许洪波, 等. 融合实体知识描述的实体联合消歧方法[J]. 中文信息学报, 2020, 34(7): 42-49, 78. DOI: 10.3969/j.issn.1003-0077.2020.07.004.
[30]XIE Q Z, DAI Z H, HOVY E, et al. Unsupervised data augmentation for consistency training[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook: Curran Associates Inc., 2020: 6256-6268.
[31]SUKHBAATAR S, GRAVE E, BOJANOWSKI P, et al. Adaptive attention span in transformers[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 331-335. DOI: 10.18653/v1/P19-1032.
[32]RAE J W, POTAPENKO A, JAYAKUMAR S M, et al. Compressive transformers for long-range sequence modelling[EB/OL]. (2019-11-13)[2024-09-28]. https://arxiv.org/abs/1911.05507. DOI: 10.48550/arXiv.1911.05507.
[33]YAO Z H, CAO L L, PAN H P. Zero-shot entity linking with efficient long range sequence modeling[C]// Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg, PA: Association for Computational Linguistics, 2020: 2517-2522. DOI: 10.18653/v1/2020.findings-emnlp.228.
[34]ZHANG S Y, LIANG Y B, GONG M, et al. Multi-view document representation learning for open-domain dense retrieval[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 5990-6000. DOI: 10.18653/v1/2022.acl-long.414.
[35]BIRD S, LOPER E. NLTK: the natural language toolkit[C]// Proceedings of the ACL Interactive Poster and Demonstration Sessions. Stroudsburg, PA: Association for Computational Linguistics, 2004: 214-217. DOI: 10.3115/1219044.1219075.
[36]GROOTENDORST M. BERTopic: neural topic modeling with a class-based TF-IDF procedure[EB/OL]. (2022-03-11)[2024-09-28]. https://arxiv.org/abs/2203.05794. DOI: 10.48550/arXiv.2203.05794.
[37]VAN DEN OORD A, LI Y Z, VINYALS O. Representation learning with contrastive predictive coding[EB/OL]. (2019-01-22)[2024-09-28]. https://arxiv.org/abs/1807.03748. DOI: 10.48550/arXiv.1807.03748.
[38]JOHNSON J, DOUZE M, JÉGOU H. Billion-scale similarity search with GPUs[J]. IEEE Transactions on Big Data, 2021, 7(3): 535-547. DOI: 10.1109/TBDATA.2019.2921572.
[39]杜帅文, 靳婷. 基于用户行为特征的深度混合推荐算法[J]. 广西师范大学学报(自然科学版), 2024, 42(5): 91-100. DOI: 10.16088/j.issn.1001-6600.2023110603.
[1] 李志欣, 刘鸣琦. 差异特征导向的解耦多模态情感分析[J]. 广西师范大学学报（自然科学版）, 2025, 43(3): 57-71.
[2] 张丽杰, 王绍卿, 张尧, 孙福振. 基于分级注意力网络和多层对比学习的社交推荐[J]. 广西师范大学学报（自然科学版）, 2025, 43(2): 133-148.
[3] 李文博, 董青, 刘超, 张奇. 基于对比学习的儿科问诊对话细粒度意图识别[J]. 广西师范大学学报（自然科学版）, 2024, 42(4): 1-10.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发