Journal of Guangxi Normal University(Natural Science Edition) ›› 2025, Vol. 43 ›› Issue (3): 23-34.doi: 10.16088/j.issn.1001-6600.2024092807

• CCIR2024 • Previous Articles     Next Articles

Topic-based Multi-view Entity Representation for Zero-Shot Entity Retrieval

QI Dandan1 , WANG Changzheng2 , GUO Shaoru1, YAN Zhichao1, HU Zhiwei1, SU Xuefeng1,3, MA Boxiang1, LI Shizhao1, LI Ru1,4*   

  1. 1. School of Computer and Information Technology, Shanxi University, Taiyuan Shanxi 030006, China;
    2. Shanxi Tongfang Knowledge Network Digital Publishing Technology Co., Ltd., Taiyuan Shanxi 030006, China;
    3. School of Modern Logistics, Shanxi Vocational University of Engineering Science and Technology, Jinzhong Shanxi 030609, China;
    4. Key Laboratory Computational Intelligence and Chinese Information Processing of Ministry of Education (Shanxi University), Taiyuan Shanxi 030006, China
  • Received:2024-09-28 Revised:2024-12-22 Online:2025-05-05 Published:2025-05-14

PDF (PC)

183

Abstract: Zero-shot entity retrieval, which aims to link mentions to entities unseen during training, plays a vital role in many natural language processing tasks. However, previous methods suffer from two main limitations: (1) The use of only the first k sentences of entity descriptions to construct multi-view representations leads to redundancy and loss of semantic information in these views, making it difficult to fully learn the matching relationship between mentions and entities; (2) The focus solely on mentions to construct positive and negative examples, with inadequate consideration of the comparative relationships between mentions and entities, results in incorrect matchings. To address these issues, a topic-based multi-view entity representations (Topic-MVER) method is proposed in this paper. This method constructs multi-view representations for entities based on topics and employs contrastive learning to model three types of relationships between mentions and entities, enhancing the matching degree between them. Finally, the method achieves Recall@1 scores of 48.13% and 73.86% on the ZESHEL and MedMentions datasets, respectively, presenting improvements of 2.73% and 1.21% over the baseline models. This validates the effectiveness of the proposed method.

Key words: entity retrieval, zero-shot, long document, topic-based multi-view, contrastive learning

CLC Number:  TP391.1
[1] PERSHINA M, HE Y F, GRISHMAN R. Personalized page rank for named entity disambiguation[C]// Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2015: 238-243. DOI: 10.3115/v1/N15-1026.
[2]ZWICKLBAUER S, SEIFERT C, GRANITZER M. Robust and collective entity disambiguation through semantic embeddings[C]// Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. New York: Association for Computing Machinery, 2016: 425-434. DOI: 10.1145/2911451.2911535.
[3]CECCARELLI D, LUCCHESE C, ORLANDO S, et al. Learning relatedness measures for entity linking[C]// Proceedings of the 22nd ACM international conference on Information & Knowledge Management. New York: Association for Computing Machinery, 2013: 139-148. DOI: 10.1145/2505515.2505711.
[4]HAN X P, SUN L, ZHAO J. Collective entity linking in web text: a graph-based method[C]// Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. New York: Association for Computing Machinery, 2011: 765-774. DOI: 10.1145/2009916.2010019.
[5]徐正斐, 辛欣. 基于大语言模型的中文实体链接实证研究[J]. 自动化学报, 2025, 51(2): 327-342. DOI: 10.16383/j.aas.c240069.
[6]龙斌. 基于预训练模型的中文短文本实体链接研究与实现[D]. 无锡: 江南大学, 2023. DOI: 10.27169/d.cnki.gwqgu.2023.001591.
[7]TANG H Y, SUN X W, JIN B H, et al. A bidirectional multi-paragraph reading model for zero-shot entity linking[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(15): 13889-13897. DOI: 10.1609/aaai.v35i15.17636.
[8]XU Z R, CHEN Y L, HU B T, et al. A read-and-select framework for zero-shot entity linking[C]// Findings of the Association for Computational Linguistics: EMNLP 2023. Stroudsburg, PA: Association for Computational Linguistics, 2023: 13657-13666. DOI: 10.18653/v1/2023.findings-emnlp.912.
[9]李明扬, 姜嘉伟, 孔芳. 融入丰富信息的高性能神经实体链接[J]. 中文信息学报, 2020, 34(1): 87-96. DOI: 10.3969/j.issn.1003-0077.2020.01.012.
[10]张玥, 李韧, 杨建喜, 等. 基于深度神经网络的实体链接研究综述[J]. 中文信息学报, 2024, 38(8): 1-14. DOI: 10.3969/j.issn.1003-0077.2024.08.001.
[11]LOGESWARAN L, CHANG M W, LEE K, et al. Zero-shot entity linking by reading entity descriptions[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 3449-3460. DOI: 10.18653/v1/P19-1335.
[12]GILLICK D, KULKARNI S, LANSING L, et al. Learning dense representations for entity retrieval[C]// Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). Stroudsburg, PA: Association for Computational Linguistics, 2019: 528-537. DOI: 10.18653/v1/K19-1049.
[13]WU L, PETRONI F, JOSIFOSKI M, et al. Scalable zero-shot entity linking with dense entity retrieval[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: Association for Computational Linguistics, 2020: 6397-6407. DOI: 10.18653/v1/2020.emnlp-main.519.
[14]MA X Y, JIANG Y, BACH N, et al. MuVER: improving first-stage entity retrieval with multi-view entity representations[C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2021: 2617-2624.
[15]ZHANG W Z, STRATOS K. Understanding hard negatives in noise contrastive estimation[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2021: 1090-1101. DOI: 10.18653/v1/2021.naacl-main.86.
[16]MOHAN S, LI D H. MedMentions: a large biomedical corpus annotated with UMLS concepts[EB/OL]. (2019-02-25)[2024-09-28]. https://arxiv.org/abs/1902.09476. DOI: 10.48550/arXiv.1902.09476.
[17]冯冲, 石戈, 郭宇航, 等. 基于词向量语义分类的微博实体链接方法[J]. 自动化学报, 2016, 42(6): 915-922. DOI: 10.16383/j.aas.2016.c150715.
[18]汪沛, 线岩团, 郭剑毅, 等. 一种结合词向量和图模型的特定领域实体消歧方法[J]. 智能系统学报, 2016, 11(3): 366-375. DOI: 10.11992/tis.201603048.
[19]陈贵龙. 基于深度学习的实体链接方法与系统[D].北京: 中国科学院大学, 2020. DOI: 10.27824/d.cnki.gzkdx.2020.000028.
[20]ROBERTSON S, ZARAGOZA H. The probabilistic relevance framework: BM25 and beyond[J]. Foundations and Trends© in Information Retrieval, 2009, 3(4): 333-389. DOI: 10.1561/1500000019.
[21]王晨旭. 面向新闻文本的实体抽取和链接技术研究[D]. 武汉: 华中科技大学, 2021. DOI: 10.27157/d.cnki.ghzku.2021.006189.
[22]AGARWAL D, ANGELL R, MONATH N, et al. Entity linking via explicit mention-mention coreference modeling[C]// Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2022: 4644-4658. DOI: 10.18653/v1/2022.naacl-main.343.
[23]WU T Q, BAI X Y, GUO W G, et al. Modeling fine-grained information via knowledge-aware hierarchical graph for zero-shot entity retrieval[C]// Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. New York: Association for Computing Machinery, 2023: 1021-1029. DOI: 10.1145/3539597.3570415.
[24]邵子洋. 基于语义强化嵌入的实体链接研究及应用[D]. 成都: 电子科技大学, 2023. DOI: 10.27005/d.cnki.gdzku.2023.002158.
[25]PARTALIDOU E, CHRISTOU D, TSOUMAKAS G. Improving zero-shot entity retrieval through effective dense representations[C]// Proceedings of the 12th Hellenic Conference on Artificial Intelligence. New York: Association for Computing Machinery, 2022: 1-5. DOI: 10.1145/3549737.3549771.
[26]SUN K, ZHANG R C, MENSAH S, et al. A transformational biencoder with in-domain negative sampling for zero-shot entity linking[C]// Findings of the Association for Computational Linguistics: ACL 2022. Stroudsburg, PA: Association for Computational Linguistics, 2022: 1449-1458. DOI: 10.18653/v1/2022.findings-acl.114.
[27]XIONG L, XIONG C Y, LI Y, et al. Approximate nearest neighbor negative contrastive learning for dense text retrieval[EB/OL]. (2020-10-20)[2024-09-28]. https://arxiv.org/abs/2007.00808. DOI: 10.48550/arXiv.2007.00808.
[28]ZHAN J T, MAO J X, LIU Y Q, et al. Optimizing dense retrieval model training with hard negatives[C]// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: Association for Computing Machinery, 2021: 1503-1512. DOI: 10.1145/3404835.3462880.
[29]范鹏程, 沈英汉, 许洪波, 等. 融合实体知识描述的实体联合消歧方法[J]. 中文信息学报, 2020, 34(7): 42-49, 78. DOI: 10.3969/j.issn.1003-0077.2020.07.004.
[30]XIE Q Z, DAI Z H, HOVY E, et al. Unsupervised data augmentation for consistency training[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook: Curran Associates Inc., 2020: 6256-6268.
[31]SUKHBAATAR S, GRAVE E, BOJANOWSKI P, et al. Adaptive attention span in transformers[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 331-335. DOI: 10.18653/v1/P19-1032.
[32]RAE J W, POTAPENKO A, JAYAKUMAR S M, et al. Compressive transformers for long-range sequence modelling[EB/OL]. (2019-11-13)[2024-09-28]. https://arxiv.org/abs/1911.05507. DOI: 10.48550/arXiv.1911.05507.
[33]YAO Z H, CAO L L, PAN H P. Zero-shot entity linking with efficient long range sequence modeling[C]// Findings of the Association for Computational Linguistics: EMNLP 2020. Stroudsburg, PA: Association for Computational Linguistics, 2020: 2517-2522. DOI: 10.18653/v1/2020.findings-emnlp.228.
[34]ZHANG S Y, LIANG Y B, GONG M, et al. Multi-view document representation learning for open-domain dense retrieval[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 5990-6000. DOI: 10.18653/v1/2022.acl-long.414.
[35]BIRD S, LOPER E. NLTK: the natural language toolkit[C]// Proceedings of the ACL Interactive Poster and Demonstration Sessions. Stroudsburg, PA: Association for Computational Linguistics, 2004: 214-217. DOI: 10.3115/1219044.1219075.
[36]GROOTENDORST M. BERTopic: neural topic modeling with a class-based TF-IDF procedure[EB/OL]. (2022-03-11)[2024-09-28]. https://arxiv.org/abs/2203.05794. DOI: 10.48550/arXiv.2203.05794.
[37]VAN DEN OORD A, LI Y Z, VINYALS O. Representation learning with contrastive predictive coding[EB/OL]. (2019-01-22)[2024-09-28]. https://arxiv.org/abs/1807.03748. DOI: 10.48550/arXiv.1807.03748.
[38]JOHNSON J, DOUZE M, JÉGOU H. Billion-scale similarity search with GPUs[J]. IEEE Transactions on Big Data, 2021, 7(3): 535-547. DOI: 10.1109/TBDATA.2019.2921572.
[39]杜帅文, 靳婷. 基于用户行为特征的深度混合推荐算法[J]. 广西师范大学学报(自然科学版), 2024, 42(5): 91-100. DOI: 10.16088/j.issn.1001-6600.2023110603.
[1] LI Zhixin, LIU Mingqi. A Dissimilarity Feature-Driven Decoupled Multimodal Sentiment Analysis [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 57-71.
[2] ZHANG Lijie, WANG Shaoqing, ZHANG Yao, SUN Fuzhen. Multi-level Attention Networks and Hierarchical Contrastive Learning for Social Recommendation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(2): 133-148.
[3] LI Wenbo, DONG Qing, LIU Chao, ZHANG Qi. Fine-grained Intent Recognition from Pediatric Medical Dialogues with Contrastive Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(4): 1-10.
[4] GAO Shengxiang, YANG Yuanzhang, WANG Linqin, MO Shangbin, YU Zhengtao, DONG Ling. Multi-level Disentangled Personalized Speech Synthesis for Out-of-Domain Speakers Adaptation Scenarios [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(4): 11-21.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Guangxi Normal University(Natural Science Edition), All Rights Reserved.
Tel: 0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
Powered by Beijing Magtech Co. Ltd