融合释义与双向交互的成语阅读理解方法研究

doi:10.16088/j.issn.1001-6600.2023021101

摘要/Abstract

摘要： 中文成语有着独特的抽象语义。在机器阅读理解任务中,为了解决模型不能充分理解成语含义的问题,本文提出一种完形填空式匹配网络。匹配网络将成语与其词典释义以注意力交互的方式融入模型,使成语得到更好的向量表征,并且在段落与候选答案之间采用双向交互的策略。将匹配网络与语言模型(BERT、 ERNIE等)相结合,本文设计出一个针对完形填空式阅读理解任务的模型。实验结果表明,该模型优于传统模型、SKER模型以及结合增强型全局注意力的BERT模型,在中文成语数据集CHID上达到77.0%的准确率。

关键词: 机器阅读理解, 中文成语, 匹配网络, 完形填空, 双向交互

Abstract: Chinese idioms have unique abstract meanings. In machine reading comprehension, in order to solve the problem that the model cannot fully understand the complex semantics of Chinese idioms, a cloze-test matching network is proposed. The matching network incorporates the idioms and their dictionary interpretations into the model in an attentional interaction manner, so that the idioms get a better idiom vector representation, and a bidirectional interaction strategy is adopted between the passages and the candidate answers. A model is designed for the cloze-test reading comprehension task using matching networks combined with BERT and ERNIE language models, respectively. The model outperforms other traditional models, SKER model and BERT model combined with enhanced global attention, and achieves 77.0% accuracy on the Chinese idiom dataset CHID.

Key words: machine reading comprehension, Chinese idioms, matching network, cloze-test, bidirectional interaction

中图分类号: TP391.1

温雪岩, 谷训开, 李祯, 黄英来, 黄鹤林. 融合释义与双向交互的成语阅读理解方法研究[J]. 广西师范大学学报（自然科学版）, 2023, 41(6): 70-79.

WEN Xueyan, GU Xunkai, LI Zhen, HUANG Yinglai, HUANG Helin. Study of Idiom Reading Comprehension Methods Integrating Interpretation and Bidirectional Interaction[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 70-79.

参考文献

[1] 徐霄玲, 郑建立, 尹梓名. 机器阅读理解的技术研究综述[J]. 小型微型计算机系统, 2020, 41(3): 464-470. DOI: 10.3969/j.issn.1000-1220.2020.03.003.
[2] LIU S S, ZHANG X, ZHANG S, et al. Neural machine reading comprehension: methods and trends[J]. Applied Sciences, 2019, 9(18): 3698. DOI: 10.3390/app9183698.
[3] 马晨辉, 施水才, 肖诗斌. 多项选择式机器阅读理解综述[J]. 北京信息科技大学学报(自然科学版), 2021, 36(5): 91-96. DOI: 10.16508/j.cnki.11-5866/n.2021.05.015.
[4] 包玥, 李艳玲, 林民. 抽取式机器阅读理解研究综述[J]. 计算机工程与应用, 2021, 57(12): 25-36. DOI: 10.3778/j.issn.1002-8331.2102-0038.
[5] 杨康, 黄定江, 高明. 面向自动问答的机器阅读理解综述[J]. 华东师范大学学报(自然科学版), 2019(5): 36-52. DOI: 10.3969/j.issn.1000-5641.2019.05.003.
[6] 吕文蓉, 郭泽晨, 马宁. 机器阅读理解技术及应用研究[J]. 西北民族大学学报(自然科学版), 2022, 43(1): 17-25. DOI: 10.14084/j.cnki.cn62-1188/n.2022.01.008.
[7] 王小捷, 白子薇, 李可, 等. 机器阅读理解的研究进展[J]. 北京邮电大学学报, 2019, 42(6): 1-9. DOI: 10.13190/ j.jbupt.2019-111.
[8] MUELLER J, THYAGARAJAN A. Siamese recurrent architectures for learning sentence similarity[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2016, 30(1): 2786-2792. DOI: 10.1609/aaai.v30i1.10350.
[9] CHUNG J Y, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL]. (2014-12-11)[2023-02-11]. http://arxiv.org/abs/1412.3555. DOI: 10.48550/arXiv.1412.3555.
[10] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186. DOI: 10.18653/v1/N19-1423.
[11] 周圣凯, 富丽贞, 宋文爱. 基于深度学习的短文本语义相似度计算模型[J]. 广西师范大学学报(自然科学版), 2022, 40(3): 49-56. DOI: 10.16088/j.issn.1001-6600.2021071001.
[12] 张超然, 裘杭萍, 孙毅, 等. 基于预训练模型的机器阅读理解研究综述[J]. 计算机工程与应用, 2020, 56(11): 17-25. DOI: 10.3778/j.issn.1002-8331.2001-0285.
[13] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook, NY: Curran Associates Inc., 2020: 1877-1901.
[14] 杜永萍, 赵以梁, 阎婧雅, 等. 基于深度学习的机器阅读理解研究综述[J]. 智能系统学报, 2022, 17(6): 1074-1083. DOI: 10.11992/tis.202107024.
[15] QIN R Y, LUO H Z, FAN Z H, et al. IBERT: idiom cloze-style reading comprehension with attention[EB/OL]. (2021-11-05)[2023-02-11]. http://arxiv.org/abs/2112.02994. DOI: 10.48550/arXiv.2112.02994.
[16] ZHENG C J, HUANG M L, SUN A X. ChID: a large-scale Chinese IDiom dataset for cloze test[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 778-787. DOI: 10.18653/v1/P19-1075.
[17] LONG S Y, WANG R, TAO K, et al. Synonym knowledge enhanced reader for Chinese idiom reading comprehension[C]// Proceedings of the 28th International Conference on Computational Linguistics. Barcelona: International Committee on Computational Linguistics, 2020: 3684-3695. DOI: 10.18653/v1/2020.coling-main.329.
[18] 谭华, 朱鸿斌, 陈昌润, 等. 基于深度学习的中文成语推荐应用[J]. 工业控制计算机, 2022, 35(4): 90-91. DOI: 10.3969/j.issn.1001-182X.2022.04.035.
[19] 徐家伟, 刘瑞芳, 高升, 等. 面向中文成语的阅读理解方法研究[J]. 中文信息学报, 2021, 35(7): 118-125. DOI: 10.3969/j.issn.1003-0077.2021.07.014.
[20] 顾迎捷, 桂小林, 李德福, 等. 基于神经网络的机器阅读理解综述[J]. 软件学报, 2020, 31(7): 2095-2126. DOI: 10.13328/j.cnki.jos.006048.
[21] 姬发家, 朱莹, 阴皓, 等. 基于混合神经网络的自然语言处理技术研究[J]. 电子设计工程, 2023, 31(10): 92-96. DOI: 10.14022/j.issn1674-6236.2023.10.020.
[22] JIN D, GAO S Y, KAO J Y, et al. MMM: multi-stage multi-task learning for multi-choice reading comprehension[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 8010-8017. DOI: 10.1609/aaai.v34i05.6310.
[23] ZHANG S L, ZHAO H, WU Y W, et al. Dual co-matching network for multi-choice reading comprehension[EB/OL]. (2019-08-20)[2023-02-11]. http://arxiv.org/abs/1901.09381v2. DOI: 10.48550/arXiv.1901.09381.
[24] ZHU P F, ZHANG Z S, ZHAO H, et al. DUMA: reading comprehension with transposition thinking[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022, 30: 269-279. DOI: 10.1109/TASLP.2021.3138683.
[25] 霍欢, 邹依婷, 周澄睿, 等. 一种应用于填空型阅读理解的句式注意力网络[J]. 小型微型计算机系统, 2019, 40(3): 482-487. DOI: 10.3969/j.issn.1000-1220.2019.03.004.
[26] FU C Z, ZHANG Y. EA reader: enhance attentive reader for cloze-style question answering via multi-space context fusion[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 6375-6382. DOI: 10.1609/aaai.v33i01.33016375.
[27] 盛艺暄, 兰曼. 利用外部知识辅助和多步推理的选择题型机器阅读理解模型[J]. 计算机系统应用, 2020, 29(4): 1-9. DOI: 10.15888/j.cnki.csa.007327.
[28] YANG B S, MITCHELL T. Leveraging knowledge bases in LSTMs for improving machine reading[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2017: 1436-1446. DOI: 10.18653/v1/P17-1132.
[29] MIHAYLOV T, FRANK A. Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2018: 821-832. DOI: 10.18653/v1/P18-1076.
[30] SUN Y B, GUO D Y, TANG D Y, et al. Knowledge based machine reading comprehension[EB/OL]. (2018-09-12)[2023-02-11]. http://arxiv.org/abs/1809.04267. DOI: 10.48550/arXiv1809.04267.
[31] WANG S H, XU Y H, FANG Y W, et al. Training data is more valuable than you think: a simple and effective method by retrieving from training data[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 3170-3179. DOI: 10.18653/v1/2022.acl-long.226.
[32] LOVENIA H, WILIE B, CHUNG W, et al. Clozer: adaptable data augmentation for cloze-style reading comprehension[C]// Proceedings of the 7th Workshop on Representation Learning for NLP. Stroudsburg, PA: Association for Computational Linguistics, 2022: 60-66. DOI: 10.18653/v1/2022.repl4nlp-1.7.
[33] SUN Y, WANG S H, LI Y K, et al. ERNIE: enhanced representation through knowledge integration[EB/OL]. (2019-04-19)[2023-02-11]. http://arxiv.org/abs/1904.09223. DOI: 10.48550/arXiv.1904.09223.
[34] CUI Y M, CHE W X, LIU T, et al. Pre-training with whole word masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514. DOI: 10.1109/TASLP.2021.3124365.
[35] WU C H, WU F Z, QI T, et al. NoisyTune: a little noise can help you finetune pretrained language models better[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 680-685. DOI: 10.18653/v1/2022.acl-short.76.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed