Journal of Guangxi Normal University(Natural Science Edition) ›› 2023, Vol. 41 ›› Issue (6): 70-79.doi: 10.16088/j.issn.1001-6600.2023021101

Previous Articles     Next Articles

Study of Idiom Reading Comprehension Methods Integrating Interpretation and Bidirectional Interaction

WEN Xueyan*, GU Xunkai, LI Zhen, HUANG Yinglai, HUANG Helin   

  1. School of Information and Computer Engineering, Northeast Forestry University, Harbin Heilongjiang 150040, China
  • Received:2023-02-11 Revised:2023-04-12 Published:2023-12-04

PDF (PC)

294

Abstract: Chinese idioms have unique abstract meanings. In machine reading comprehension, in order to solve the problem that the model cannot fully understand the complex semantics of Chinese idioms, a cloze-test matching network is proposed. The matching network incorporates the idioms and their dictionary interpretations into the model in an attentional interaction manner, so that the idioms get a better idiom vector representation, and a bidirectional interaction strategy is adopted between the passages and the candidate answers. A model is designed for the cloze-test reading comprehension task using matching networks combined with BERT and ERNIE language models, respectively. The model outperforms other traditional models, SKER model and BERT model combined with enhanced global attention, and achieves 77.0% accuracy on the Chinese idiom dataset CHID.

Key words: machine reading comprehension, Chinese idioms, matching network, cloze-test, bidirectional interaction

CLC Number:  TP391.1
[1] 徐霄玲, 郑建立, 尹梓名. 机器阅读理解的技术研究综述[J]. 小型微型计算机系统, 2020, 41(3): 464-470. DOI: 10.3969/j.issn.1000-1220.2020.03.003.
[2] LIU S S, ZHANG X, ZHANG S, et al. Neural machine reading comprehension: methods and trends[J]. Applied Sciences, 2019, 9(18): 3698. DOI: 10.3390/app9183698.
[3] 马晨辉, 施水才, 肖诗斌. 多项选择式机器阅读理解综述[J]. 北京信息科技大学学报(自然科学版), 2021, 36(5): 91-96. DOI: 10.16508/j.cnki.11-5866/n.2021.05.015.
[4] 包玥, 李艳玲, 林民. 抽取式机器阅读理解研究综述[J]. 计算机工程与应用, 2021, 57(12): 25-36. DOI: 10.3778/j.issn.1002-8331.2102-0038.
[5] 杨康, 黄定江, 高明. 面向自动问答的机器阅读理解综述[J]. 华东师范大学学报(自然科学版), 2019(5): 36-52. DOI: 10.3969/j.issn.1000-5641.2019.05.003.
[6] 吕文蓉, 郭泽晨, 马宁. 机器阅读理解技术及应用研究[J]. 西北民族大学学报(自然科学版), 2022, 43(1): 17-25. DOI: 10.14084/j.cnki.cn62-1188/n.2022.01.008.
[7] 王小捷, 白子薇, 李可, 等. 机器阅读理解的研究进展[J]. 北京邮电大学学报, 2019, 42(6): 1-9. DOI: 10.13190/ j.jbupt.2019-111.
[8] MUELLER J, THYAGARAJAN A. Siamese recurrent architectures for learning sentence similarity[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2016, 30(1): 2786-2792. DOI: 10.1609/aaai.v30i1.10350.
[9] CHUNG J Y, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL]. (2014-12-11)[2023-02-11]. http://arxiv.org/abs/1412.3555. DOI: 10.48550/arXiv.1412.3555.
[10] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186. DOI: 10.18653/v1/N19-1423.
[11] 周圣凯, 富丽贞, 宋文爱. 基于深度学习的短文本语义相似度计算模型[J]. 广西师范大学学报(自然科学版), 2022, 40(3): 49-56. DOI: 10.16088/j.issn.1001-6600.2021071001.
[12] 张超然, 裘杭萍, 孙毅, 等. 基于预训练模型的机器阅读理解研究综述[J]. 计算机工程与应用, 2020, 56(11): 17-25. DOI: 10.3778/j.issn.1002-8331.2001-0285.
[13] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook, NY: Curran Associates Inc., 2020: 1877-1901.
[14] 杜永萍, 赵以梁, 阎婧雅, 等. 基于深度学习的机器阅读理解研究综述[J]. 智能系统学报, 2022, 17(6): 1074-1083. DOI: 10.11992/tis.202107024.
[15] QIN R Y, LUO H Z, FAN Z H, et al. IBERT: idiom cloze-style reading comprehension with attention[EB/OL]. (2021-11-05)[2023-02-11]. http://arxiv.org/abs/2112.02994. DOI: 10.48550/arXiv.2112.02994.
[16] ZHENG C J, HUANG M L, SUN A X. ChID: a large-scale Chinese IDiom dataset for cloze test[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 778-787. DOI: 10.18653/v1/P19-1075.
[17] LONG S Y, WANG R, TAO K, et al. Synonym knowledge enhanced reader for Chinese idiom reading comprehension[C]// Proceedings of the 28th International Conference on Computational Linguistics. Barcelona: International Committee on Computational Linguistics, 2020: 3684-3695. DOI: 10.18653/v1/2020.coling-main.329.
[18] 谭华, 朱鸿斌, 陈昌润, 等. 基于深度学习的中文成语推荐应用[J]. 工业控制计算机, 2022, 35(4): 90-91. DOI: 10.3969/j.issn.1001-182X.2022.04.035.
[19] 徐家伟, 刘瑞芳, 高升, 等. 面向中文成语的阅读理解方法研究[J]. 中文信息学报, 2021, 35(7): 118-125. DOI: 10.3969/j.issn.1003-0077.2021.07.014.
[20] 顾迎捷, 桂小林, 李德福, 等. 基于神经网络的机器阅读理解综述[J]. 软件学报, 2020, 31(7): 2095-2126. DOI: 10.13328/j.cnki.jos.006048.
[21] 姬发家, 朱莹, 阴皓, 等. 基于混合神经网络的自然语言处理技术研究[J]. 电子设计工程, 2023, 31(10): 92-96. DOI: 10.14022/j.issn1674-6236.2023.10.020.
[22] JIN D, GAO S Y, KAO J Y, et al. MMM: multi-stage multi-task learning for multi-choice reading comprehension[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 8010-8017. DOI: 10.1609/aaai.v34i05.6310.
[23] ZHANG S L, ZHAO H, WU Y W, et al. Dual co-matching network for multi-choice reading comprehension[EB/OL]. (2019-08-20)[2023-02-11]. http://arxiv.org/abs/1901.09381v2. DOI: 10.48550/arXiv.1901.09381.
[24] ZHU P F, ZHANG Z S, ZHAO H, et al. DUMA: reading comprehension with transposition thinking[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2022, 30: 269-279. DOI: 10.1109/TASLP.2021.3138683.
[25] 霍欢, 邹依婷, 周澄睿, 等. 一种应用于填空型阅读理解的句式注意力网络[J]. 小型微型计算机系统, 2019, 40(3): 482-487. DOI: 10.3969/j.issn.1000-1220.2019.03.004.
[26] FU C Z, ZHANG Y. EA reader: enhance attentive reader for cloze-style question answering via multi-space context fusion[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 6375-6382. DOI: 10.1609/aaai.v33i01.33016375.
[27] 盛艺暄, 兰曼. 利用外部知识辅助和多步推理的选择题型机器阅读理解模型[J]. 计算机系统应用, 2020, 29(4): 1-9. DOI: 10.15888/j.cnki.csa.007327.
[28] YANG B S, MITCHELL T. Leveraging knowledge bases in LSTMs for improving machine reading[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2017: 1436-1446. DOI: 10.18653/v1/P17-1132.
[29] MIHAYLOV T, FRANK A. Knowledgeable reader: enhancing cloze-style reading comprehension with external commonsense knowledge[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2018: 821-832. DOI: 10.18653/v1/P18-1076.
[30] SUN Y B, GUO D Y, TANG D Y, et al. Knowledge based machine reading comprehension[EB/OL]. (2018-09-12)[2023-02-11]. http://arxiv.org/abs/1809.04267. DOI: 10.48550/arXiv1809.04267.
[31] WANG S H, XU Y H, FANG Y W, et al. Training data is more valuable than you think: a simple and effective method by retrieving from training data[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 3170-3179. DOI: 10.18653/v1/2022.acl-long.226.
[32] LOVENIA H, WILIE B, CHUNG W, et al. Clozer: adaptable data augmentation for cloze-style reading comprehension[C]// Proceedings of the 7th Workshop on Representation Learning for NLP. Stroudsburg, PA: Association for Computational Linguistics, 2022: 60-66. DOI: 10.18653/v1/2022.repl4nlp-1.7.
[33] SUN Y, WANG S H, LI Y K, et al. ERNIE: enhanced representation through knowledge integration[EB/OL]. (2019-04-19)[2023-02-11]. http://arxiv.org/abs/1904.09223. DOI: 10.48550/arXiv.1904.09223.
[34] CUI Y M, CHE W X, LIU T, et al. Pre-training with whole word masking for Chinese BERT[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3504-3514. DOI: 10.1109/TASLP.2021.3124365.
[35] WU C H, WU F Z, QI T, et al. NoisyTune: a little noise can help you finetune pretrained language models better[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Stroudsburg, PA: Association for Computational Linguistics, 2022: 680-685. DOI: 10.18653/v1/2022.acl-short.76.
[1] SUN Xu, SHEN Bin, YAN Xin, ZHANG Jinpeng, XU Guangyi. Microblog Opinion Summarization Method Based on Transformer and TextRank [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(4): 96-108.
[2] PAN Haiming, CHEN Qingfeng, QIU Jie, HE Naixu, LIU Chunyu, DU Xiaojing. Multi-hop Knowledge Graph Question Answering Based on Convolution Reasoning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(1): 102-112.
[3] HAO Yaru, DONG Li, XU Ke, LI Xianxian. Interpretability of Pre-trained Language Models: A Survey [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(5): 59-71.
[4] CHAO Rui, ZHANG Kunli, WANG Jiajia, HU Bin, ZHANG Weicong, HAN Yingjie, ZAN Hongying. Construction of Chinese Multimodal Knowledge Base [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 31-39.
[5] LI Zhengguang, CHEN Heng, LIN Hongfei. Identification of Adverse Drug Reaction on Social Media Using Bi-directional Language Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 40-48.
[6] ZHOU Shengkai, FU Lizhen, SONG Wen’ai. Semantic Similarity Computing Model for Short Text Based on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 49-56.
[7] SUN Yansong, YANG Liang, LIN Hongfei. Humor Recognition of Sitcom Based on Multi-granularity of Segmentation Enhancement and Semantic Enhancement [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 57-65.
[8] WANG Jian, ZHENG Qifan, LI Chao, SHI Jing. Remote Supervision Relationship Extraction Based on Encoder and Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(4): 53-60.
[9] SONG Jun, HAN Xiao-yu, HUANG Yu, HUANG Ting-lei, FU Kun. A Method for Entity-Oriented Timeline Summarization [J]. Journal of Guangxi Normal University(Natural Science Edition), 2015, 33(2): 36-41.
[10] ZHANG Fen, QU Wei-guang, ZHAO Hong-yan, ZHOU Jun-sheng. Shallow Parsing Based on CRF and Transformation-basedError-driven Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(3): 147-150.
[11] ZHUO Guang-ping, SUN Jing-yu, LI Xian-hua, YU Xue-li. Personalized Recommendation Algorithm Based on CBR [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(3): 151-156.
[12] CHENG Xian-yi, PAN Yan, ZHU Qian, SUN Ping. Automatic Generating Algorithm of Event-oriented Multi-documentSummarization [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(1): 147-150.
[13] YANG Liang, PAN Feng-ming, LIN Hong-fei. Chunk-based Opinion Object Extraction and Application in OpinionAnalysis [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(1): 151-156.
[14] CHENG Xian-yi, ZHU Qian, HAN Fei. Semantic Chunk of Question Sentence Analysis Based on HNC and Description Logics [J]. Journal of Guangxi Normal University(Natural Science Edition), 2010, 28(3): 131-134.
[15] XIA Ning, LIN Hong-fei, YANG Zhi-hao, LI Yan-peng. Gene Mention Normalization Based on Semantic Featured Machine Learning Disambiguation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2010, 28(3): 144-147.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] DONG Shulong, MA Jiangming, XIN Wenjie. Research Progress and Trend of Landscape Visual Evaluation —Knowledge Atlas Analysis Based on CiteSpace[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 1 -13 .
[2] MA Qianran, WEI Duqu. Chaos Prediction of a Motor System with Two Linearly Coupled Reservoir Computers[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 1 -7 .
[3] YAN Minxiu, JIN Qisen. Construction of Multi-dimensional Chaotic Systems and Its Multi-channel Adaptive Control[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 8 -21 .
[4] ZHAO Wei, TIAN Shuai, ZHANG Qiang, WANG Yaoshen, WANG Sibo, SONG Jiang. Fritillaria ussuriensis Maxim Detection Model Based on Improved YOLOv5[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 22 -32 .
[5] GAO Fei, GUO Xiaobin, YUAN Dongfang, CAO Fujun. Improved PINNs Method for Solving the Convective Dominant Diffusion Equation with Boundary Layer[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 33 -50 .
[6] ZHOU Qiao, ZHAI Jiangtao, JIA Dongsheng, SUN Haoxiang. A Web Attack Detection Method Based on Convolutional Gated Recurrent Neural Network[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 51 -61 .
[7] LIN Wancong, HAN Mingjie, JIN Ting. Multi-level Argument Position Classification Method via Data Augmentation[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 62 -69 .
[8] SONG Guanwu, CHEN Zhiming, LI Jianjun. Remote Sensing Image Classification with Cascade Attention Based on ResNet-50[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 80 -91 .
[9] XU Ziyu, WU Keqing. Uniqueness of Positive Solutions for Caputo Fractional Differential Systems[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 92 -104 .
[10] GUO Jie, SUO Hongmin, ZHU Yiying, GUO Jiachao. Existence of Solutions for a Class of Kirchhoff Type Problems with Critical Exponent and Indefinite Potential[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 105 -112 .
Copyright © Journal of Guangxi Normal University(Natural Science Edition), All Rights Reserved.
Tel: 0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
Powered by Beijing Magtech Co. Ltd