基于对比学习的儿科问诊对话细粒度意图识别

doi:10.16088/j.issn.1001-6600.2023111304

摘要/Abstract

摘要： 问诊对话系统的基础是自然语言理解。自然语言理解是指从对话信息中提取出意图信息和实体信息,并将其转换为结构化表达,主要包括意图识别和槽填充2种任务。意图识别是一种典型的文本分类任务,槽填充则是使用序列算法从对话文本中根据预先设定好的槽位抽取对应的槽位值。传统的方法通常对意图识别和槽填充2个任务分别构建模型,并在意图识别的基础上根据意图进行槽填充,但是这种方式容易造成错误传播。针对该问题,本文提出一种基于对比学习方法的融合对话意图分类和语义槽取值的细粒度意图识别方法。该方法结合意图分类和语义槽取值任务,使用BART作为骨干模型进行改进和创新,该模型使用编解码架构,意图识别和槽填充任务共享一个编码层,解码层采用字级别标签,通过将意图信息融合进槽填充任务,并在样本构造过程中引入对比学习。实验结果表明,本文算法在医患对话数据集上的意图识别准确率达到81.96%,槽填充的F₁分数达到85.26%,与其他算法相比有明显的效果提升。另外,通过消融实验和样例分析,进一步证明了本文算法的效果。

关键词: 对比学习, 意图识别, 槽填充, 细粒度, 医疗对话

Abstract: The foundation of the inquiry dialogue system is rooted in natural language understanding (NLU), where NLU involves the extraction of intent and entity information from conversational data, transforming it into a structured representation. This process primarily encompasses two tasks: intent recognition and slot filling. Intent recognition, a typical text classification task, aims to discern the underlying purpose of the dialogue, while slot filling utilizes sequential algorithms to extract corresponding slot values based on predefined positions within the conversation. Conventional approaches often build separate models for intent recognition and slot filling, subsequently performing slot filling based on the recognized intent. However, this methodology is susceptible to error propagation. To address this issue, this paper proposes a fine-grained intent recognition method that integrates dialogue intent classification and semantic slot value extraction using a contrastive learning approach. The method combines intent classification and slot value tasks, leveraging BART as the backbone model for improvement and innovation. This model, employing an encoder-decoder architecture, shares an encoding layer for intent recognition and slot filling tasks. Additionally, it adopts character-level labels in the decoding layer, thereby integrating intent information into the slot filling task. Contrastive learning is introduced during the sample construction process. Experimental results demonstrate that the proposed algorithm achieves an intent recognition accuracy of 81.96% and a slot filling F₁ score of 85.26% on a medical dialogue dataset, showing significant performance improvements compared with other algorithms. The paper also conducts ablation experiments on contrastive learning, historical information, and sentence-level intent to further substant the effectiveness of the proposed method.

Key words: contrastive learning, intent recognition, slot-filling, fine-grained, medical dialogue

中图分类号: TP391

李文博, 董青, 刘超, 张奇. 基于对比学习的儿科问诊对话细粒度意图识别[J]. 广西师范大学学报（自然科学版）, 2024, 42(4): 1-10.

LI Wenbo, DONG Qing, LIU Chao, ZHANG Qi. Fine-grained Intent Recognition from Pediatric Medical Dialogues with Contrastive Learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(4): 1-10.

参考文献

[1] ZHANG Y J, HUANG L S, ZHOU X, et al. Characteristics and workload of pediatricians in China[J]. Pediatrics, 2019, 144(1): e20183532. DOI: 10.1542/peds.2018-3532.
[2] LIANG H Y, TSUI B Y, NI H, et al. Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence[J]. Nature Medicine, 2019, 25(3): 433-438. DOI: 10.1038/s41591-018-0335-9.
[3] BAKER A, PEROV Y, MIDDLETON K, et al. A comparison of artificial intelligence and human doctors for the purpose of triage and diagnosis[J]. Frontiers in Artificial Intelligence, 2020, 3: 543405. DOI: 10.3389/frai.2020.543405.
[4] 魏鹏飞, 曾碧, 汪明慧, 等. 基于深度学习的口语理解联合建模算法综述[J]. 软件学报, 2022, 33(11): 4192-4216. DOI: 10.13328/j.cnki.jos.006385.
[5] 陈燕, 龚庆悦, 戴彩艳. 基于句法抽取与图结构编码的患者问询意图识别[J]. 计算机与数字工程, 2021, 49(11): 2276-2281, 2334. DOI: 10.3969/j.issn.1672-9722.2021.11.020.
[6] 王志明, 郑凯. 基于BERT的中文医疗问答系统[J]. 计算机系统应用, 2023, 32(6): 115-120. DOI: 10.15888/j.cnki.csa.009140.
[7] 余慧, 冯旭鹏, 刘利军, 等. 聊天机器人中用户就医意图识别方法[J]. 计算机应用, 2018, 38(8): 2170-2174. DOI: 10.11772/j.issn.1001-9081.2018010190.
[8] 王宇亮, 杨观赐, 罗可欣. 基于意图—槽位注意机制的医疗咨询意图理解与实体抽取算法[J]. 计算机应用研究, 2023, 40(5): 1402-1409. DOI: 10.19734/j.issn.1001-3695.2022.11.0547.
[9] FAN J F, WANG M L, LI C L, et al. Intent-slot correlation modeling for joint intent prediction and slot filling[J]. Journal of Computer Science and Technology, 2022, 37(2): 309-319. DOI: 10.1007/s11390-020-0326-4.
[10] 侯丽仙, 李艳玲, 林民, 等. 融合多约束条件的意图和语义槽填充联合识别[J]. 计算机科学与探索, 2020, 14(9): 1545-1553. DOI: 10.3778/j.issn.1673-9418.1909009.
[11] 马天宇, 覃俊, 刘晶, 等. 基于BERT的意图分类与槽填充联合方法[J]. 中文信息学报, 2022, 36(8): 127-134. DOI: 10.3969/j.issn.1003-0077.2022.08.016.
[12] 马常霞, 张晨. 中文对话理解中基于预训练的意图分类和槽填充联合模型[J]. 山东大学学报(工学版), 2020, 50(6): 68-75. DOI: 10.6040/j.issn.1672-3961.0.2020.236.
[13] 周天益, 范永全, 杜亚军, 等. 基于细粒度信息集成的意图识别和槽填充联合模型[J]. 计算机应用研究, 2023, 40(9): 2669-2673. DOI: 10.19734/j.issn.1001-3695.2023.01.0015.
[14] 孟佳娜, 单明, 孙世昶, 等. 融入历史信息的多轮对话意图识别[J]. 大连民族大学学报, 2023, 25(3): 244-249. DOI: 10.3969/j.issn.1009-315X.2023.03.009.
[15] WEI Z Y, LIU Q L, PENG B L, et al. Task-oriented dialogue system for automatic diagnosis[C] //Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 201-207. DOI: 10.18653/v1/P18-2033.
[16] CHEN J Y, LI D F, CHEN Q C, et al. Diaformer: automatic diagnosis via symptoms sequence generation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(4): 4432-4440. DOI: 10.1609/aaai.v36i4.20365.
[17] XU L, ZHOU Q X, GONG K, et al. End-to-end knowledge-routed relational dialogue system for automatic diagnosis[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 7346-7353. DOI: 10.1609/aaai.v33i01.33017346.
[18] LIN X Z, HE X H, CHEN Q, et al. Enhancing dialogue symptom diagnosis with global attention and symptom graph[C] //Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2019: 5033-5042. DOI: 10.18653/v1/D19-1508.
[19] ZHONG C, LIAO K G E, CHEN W, et al. Hierarchical reinforcement learning for automatic disease diagnosis[J]. Bioinformatics, 2022, 38(16): 3995-4001. DOI: 10.1093/bioinformatics/btac408.
[20] HADSELL R, CHOPRA S, LECUN Y. Dimensionality reduction by learning an invariant mapping[C] //2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2006: 1735-1742. DOI: 10.1109/CVPR.2006.100.
[21] BOSE A J, LING H, CAO Y S. Adversarial contrastive estimation[C] //Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 1021-1032. DOI: 10.18653/v1/P18-1094.
[22] YANG Z H, CHENG Y, LIU Y, et al. Reducing word omission errors in neural machine translation: a contrastive learning approach[C] //Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 6191-6196. DOI: 10.18653/v1/P19-1623.
[23] YU Y, ZUO S M, JIANG H M, et al. Fine-tuning pre-trained language model with weak supervision: a contrastive-regularized self-training approach[C] //Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2021: 1063-1077. DOI: 10.18653/v1/2021.naacl-main.84.
[24] GAO T Y, YAO X C, CHEN D Q. SimCSE: simple contrastive learning of sentence embeddings[C] //Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2021: 6894-6910. DOI: 10.18653/v1/2021.emnlp-main.552.
[25] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Advances in Neural Information Processing Systems 30 (NIPS 2017). Red Hook, NY: Curran Associates Inc., 2017: 6000-6010.
[26] 向卓元, 陈浩, 王倩, 等. 面向任务型对话的小样本语言理解模型研究[J]. 数据分析与知识发现, 2023, 7(9): 64-77. DOI: 10.11925/infotech.2096-3467.2022.0825.
[27] GE W F, HUANG W L, DONG D K, et al. Deep metric learning with hierarchical triplet loss[C] //Computer Vision-ECCV 2018: LNCS Volume 11210. Cham: Springer, 2018: 272-288. DOI: 10.1007/978-3-030-01231-1_17.
[28] 姚倩媛.医疗对话文本意图识别和槽填充联合模型研究[D]. 上海:复旦大学,2021.
[29] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[C] //Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2017: 427-431.
[30] DYER C, BALLESTEROS M, LING W, et al. Transition-based dependency parsing with stack long short-term memory[C] //Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2015: 334-343. DOI: 10.3115/v1/P15-1033.
[31] LIU B, LANE I. Attention-based recurrent neural network models for joint intent detection and slot filling[C] //Proceedings of the Interspeech 2016. Red Hook, NY: Curran Associates, Inc., 2016: 685-689. DOI: 10.21437/Interspeech.2016-1352.
[32] GOO C W, GAO G, HSU Y K, et al. Slot-gated modeling for joint slot filling and intent prediction[C] //Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2018: 753-757. DOI: 10.18653/v1/N18-2118.
[33] QIN L B, CHE W X, LI Y M, et al. A stack-propagation framework with token-level intent detection for spoken language understanding[C] //Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2019: 2078-2087. DOI: 10.18653/v1/D19-1214.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed