Journal of Guangxi Normal University(Natural Science Edition) ›› 2024, Vol. 42 ›› Issue (4): 1-10.doi: 10.16088/j.issn.1001-6600.2023111304

    Next Articles

Fine-grained Intent Recognition from Pediatric Medical Dialogues with Contrastive Learning

LI Wenbo1, DONG Qing2*, LIU Chao2, ZHANG Qi1   

  1. 1. School of Computer Science, Fudan University, Shanghai 200433, China;
    2. The Affiliated Taian City Central Hospital of Qingdao University, Taian Shandong 271000, China
  • Received:2023-11-13 Revised:2024-02-05 Online:2024-07-25 Published:2024-09-05

Abstract: The foundation of the inquiry dialogue system is rooted in natural language understanding (NLU), where NLU involves the extraction of intent and entity information from conversational data, transforming it into a structured representation. This process primarily encompasses two tasks: intent recognition and slot filling. Intent recognition, a typical text classification task, aims to discern the underlying purpose of the dialogue, while slot filling utilizes sequential algorithms to extract corresponding slot values based on predefined positions within the conversation. Conventional approaches often build separate models for intent recognition and slot filling, subsequently performing slot filling based on the recognized intent. However, this methodology is susceptible to error propagation. To address this issue, this paper proposes a fine-grained intent recognition method that integrates dialogue intent classification and semantic slot value extraction using a contrastive learning approach. The method combines intent classification and slot value tasks, leveraging BART as the backbone model for improvement and innovation. This model, employing an encoder-decoder architecture, shares an encoding layer for intent recognition and slot filling tasks. Additionally, it adopts character-level labels in the decoding layer, thereby integrating intent information into the slot filling task. Contrastive learning is introduced during the sample construction process. Experimental results demonstrate that the proposed algorithm achieves an intent recognition accuracy of 81.96% and a slot filling F1 score of 85.26% on a medical dialogue dataset, showing significant performance improvements compared with other algorithms. The paper also conducts ablation experiments on contrastive learning, historical information, and sentence-level intent to further substant the effectiveness of the proposed method.

Key words: contrastive learning, intent recognition, slot-filling, fine-grained, medical dialogue

CLC Number:  TP391
[1] ZHANG Y J, HUANG L S, ZHOU X, et al. Characteristics and workload of pediatricians in China[J]. Pediatrics, 2019, 144(1): e20183532. DOI: 10.1542/peds.2018-3532.
[2] LIANG H Y, TSUI B Y, NI H, et al. Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence[J]. Nature Medicine, 2019, 25(3): 433-438. DOI: 10.1038/s41591-018-0335-9.
[3] BAKER A, PEROV Y, MIDDLETON K, et al. A comparison of artificial intelligence and human doctors for the purpose of triage and diagnosis[J]. Frontiers in Artificial Intelligence, 2020, 3: 543405. DOI: 10.3389/frai.2020.543405.
[4] 魏鹏飞, 曾碧, 汪明慧, 等. 基于深度学习的口语理解联合建模算法综述[J]. 软件学报, 2022, 33(11): 4192-4216. DOI: 10.13328/j.cnki.jos.006385.
[5] 陈燕, 龚庆悦, 戴彩艳. 基于句法抽取与图结构编码的患者问询意图识别[J]. 计算机与数字工程, 2021, 49(11): 2276-2281, 2334. DOI: 10.3969/j.issn.1672-9722.2021.11.020.
[6] 王志明, 郑凯. 基于BERT的中文医疗问答系统[J]. 计算机系统应用, 2023, 32(6): 115-120. DOI: 10.15888/j.cnki.csa.009140.
[7] 余慧, 冯旭鹏, 刘利军, 等. 聊天机器人中用户就医意图识别方法[J]. 计算机应用, 2018, 38(8): 2170-2174. DOI: 10.11772/j.issn.1001-9081.2018010190.
[8] 王宇亮, 杨观赐, 罗可欣. 基于意图—槽位注意机制的医疗咨询意图理解与实体抽取算法[J]. 计算机应用研究, 2023, 40(5): 1402-1409. DOI: 10.19734/j.issn.1001-3695.2022.11.0547.
[9] FAN J F, WANG M L, LI C L, et al. Intent-slot correlation modeling for joint intent prediction and slot filling[J]. Journal of Computer Science and Technology, 2022, 37(2): 309-319. DOI: 10.1007/s11390-020-0326-4.
[10] 侯丽仙, 李艳玲, 林民, 等. 融合多约束条件的意图和语义槽填充联合识别[J]. 计算机科学与探索, 2020, 14(9): 1545-1553. DOI: 10.3778/j.issn.1673-9418.1909009.
[11] 马天宇, 覃俊, 刘晶, 等. 基于BERT的意图分类与槽填充联合方法[J]. 中文信息学报, 2022, 36(8): 127-134. DOI: 10.3969/j.issn.1003-0077.2022.08.016.
[12] 马常霞, 张晨. 中文对话理解中基于预训练的意图分类和槽填充联合模型[J]. 山东大学学报(工学版), 2020, 50(6): 68-75. DOI: 10.6040/j.issn.1672-3961.0.2020.236.
[13] 周天益, 范永全, 杜亚军, 等. 基于细粒度信息集成的意图识别和槽填充联合模型[J]. 计算机应用研究, 2023, 40(9): 2669-2673. DOI: 10.19734/j.issn.1001-3695.2023.01.0015.
[14] 孟佳娜, 单明, 孙世昶, 等. 融入历史信息的多轮对话意图识别[J]. 大连民族大学学报, 2023, 25(3): 244-249. DOI: 10.3969/j.issn.1009-315X.2023.03.009.
[15] WEI Z Y, LIU Q L, PENG B L, et al. Task-oriented dialogue system for automatic diagnosis[C] //Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 201-207. DOI: 10.18653/v1/P18-2033.
[16] CHEN J Y, LI D F, CHEN Q C, et al. Diaformer: automatic diagnosis via symptoms sequence generation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(4): 4432-4440. DOI: 10.1609/aaai.v36i4.20365.
[17] XU L, ZHOU Q X, GONG K, et al. End-to-end knowledge-routed relational dialogue system for automatic diagnosis[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 7346-7353. DOI: 10.1609/aaai.v33i01.33017346.
[18] LIN X Z, HE X H, CHEN Q, et al. Enhancing dialogue symptom diagnosis with global attention and symptom graph[C] //Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2019: 5033-5042. DOI: 10.18653/v1/D19-1508.
[19] ZHONG C, LIAO K G E, CHEN W, et al. Hierarchical reinforcement learning for automatic disease diagnosis[J]. Bioinformatics, 2022, 38(16): 3995-4001. DOI: 10.1093/bioinformatics/btac408.
[20] HADSELL R, CHOPRA S, LECUN Y. Dimensionality reduction by learning an invariant mapping[C] //2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2006: 1735-1742. DOI: 10.1109/CVPR.2006.100.
[21] BOSE A J, LING H, CAO Y S. Adversarial contrastive estimation[C] //Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 1021-1032. DOI: 10.18653/v1/P18-1094.
[22] YANG Z H, CHENG Y, LIU Y, et al. Reducing word omission errors in neural machine translation: a contrastive learning approach[C] //Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2019: 6191-6196. DOI: 10.18653/v1/P19-1623.
[23] YU Y, ZUO S M, JIANG H M, et al. Fine-tuning pre-trained language model with weak supervision: a contrastive-regularized self-training approach[C] //Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2021: 1063-1077. DOI: 10.18653/v1/2021.naacl-main.84.
[24] GAO T Y, YAO X C, CHEN D Q. SimCSE: simple contrastive learning of sentence embeddings[C] //Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2021: 6894-6910. DOI: 10.18653/v1/2021.emnlp-main.552.
[25] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] //Advances in Neural Information Processing Systems 30 (NIPS 2017). Red Hook, NY: Curran Associates Inc., 2017: 6000-6010.
[26] 向卓元, 陈浩, 王倩, 等. 面向任务型对话的小样本语言理解模型研究[J]. 数据分析与知识发现, 2023, 7(9): 64-77. DOI: 10.11925/infotech.2096-3467.2022.0825.
[27] GE W F, HUANG W L, DONG D K, et al. Deep metric learning with hierarchical triplet loss[C] //Computer Vision-ECCV 2018: LNCS Volume 11210. Cham: Springer, 2018: 272-288. DOI: 10.1007/978-3-030-01231-1_17.
[28] 姚倩媛.医疗对话文本意图识别和槽填充联合模型研究[D]. 上海:复旦大学,2021.
[29] JOULIN A, GRAVE E, BOJANOWSKI P, et al. Bag of tricks for efficient text classification[C] //Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2017: 427-431.
[30] DYER C, BALLESTEROS M, LING W, et al. Transition-based dependency parsing with stack long short-term memory[C] //Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2015: 334-343. DOI: 10.3115/v1/P15-1033.
[31] LIU B, LANE I. Attention-based recurrent neural network models for joint intent detection and slot filling[C] //Proceedings of the Interspeech 2016. Red Hook, NY: Curran Associates, Inc., 2016: 685-689. DOI: 10.21437/Interspeech.2016-1352.
[32] GOO C W, GAO G, HSU Y K, et al. Slot-gated modeling for joint slot filling and intent prediction[C] //Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2018: 753-757. DOI: 10.18653/v1/N18-2118.
[33] QIN L B, CHE W X, LI Y M, et al. A stack-propagation framework with token-level intent detection for spoken language understanding[C] //Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2019: 2078-2087. DOI: 10.18653/v1/D19-1214.
[1] WANG Yuhang, ZHANG Canlong, LI Zhixin, WANG Zhiwen. Image Captioning According to User’s Intention and Style [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(4): 91-103.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] ZHAO Jie, SONG Shuang, WU Bin. Overview of Image USM Sharpening Forensics and Anti-forensics Techniques[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 1 -16 .
[2] AI Congcong, GONG Guoli, JIAO Xiaoyu, TIAN Lu, GAI Zhongchao, GOU Jingxuan, LI Hui. Komagataella phaffii Serves as a Model Organism for Emerging Basic Research[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 17 -26 .
[3] ZHAI Yanhao, WANG Yanwu, LI Qiang, LI Jingkun. Progress of Dissolved Organic Matter in Inland Water by Three-Dimensional Fluorescence Spectroscopy Based on CiteSpace[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 34 -46 .
[4] CHEN Li, TANG Mingzhu, GUO Shenghui. Cyber-Physical Systems State Estimation and Actuator Attack Reconstruction of Intelligent Vehicles[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 59 -69 .
[5] LI Chengqian, SHI Chen, DENG Minyi. Study for the Electrocardiographic Signal of Brugada Syndrome Patients Using Cellular Automaton[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 86 -98 .
[6] LÜ Hui, LÜ Weifeng. Fundus Hemorrhagic Spot Detection Algorithm Based on Improved YOLOv5[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 99 -107 .
[7] YI Jianbing, PENG Xin, CAO Feng, LI Jun, XIE Weijia. Research on Point Cloud Registration Algorithm Based on Multi-scale Feature Fusion[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 108 -120 .
[8] LI Li, LI Haoze, LI Tao. Multi-primary-node Byzantine Fault-Tolerant Consensus Mechanism Based on Raft[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 121 -130 .
[9] ZHAO Xiaomei, DING Yong, WANG Haitao. Maximum Likelihood DOA Estimation Based on Improved Monarch Butterfly Algorithm[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 131 -140 .
[10] ZHU Yan, CAI Jing, LONG Fang. Statistical Analysis of Partially Step Stress Accelerated Life Tests for Compound Rayleigh Distribution Competing Failure Model Under Progressive Type-Ι Hybrid Censoring[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 159 -169 .