广西师范大学学报（自然科学版） ›› 2022, Vol. 40 ›› Issue (4): 79-90.doi: 10.16088/j.issn.1001-6600.2021093002

• 研究论文 • 上一篇    下一篇

基于生成对抗网络的类别文本生成

蔡丽坤1,2,3, 吴运兵1,2,3, 陈甘霖1,2,3, 刘翀凌1,2,3, 廖祥文1,2,3*   

  1. 1. 福州大学计算机与大数据学院,福建福州 350108;
    2. 福建省网络计算与智能信息处理重点实验室(福州大学),福建福州 350108;
    3. 福州大学数字福建金融大数据研究所,福建福州 350108
  • 发布日期:2022-08-05
  • 通讯作者: 廖祥文(1980—), 男, 福建安溪人, 福州大学教授, 博导。E-mail: liaoxw@fzu.edu.cn
  • 基金资助:
    国家自然科学基金(61976054); 福建省科技计划项目引导性项目(2019H0040)

Category Text Generation Based on Generative Adversarial Network

CAI Likun1,2,3, WU Yunbing1,2,3, CHEN Ganlin1,2,3, LIU Chongling1,2,3, LIAO Xiangwen1,2,3*   

  1. 1. College of Computer and Data Science, Fuzhou University, Fuzhou Fujian 350108, China;
    2. Fujian Provincial Key Laboratory of Networking Computing and Intelligent Information Processing (Fuzhou University), Fuzhou Fujian 350108, China;
    3. Digital Fujian Institute of Financial Big Data, Fuzhou Fujian 350108, China
  • Published:2022-08-05

PDF (PC)

230

摘要： 类别文本生成旨在让机器生成人类可理解的文本,并且赋予生成文本特定的类别属性。现有工作主要采用基于生成对抗网络的文本生成框架,往往直接采用卷积神经网络进行文本特征提取,缺乏对文本全局语义的关注;此外,简单地在生成网络中引入注意力无法有效消除解码过程中的噪声。针对上述问题,本文提出一种将文本全局特征与局部特征联合建模的方法,通过将长短时记忆网络提取的全局语义信息与卷积神经网络提取的局部语义信息进行融合,增强生成过程中对文本全局语义信息的关注,并且引入双重注意力,进一步过滤掉序列生成中的无关信息。与基准模型相比,本文提出的方法分别在2个公开的真实数据集(Movie Review和Amazon Review)上取得了至少0.01和0.004的BLEU值的提升,表明了本文方法的有效性。

关键词: 文本生成, 生成对抗网络, 双重注意力, 特征融合, 进化学习算法

Abstract: The research of category text generation is a task to enable machines to generate human-understandable text, and give the text specific attributes. The CNN network has been used to extract the features of the text in most of the existing text generation work based on Generative Adversarial Networks that lacks attention to the global semantics. In addition, the attention mechanism is simply introduced to the generator, which can not effectively eliminate the noise in the decoding process. For this reason, this paper proposes a joint modeling method of local semantics and global semantics. The global semantic information extracted by the LSTM and the local semantic information extracted by the CNN are fused, so that the attention on the semantic information is enhanced during the generation. Moreover, the attention on attention is introduced in the generator to further filter irrelevant information during the sequence generation. Compared with the benchmark models, the method proposed in this paper achieves at least 0.01 and 0.004 improvement in BLEU values on two public real datasets (Movie Review and Amazon Review), demonstrating the effectiveness of the proposed method.

Key words: text generation, generative adversarial networks, attention on attention, feature fusion, evolutionary learning algorithm

中图分类号: 

  • TP391.1
[1] LI Y, PAN Q, WANG S H, et al. A Generative Model for category text generation[J]. Information Sciences, 2018, 450: 301-315. DOI: 10.1016/j.ins.2018.03.050.
[2]张建华, 陈家骏. 自然语言生成综述[J]. 计算机应用研究, 2006,23(8): 1-3,13. DOI:10.3969/j.issn.1001-3695.2006.08.001.
[3]蒋晶晶, 牟向伟, 胡家兴, 等. 文本生成模型研究[J]. 价值工程, 2015, 34(13): 185-188. DOI: 10.14018/j.cnki.cn13-1085/n.2015.13.068.
[4]袁江林, 郭志刚, 陈刚, 等. 基于深度学习的文本自动生成技术研究综述[J]. 信息工程大学学报, 2018, 19(5): 616-620. DOI: 10.3969/j.issn.1671-0673.2018.05.020.
[5]梁俊杰, 韦舰晶, 蒋正锋. 生成对抗网络GAN综述[J]. 计算机科学与探索, 2020, 14(1): 1-17.DOI: 10.3778/j.issn.1673-9418.1910026.
[6]NIE W L, NARODYTSKA N, PATEL A. RelGAN: relational generative adversarial networks for text generation[C]// International Conference on Learning Representations 2019. New Orleans, LA: ICLR, 2019: 1-20.
[7]石磊, 王毅, 成颖, 等. 自然语言处理中的注意力机制研究综述[J]. 数据分析与知识发现, 2020, 4(5): 1-14. DOI: 10.11925/infotech.2096-3467.2019.1317.
[8]WANG K, WAN X J. SentiGAN: generating sentimental texts via mixture adversarial networks[C]// Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. Stockholm, Sweden: IJCAI, 2018: 4446-4452. DOI: 10.24963/ijcai.2018/618.
[9]LIU Z Y, WANG J H, LIANG Z W. CatGAN: category-aware generative adversarial networks with hierarchical evolutionary learning for category text generation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 8425-8432. DOI: 10.1609/aaai.v34i05.6361.
[10]周飞燕, 金林鹏, 董军.卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251. DOI: 10.11897/SP.J.1016.2017.01229.
[11]杨丽, 吴雨茜, 王俊丽, 等. 循环神经网络研究综述[J]. 计算机应用, 2018, 38(S2): 1-6,26.
[12]李良友, 贡正仙, 周国栋. 机器翻译自动评价综述[J]. 中文信息学报, 2014, 28(3): 81-91.DOI: 10.3969/j.issn.1003-0077.2014.03.011.
[13]ZHU Y M, LU S D, ZHENG L, et al. Texygen: a benchmarking platform for text generation models[C]// The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. New York, NY: ACM Press, 2018: 1097-1100. DOI: 10.1145/3209978.3210080.
[14]KUSNER M J, HERNNDEZ-LOBATO J M. GANS for sequences of discrete elements with the gumbel-softmax distribution[EB/OL]. (2016-11-12)[2021-09-30]. http://arxiv.org/abs/1611.04051. DOI: 10.48550/arXiv.1611.04051.
[15]YU L T, ZHANG W N, WANG J, et al. SeqGAN: sequence generative adversarial nets with policy gradient[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2017, 31(1): 2852-2858.
[16]刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1-27. DOI: 10.11897/SP.J.1016.2018.00001.
[17]RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. (2018-06-09)[2021-09-30].http://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[18]RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[EB/OL]. (2019-02-14)[2021-09-30]. http://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
[19]BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook, NY: Curran Associates, Inc., 2020: 1877-1901.
[20]LI J T, SONG Y, ZHANG H S, et al. Generating classical Chinese poems via conditional variational autoencoder and adversarial training[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2018: 3890-3900. DOI: 10.18653/v1/D18-1423.
[21]DUAN Y, XU C W, PEI J X, et al. Pre-train and Plug-in: flexible conditional text generation with variational auto-encoders[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2020: 253-262. DOI: 10.18653/v1/2020.acl-main.23.
[22]WANG K, WAN X J. Automatic generation of sentimental texts via mixture adversarial networks[J]. Artificial Intelligence, 2019, 275: 540-558. DOI: 10.1016/j.artint.2019.07.003.
[23]王丽萍, 丰美玲, 邱启仓,等. 偏好多目标进化算法研究综述[J]. 计算机学报, 2019, 42(6): 1289-1315. DOI: 10.11897/SP.J.1016.2019.01289.
[24]SANTORO A, FAULKNER R, RAPOSO D, et al.Relational recurrent neural networks[C]// Advances in Neural Information Processing Systems 31 (NeurIPS 2018). Red Hook, NY: Curran Associates, Inc., 2018: 1-12.
[25]HUANG L, WANG W M, CHEN J, et al. Attention on attention for image captioning[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV 2019). Los Alamitos, CA: IEEE Computer Society, 2019: 4633-4642. DOI: 10.1109/ICCV.2019.00473.
[26]HE R N, RAVULA A, KANAGAL B, et al. RealFormer: transformer likes residual attention[C]// Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Stroudsburg, PA: Association for Computational Linguistics, 2021: 929-943. DOI: 10.18653/v1/2021.findings-acl.81.
[27]JOLICOEUR-MARTINEAU A.The relativistic discriminator: a key element missing from standard GAN[C]// International Conference on Learning Representations 2019. New Orleans, LA: ICLR, 2019: 1-26.
[28]SOCHER R, PERELYGIN A, WU J, et al.Recursive deep models for semantic compositionality over a sentiment treebank[C]// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2013: 1631-1642.
[29]McAULEY J, TARGETT C, SHI Q F, et al. Image-based recommendations on styles and substitutes[C]// Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY: ACM Press, 2015: 43-52. DOI: 10.1145/2766462.2767755.
[1] 彭涛, 唐经, 何凯, 胡新荣, 刘军平, 何儒汉. 基于多步态特征融合的情感识别[J]. 广西师范大学学报（自然科学版）, 2022, 40(3): 104-111.
[2] 张伟彬, 吴军, 易见兵. 基于RFB网络的特征融合管制物品检测算法研究[J]. 广西师范大学学报（自然科学版）, 2021, 39(4): 34-46.
[3] 白捷, 高海力, 王永众, 杨来邦, 项晓航, 楼雄伟. 基于多路特征融合的Faster R-CNN与迁移学习的学生课堂行为检测[J]. 广西师范大学学报（自然科学版）, 2020, 38(5): 1-11.
[4] 张灿龙, 李燕茹, 李志欣, 王智文. 基于核相关滤波与特征融合的分块跟踪算法[J]. 广西师范大学学报（自然科学版）, 2020, 38(5): 12-23.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 广西师范大学学报（自然科学版）编辑部
地址：广西桂林市三里店育才路15号　邮编：541004
电话：0773-5857325　E-mail: gxsdzkb@mailbox.gxnu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发