基于生成对抗网络的类别文本生成

doi:10.16088/j.issn.1001-6600.2021093002

Abstract

Abstract: The research of category text generation is a task to enable machines to generate human-understandable text, and give the text specific attributes. The CNN network has been used to extract the features of the text in most of the existing text generation work based on Generative Adversarial Networks that lacks attention to the global semantics. In addition, the attention mechanism is simply introduced to the generator, which can not effectively eliminate the noise in the decoding process. For this reason, this paper proposes a joint modeling method of local semantics and global semantics. The global semantic information extracted by the LSTM and the local semantic information extracted by the CNN are fused, so that the attention on the semantic information is enhanced during the generation. Moreover, the attention on attention is introduced in the generator to further filter irrelevant information during the sequence generation. Compared with the benchmark models, the method proposed in this paper achieves at least 0.01 and 0.004 improvement in BLEU values on two public real datasets (Movie Review and Amazon Review), demonstrating the effectiveness of the proposed method.

Key words: text generation, generative adversarial networks, attention on attention, feature fusion, evolutionary learning algorithm

CLC Number:

TP391.1

CAI Likun, WU Yunbing, CHEN Ganlin, LIU Chongling, LIAO Xiangwen. Category Text Generation Based on Generative Adversarial Network[J].Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(4): 79-90.

References

[1] LI Y, PAN Q, WANG S H, et al. A Generative Model for category text generation[J]. Information Sciences, 2018, 450: 301-315. DOI: 10.1016/j.ins.2018.03.050.
[2]张建华, 陈家骏. 自然语言生成综述[J]. 计算机应用研究, 2006,23(8): 1-3,13. DOI:10.3969/j.issn.1001-3695.2006.08.001.
[3]蒋晶晶, 牟向伟, 胡家兴, 等. 文本生成模型研究[J]. 价值工程, 2015, 34(13): 185-188. DOI: 10.14018/j.cnki.cn13-1085/n.2015.13.068.
[4]袁江林, 郭志刚, 陈刚, 等. 基于深度学习的文本自动生成技术研究综述[J]. 信息工程大学学报, 2018, 19(5): 616-620. DOI: 10.3969/j.issn.1671-0673.2018.05.020.
[5]梁俊杰, 韦舰晶, 蒋正锋. 生成对抗网络GAN综述[J]. 计算机科学与探索, 2020, 14(1): 1-17.DOI: 10.3778/j.issn.1673-9418.1910026.
[6]NIE W L, NARODYTSKA N, PATEL A. RelGAN: relational generative adversarial networks for text generation[C]// International Conference on Learning Representations 2019. New Orleans, LA: ICLR, 2019: 1-20.
[7]石磊, 王毅, 成颖, 等. 自然语言处理中的注意力机制研究综述[J]. 数据分析与知识发现, 2020, 4(5): 1-14. DOI: 10.11925/infotech.2096-3467.2019.1317.
[8]WANG K, WAN X J. SentiGAN: generating sentimental texts via mixture adversarial networks[C]// Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. Stockholm, Sweden: IJCAI, 2018: 4446-4452. DOI: 10.24963/ijcai.2018/618.
[9]LIU Z Y, WANG J H, LIANG Z W. CatGAN: category-aware generative adversarial networks with hierarchical evolutionary learning for category text generation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(5): 8425-8432. DOI: 10.1609/aaai.v34i05.6361.
[10]周飞燕, 金林鹏, 董军.卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251. DOI: 10.11897/SP.J.1016.2017.01229.
[11]杨丽, 吴雨茜, 王俊丽, 等. 循环神经网络研究综述[J]. 计算机应用, 2018, 38(S2): 1-6,26.
[12]李良友, 贡正仙, 周国栋. 机器翻译自动评价综述[J]. 中文信息学报, 2014, 28(3): 81-91.DOI: 10.3969/j.issn.1003-0077.2014.03.011.
[13]ZHU Y M, LU S D, ZHENG L, et al. Texygen: a benchmarking platform for text generation models[C]// The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. New York, NY: ACM Press, 2018: 1097-1100. DOI: 10.1145/3209978.3210080.
[14]KUSNER M J, HERNNDEZ-LOBATO J M. GANS for sequences of discrete elements with the gumbel-softmax distribution[EB/OL]. (2016-11-12)[2021-09-30]. http://arxiv.org/abs/1611.04051. DOI: 10.48550/arXiv.1611.04051.
[15]YU L T, ZHANG W N, WANG J, et al. SeqGAN: sequence generative adversarial nets with policy gradient[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2017, 31(1): 2852-2858.
[16]刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1-27. DOI: 10.11897/SP.J.1016.2018.00001.
[17]RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. (2018-06-09)[2021-09-30].http://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[18]RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[EB/OL]. (2019-02-14)[2021-09-30]. http://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
[19]BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]// Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Red Hook, NY: Curran Associates, Inc., 2020: 1877-1901.
[20]LI J T, SONG Y, ZHANG H S, et al. Generating classical Chinese poems via conditional variational autoencoder and adversarial training[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2018: 3890-3900. DOI: 10.18653/v1/D18-1423.
[21]DUAN Y, XU C W, PEI J X, et al. Pre-train and Plug-in: flexible conditional text generation with variational auto-encoders[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2020: 253-262. DOI: 10.18653/v1/2020.acl-main.23.
[22]WANG K, WAN X J. Automatic generation of sentimental texts via mixture adversarial networks[J]. Artificial Intelligence, 2019, 275: 540-558. DOI: 10.1016/j.artint.2019.07.003.
[23]王丽萍, 丰美玲, 邱启仓,等. 偏好多目标进化算法研究综述[J]. 计算机学报, 2019, 42(6): 1289-1315. DOI: 10.11897/SP.J.1016.2019.01289.
[24]SANTORO A, FAULKNER R, RAPOSO D, et al.Relational recurrent neural networks[C]// Advances in Neural Information Processing Systems 31 (NeurIPS 2018). Red Hook, NY: Curran Associates, Inc., 2018: 1-12.
[25]HUANG L, WANG W M, CHEN J, et al. Attention on attention for image captioning[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV 2019). Los Alamitos, CA: IEEE Computer Society, 2019: 4633-4642. DOI: 10.1109/ICCV.2019.00473.
[26]HE R N, RAVULA A, KANAGAL B, et al. RealFormer: transformer likes residual attention[C]// Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Stroudsburg, PA: Association for Computational Linguistics, 2021: 929-943. DOI: 10.18653/v1/2021.findings-acl.81.
[27]JOLICOEUR-MARTINEAU A.The relativistic discriminator: a key element missing from standard GAN[C]// International Conference on Learning Representations 2019. New Orleans, LA: ICLR, 2019: 1-26.
[28]SOCHER R, PERELYGIN A, WU J, et al.Recursive deep models for semantic compositionality over a sentiment treebank[C]// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics, 2013: 1631-1642.
[29]McAULEY J, TARGETT C, SHI Q F, et al. Image-based recommendations on styles and substitutes[C]// Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York, NY: ACM Press, 2015: 43-52. DOI: 10.1145/2766462.2767755.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Category Text Generation Based on Generative Adversarial Network

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 5

Metrics

Comments

Recommended 0

[1]	PENG Tao, TANG Jing, HE Kai, HU Xinrong, LIU Junping, HE Ruhan. Emotion Recognition Based on Multi-gait Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 104-111.
[2]	XUE Qiwei, WU Xiru. Vehicle Detection for Autonomous Vehicle System Based on Multi-modal Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(2): 37-48.
[3]	ZHANG Weibin, WU Jun, YI Jianbing. Research on Feature Fusion Controlled Items Detection Algorithm Based on RFB Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 34-46.
[4]	BAI Jie, GAO Haili, WANG Yongzhong, YANG Laibang, XIANG Xiaohang, LOU Xiongwei. Detection of Students’ Classroom Performance Based on Faster R-CNN and Transfer Learning with Multi-Channel Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(5): 1-11.
[5]	ZHANG Canlong, LI Yanru, LI Zhixin, WANG Zhiwen. Block Target Tracking Based on Kernel Correlation Filter and Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(5): 12-23.