一种基于BERT和DPCNN的Web服务分类方法

doi:10.16088/j.issn.1001-6600.2020111402

摘要/Abstract

摘要： Web服务是一种基于Web环境的具有自适应、自描述、模块化和互操作能力的应用程序,这些特点使其具有很好的重用性。软件重用是目前降低软件开发成本的一种颇具前景的方法,Web服务的自动分类在软件重用中起着至关重要的作用。近年来人们将机器学习技术广泛应用于服务分类,取得了一些成果。但是传统机器学习方法的性能高度取决于特征工程的质量。本文提出一种基于预训练模型BERT和深度金字塔卷积神经网络DPCNN相结合的BERT-DPCNN深度神经网络模型,该模型可以自动抽取服务描述的低级表现形式并抽象为高级特征而无需进行特征工程。为了验证该方法的有效性,本文在50个类别、10 184个真实Web服务的数据集上与传统的机器学习方法和一些深度神经网络模型进行全面比较,结果表明,本文提出的模型相比其他方法具有更高的准确率。

关键词: Web服务, 软件重用, 机器学习, 深度学习, BERT, DPCNN

Abstract: Web Services is an application based on the Web environment with self-adaptation, self-description, modularization, and interoperability. These features make it extremely reusable. Software reuse is a promising method to reduce software development costs. The automatic classification of Web services plays a vital role in software reuse. In recent years, machine learning techniques are widely used in service classification and have achieved some results. But the performance of traditional machine learning methods highly depends on the quality of feature engineering. This paper proposes a Bert DPCNN deep neural network model, which is based on the combination of Bert pre-training model and DPCNN deep pyramid convolutional neural network. This model can automatically extract low-level representations of service descriptions and abstract them into high-level features without feature engineering. In order to demonstrate the effectiveness of the proposed method, a comprehensive comparison is made with the traditional machine learning method and some deep neural network models on the datasets of 50 categories and 10 184 real Web services. The results show that the proposed model has higher accuracy rate than the other methods.

Key words: Web services, software reuse, machine learning, deep learning, BERT, DPCNN

中图分类号:

TP183

路凯峰, 杨溢龙, 李智. 一种基于BERT和DPCNN的Web服务分类方法[J]. 广西师范大学学报（自然科学版）, 2021, 39(6): 87-98.

LU Kaifeng, YANG Yilong, LI Zhi. A Web Service Classification Method Using BERT and DPCNN[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(6): 87-98.

参考文献

[1] YANG Y L, ZU Q, LIU P, et al. Microshare: privacy-preserved medical resource sharing through microservice architecture[J]. International Journal of Biological Sciences, 2018, 14(8): 907-919. DOI:10.7150/ijbs.24617.
[2] 童红霞, 曹健, 张申生, 等. 面向服务的Agent模型研究[J]. 广西师范大学学报(自然科学版), 2008, 26(2): 110-113. DOI:10.3969/j.issn.1001-6600.2008.02.028.
[3] YANG Y L, LI X S, QAMAR N, et al. Medshare: a novel hybrid cloud for medical resource sharing among autonomous healthcare providers[J]. IEEE Access, 2018, 6: 46949-46961. DOI:10.1109/ACCESS.2018.2865535.
[4] HAJLAOUI J E, OMRI M N, BENSLIMANE D, et al. QoS based framework for configurable IaaS cloud services discovery[C]//2017 IEEE International Conference on Web Services (ICWS). Los Alamitos, CA: IEEE Computer Society, 2017: 460-467. DOI:10.1109/ICWS.2017.53.
[5] HASSELMEYER P. On service discovery process types[C]//Service-Oriented Computing-ICSOC 2005: LNCS Volume 3826. Berlin: Springer, 2005: 144-156. DOI:10.1007/11596141_1.
[6] CHENG B, ZHAO S, LI C B, et al. MISDA: web services discovery approach based on mining interface semantics[C]//2016 IEEE International Conference on Web Services (ICWS). Los Alamitos, CA: IEEE Computer Society, 2016: 332-339. DOI:10.1109/ICWS.2016.50.
[7] YANG Y L, YANG J, LI X S, et al. An integrated framework for semantic service composition using answer set programming[J]. International Journal of Web Services Research, 2014, 11(4): 47-61. DOI:10.4018/IJWSR. 2014100104.
[8] FANG L, WANG L J, LI M, et al. Towards automatic tagging for web services[C]//2012 IEEE 19th International Conference on Web Services. Los Alamitos, CA: IEEE Computer Society, 2012: 528-535. DOI:10.1109/ICWS.2012.99.
[9] KATAKIS I, MEDITSKOS G, TSOUMAKAS G, et al. On the combination of textual and semantic descriptions for automated semantic web service classification[C]//Artificial Intelligence Applications and Innovations III: Proceedings of the 5^TH IFIP Conference on Artificial Intelligence Applications and Innovations (AIAI’2009). Boston, MA: Springer, 2009: 95-104. DOI:10.1007/978-1-4419-0221-4_13.
[10] WANG H B, SHI Y Q, ZHOU X, et al. Web service classification using support vector machine[C]//2010 22nd IEEE International Conference on Tools with Artificial Intelligence: Volume 1. Los Alamitos, CA: IEEE Computer Society, 2010: 3-6. DOI:10.1109/ICTAI.2010.9.
[11] SU J, SAYYAD-SHIRABAD J, MATWIN S. Large scale text classification using semi-supervised multinomial naive bayes[C]//Proceedings of the 28th International Conference on International Conference on Machine Learning. Madison, WI: Omnipress, 2011: 97-104.
[12] ZHANG P Y, SHU S, ZHOU M C. An online fault detection model and strategies based on SVM-grid in clouds[J]. IEEE/CAA Journal of Automatica Sinica, 2018, 5(2): 445-456. DOI:10.1109/JAS.2017.7510817.
[13] ALMALAWI A M, FAHAD A, TARI Z, et al. kNNVWC: an efficient k-nearest neighbors approach based on various-widths clustering[J]. IEEE Transactions on Knowledge and Data Engineering, 2016, 28(1): 68-81. DOI:10.1109/TKDE. 2015.2460735.
[14] MALIAH S, SHANI G. MDP-based cost sensitive classification using decision trees[C]//The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18). Palo Alto, CA: AAAI Press, 2018: 3746-3753.
[15] KAPITSAKI G M. Annotating web service sections with combined classification[C]//2014 IEEE International Conference on Web Services. Los Alamitos, CA: IEEE Computer Society, 2014: 622-629. DOI:10.1109/ICWS.2014.92.
[16] LIU X M, AGARWAL S, DING C, et al. An LDA-SVM active learning framework for web service classification[C]//2016 IEEE International Conference on Web Services (ICWS). Los Alamitos, CA: IEEE Computer Society, 2016: 49-56. DOI:10.1109/ICWS.2016.16.
[17] CONG Y L, CHEN B, LIU H W, et al. Deep latent Dirichlet allocation with topic-layer-adaptive stochastic gradient Riemannian MCMC[EB/OL]. (2017-06-06)[2020-11-14]. https://arxiv.org/pdf/1706.01724.
[18] LIANG T T, CHEN L, WU J, et al. Exploiting heterogeneous information for tag recommendation in API management[C]//2016 IEEE International Conference on Web Services (ICWS). Los Alamitos, CA: IEEE Computer Society, 2016: 436-443. DOI:10.1109/ICWS.2016.63.
[19] SHI W S, LIU X M, YU Q. Correlation-aware multi-label active learning for web service tag recommendation[C]//2017 IEEE International Conference on Web Services (ICWS). Los Alamitos, CA: IEEE Computer Society, 2017: 229-236. DOI:10.1109/ICWS.2017.37.
[20] 陈婷, 刘建勋, 曹步清, 等. 基于BTM主题模型的Web服务聚类方法研究[J]. 计算机工程与科学, 2018, 40(10): 1737-1745. DOI:10.3969/j.issn.1007-130X.2018.10.004.
[21] YAN X H, GUO J F, LAN Y Y, et al. A biterm topic model for short texts[C]//WWW′13: Proceedings of the 22nd International Conference on World Wide Web. New York, NY: Association for Computing Machinery, 2013: 1445-1456. DOI:10.1145/2488388.2488514.
[22] 石敏, 刘建勋, 周栋, 等. 基于多重关系主题模型的Web服务聚类方法[J]. 计算机学报, 2019, 42(4): 820-836. DOI:10.11897/SP.J.1016.2019.00820.
[23] 曹步清, 肖巧翔, 张祥平, 等. 融合SOM功能聚类与DeepFM质量预测的API服务推荐方法[J]. 计算机学报, 2019, 42(6): 1367-1383. DOI:10.11897/SP.J.1016.2019.01367.
[24] KOHONEN T. The self-organizing map[J]. Proceedings of the IEEE, 1990, 78(9): 1464-1480. DOI:10.1109/5.58325.
[25] BENGIO Y, COURVILLE A, VINCENT P. Representation learning: a review and new perspectives[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(8): 1798-1828. DOI:10.1109/TPAMI.2013.50.
[26] GOODFELLOWI, BENGIO Y, COURVILLE A. Deep learning[M]. Cambridge, MA: MIT Press, 2016.
[27] YANG Y L, KE W, WANG W R, et al. Deep learning for web services classification[C]//2019 IEEE International Conference on Web Services (ICWS). Los Alamitos, CA: IEEE Computer Society, 2019: 440-442. DOI:10.1109/ICWS. 2019.00079.
[28] 岳天驰, 张绍武, 杨亮, 等. 基于两阶段注意力机制的立场检测方法[J]. 广西师范大学学报(自然科学版), 2019, 37(1): 42-49. DOI:10.16088/j.issn.1001-6600.2019.01.005.
[29] 武文雅, 陈钰枫, 徐金安, 等. 基于高层语义注意力机制的中文实体关系抽取[J]. 广西师范大学学报(自然科学版), 2019, 37(1): 32-41. DOI:10.16088/j.issn.1001-6600.2019.01.004.
[30] 刘恒讯, 艾中良. 一种基于词向量的恶意代码分类模型[J]. 电子设计工程, 2020, 28(6): 10-16. DOI:10.14022/j.issn1674-6236.2020.06.003.
[31] DEVLIN J, CHANG M W, LEE K, et al. Bert: pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018-10-11)[2020-11-14]. https://arxiv.org/pdf/1810.04805.
[32] JOHNSON R, ZHANG T. Deep pyramid convolutional neural networks for text categorization[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: Association for Computational Linguistics, 2017: 562-570. DOI:10.18653/v1/P17-1052.
[33] HAGHIRI S, GARREAU D, VON LUXBURG U. Comparison-based random forests[EB/OL]. (2018-06-18)[2020-11-14]. https://arxiv.org/pdf/1806.06616.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed