基于流形正则的堆叠胶囊自编码器优化算法

doi:10.16088/j.issn.1001-6600.2022091102

摘要/Abstract

摘要： 针对堆叠胶囊自编码器存在检测性能慢、不能更好挖掘图像局部特征的问题,本文提出基于流形正则的堆叠胶囊自编码器优化算法。采用Scharr滤波器对堆叠胶囊自编码器模型中的图像进行重建,加强图像目标检测的精度,并在损失函数中引入流形正则项,从而加强对原始数据空间局部特征的提取,最终使用基于流形正则的堆叠胶囊自编码器学习参数,选择出更加具有区别性的特征。在MNIST和Fashion MNIST数据集上的实验结果显示,该优化算法相比于原网络结构,图像分类准确率分别提高了0.26和9.23个百分点,且模型训练速度也得到较大提高。

关键词: 深度学习, 图像分类, 堆叠胶囊自编码器, 流形正则, 滤波器

Abstract: To solve the problem that stacked capsule autoencoder has slow detection performance and cannot better mine local features of images, an optimization algorithm of stacked capsule autoencoders based on manifold regularization was proposed. Firstly, by using the Scharr filter to reconstruct the image in the stacked capsule autoencoders model, the accuracy of image target detection was enhanced.Then, a manifold regular term was introduced into the loss function to enhance the extraction of local features in the original data space.Finally, the stacked capsule autoencoders based on manifold regularization was used to learn parameters to select more discriminative features. Experiments results on MNIST and Fashion MNIST datasets show that compared with the original network structure, the accuracy of image classification is improved by 0.26% and 9.23%, respectively, which greatly improves the training speed of the model.

Key words: deep learning, image classification, stacked capsule autoencoders, manifold regularization, filter

中图分类号:

TP391.41

王鲁娜, 杜洪波, 朱立军. 基于流形正则的堆叠胶囊自编码器优化算法[J]. 广西师范大学学报（自然科学版）, 2023, 41(2): 76-85.

WANG Luna, DU Hongbo, ZHU Lijun. Stacked Capsule Autoencoders Optimization Algorithm Based on Manifold Regularization[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 76-85.

参考文献

[1] TAN J N, OYEKAN J.Attention augmented convolutional neural network for acoustics based machine state estimation[J]. Applied Soft Computing,2021,110: 107630.DOI: 10.1016/J.ASOC.2021.107630.
[2] 陈兵,蒋行国.卷积神经网络用于人脸特征提取[J].现代电子技术,2022,45(18): 182-186. DOI: 10.16652/j.issn.1004-373x.2022.18.035.
[3] 田晟, 宋霖.基于CNN和Bagging集成的交通标志识别[J].广西师范大学学报(自然科学版),2022, 40(4): 35-46.DOI: 10.16088/j.issn.1001-6600.2021102203.
[4] ZHAO Z J, LI X W, LIU H Z, et al. Improved target detection algorithm based on libra R-CNN[J]. IEEE Access, 2020,8: 114044-114056.DOI: 10.1109/ACCESS.2020.3002860.
[5] 王连云,陶洪峰,徐琛, 等.基于多层训练干扰的CNN轴承故障诊断[J].控制工程,2022,29(9): 1652-1657. DOI: 10.14107/j.cnki.kzgc.20200786.
[6] LI G Q, WU J, DENG C, et al. Convolutional neural network-based Bayesian Gaussian mixture for intelligent fault diagnosis of rotating machinery[J]. IEEE Transactions on Instrumentation and Measurement, 2021,70: 3517410. DOI: 10.1109/TIM.2021.3080402.
[7] GUO S,YANG T,HUA H C,et al. Coupling fault diagnosis of wind turbine gearbox based on multitask parallel convolutional neural networks with overall information[J]. Renewable Energy,2021,178: 639-650.DOI: 10.1016/j.renene.2021.06.088.
[8] 郭文娟,冯全,李相周.基于农作物病害检测与识别的卷积神经网络模型研究进展[J].中国农机化学报,2022,43(10): 157-166.DOI: 10.13733/j.jcam.issn.2095-5553.2022.10.023.
[9] 杜菲,马天兵,胡伟康,等.基于小波变换和改进卷积神经网络的刚性罐道故障诊断[J].工矿自动化,2022,48(9): 42-48,62.DOI: 10.13272/j.issn.1671-251x.17964.
[10] SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]// Advances in Neural Information Processing Systems 30 (NIPS 2017). Red Hook, NY: Curran Associates, Inc., 2017: 3859-3869.
[11] HINTON G E, SABOUR S, FROSST N. Matrix capsules with EM routing[C]// International Conference on Learning Representations 2018. Vancouver: ICLR,2018: 1-15.
[12] ZHANG L H, EDRAKI M, QI G J.CapProNet: deep feature learning via orthogonal projections onto capsule subspace[C]// Advances in Neural Information Processing Systems 31 (NeurIPS 2018). Red Hook, NY: Curran Associates, Inc., 2018: 5819-5828.
[13] SHAHROUDNEJAD A, AFSHAR P, PLATANIOTIS K N. et al. Improved explainability of capsule networks: relevance path by agreement[C]// 2018 IEEE Global Conference on Signal and Information Processing(GlobalSIP). Piscataway, NJ: IEEE, 2018: 549-553. DOI: 10.1109/GlobalSIP.2018.8646474.
[14] DELIÈGE A, CIOPPA A, VAN DROOGENBROECK M. Hitnet: a neural network with capsules embedded in a Hit-or-Miss layer, extended with hybrid data augmentation and ghost capsules[EB/OL].(2018-06-18)[2022-09-11]. https://arxiv.org/abs/1806.06519. DOI: 10.48550/arXiv.1806.06519.
[15] DUARTE K, RAWAT Y S,SHAH M. VideoCapsuleNet: a simplified network for action detection[C]// Advances in Neural Information Processing Systems 31 (NeurIPS 2018). Red Hook, NY: Curran Associates, Inc., 2018: 7621-7630.
[16] GUGGLBERGER J,PEER D,RODRÍGUEZ-SÁNCHEZ A.Training deep capsule networks with residual connections[C]// Artificial Neural Networks and Machine Learning - ICANN 2021: LNCS Volume 12891. Cham: Springer Nature Switzerland AG, 2021: 541-552. DOI: 10.1007/978-3-030-86362-3_44.
[17] 曾岚蔚,许青林.基于深度卷积注意胶囊网络的微表情识别方法[J].计算机工程与设计,2022,43(9): 2631-2637. DOI: 10.16208/j.issn1000-7024.2022.09.030.
[18] 张凌慷,仝明磊.基于谱聚类胶囊网络的文本分类方法[J].科技创新与应用,2022,12(4): 16-19. DOI: 10.19981/j.CN23-1581/G3.2022.04.004.
[19] 暴雨轩. 基于深度学习的伪造人脸视频检测方法研究[D].北京: 中国人民公安大学,2022. DOI: 10.27634/d.cnki.gzrgu.2022.000196.
[20] SAHU S K, KUMAR P, SINGH A P. Dynamic routing using inter capsule routing protocol between capsules[C]// 2018 UKSim-AMSS 20th International Conference on Computer Modelling and Simulation(UKSim).Los Alamitos, CA: IEEE Computer Society, 2018: 1-5.DOI: 10.1109/UKSim.2018.00012.
[21] 林凯迪,杜洪波,朱立军.基于DPC优化动态路由的胶囊网络算法[J].沈阳工程学院学报(自然科学版),2021,17(2): 61-67.DOI: 10.13888/j.cnki.jsie(ns).2021.02.012.
[22] 李冰,崔艳荣.基于动态路由规则的胶囊网络模型研究[J].电脑编程技巧与维护,2022(8): 165-167, 175. DOI: 10.16184/j.cnki.comprg.2022.08.010.
[23] 雷建云,陈楚,郑禄,等.基于改进残差网络的水稻害虫识别[J].江苏农业科学,2022,50(14): 190-198. DOI: 10.15889/j.issn.1002-1302.2022.14.027.
[24] 张贤坤,陶健伟,董梅,等.多尺度自路由胶囊网络的构建方法[J].天津科技大学学报,2022,37(3): 59-66. DOI: 10.13364/j.issn.1672-6510.20210267.
[25] KOSIOREK A R, SABOUR S, TEH Y W, et al. Stacked capsule autoencoders[C]// Advances in Neural Information Processing Systems 32 (NeurIPS 2019).Red Hook, NY: Curran Associates, Inc., 2019: 15512-15522.
[26] DAI J Z, XIONG S W. An evasion attack against stacked capsule autoencoder[J]. Algorithms, 2022, 15(2): 32.DOI: 10.3390/a15020032.
[27] XIANG C Q,WANG Z N,ZOU W B,et al.DPR-CAE: capsule autoencoder with dynamic part represention for image parsing[EB/OL].(2021-09-07)[2022-09-11].https://arxiv.org/abs/2104.14735. DOI: 10.48550/arXiv.2104.14735.
[28] HONG C Q,CHEN L,LIANG Y X,et al.Stacked capsule graph autoencoders for geometry-aware 3D head pose estimation[J].Computer Vision and Image Understanding, 2021, 208/209: 103224.DOI: 10.1016/j.cviu.2021.103224.
[29] TANG C, BIAN M R, LIU X W, et al.Unsupervised feature selection via latent representation learning and manifold regularization[J]. Neural Networks, 2019,117: 163-178.DOI: 10.1016/j.neunet.2019.04.015.
[30] LIU X W, WANG L, ZHANG J, et al. Global and local structure preservation for feature selection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(6): 1083-1095.DOI: 10.1109/TNNLS.2013.2287275.
[31] PEÑA J M,LOZANO J A,LARRAÑAGA P. An empirical comparison of four initialization methods for the K-Means algorithm[J]. Pattern Recognition Letters,1999,20(10): 1027-1040. DOI: 10.1016/S0167-8655(99)00069-0.
[32] 袁非牛,章琳,史劲亭,等.自编码神经网络理论及应用综述[J].计算机学报,2019,42(1): 203-230.DOI: 10.11897/SP.J.1016.2019.00203.
[33] YANG B,FU X, SIDIROPOULOS N D,et al.Towards K-means-friendly spaces: simultaneous deep learning and clustering[C]// Proceedings of the 34th International Conference on Machine Learning: PMLR Volume 70. Sydney: PMLR, 2017: 3861-3870.
[34] JIANG Z X, ZHENG Y, TAN H C, et al. Variational deep embedding: an unsupervised and generative approach to clustering[C]// Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17).Melbourne: International Joint Conferences on Artifical Intelligence,2017: 1965-1972. DOI: 10.24963/ijcai.2017/273.
[35] 容培盛.基于无监督深度学习的图像聚类研究[D].杭州: 杭州电子科技大学,2020. DOI: 10.27075/d.cnki.ghzdc.2020.000288.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed