Journal of Guangxi Normal University(Natural Science Edition) ›› 2023, Vol. 41 ›› Issue (2): 76-85.doi: 10.16088/j.issn.1001-6600.2022091102

Previous Articles     Next Articles

Stacked Capsule Autoencoders Optimization Algorithm Based on Manifold Regularization

WANG Luna1, DU Hongbo1*, ZHU Lijun2   

  1. 1. School of Science, Shenyang University of Technology, Shenyang Liaoning 110870, China;
    2. School of Information and Computing Science, Northern University for Nationalities, Yinchuan Ningxia 750021, China
  • Received:2022-09-11 Revised:2022-10-24 Online:2023-03-25 Published:2023-04-25

Abstract: To solve the problem that stacked capsule autoencoder has slow detection performance and cannot better mine local features of images, an optimization algorithm of stacked capsule autoencoders based on manifold regularization was proposed. Firstly, by using the Scharr filter to reconstruct the image in the stacked capsule autoencoders model, the accuracy of image target detection was enhanced.Then, a manifold regular term was introduced into the loss function to enhance the extraction of local features in the original data space.Finally, the stacked capsule autoencoders based on manifold regularization was used to learn parameters to select more discriminative features. Experiments results on MNIST and Fashion MNIST datasets show that compared with the original network structure, the accuracy of image classification is improved by 0.26% and 9.23%, respectively, which greatly improves the training speed of the model.

Key words: deep learning, image classification, stacked capsule autoencoders, manifold regularization, filter

CLC Number: 

  • TP391.41
[1] TAN J N, OYEKAN J.Attention augmented convolutional neural network for acoustics based machine state estimation[J]. Applied Soft Computing,2021,110: 107630.DOI: 10.1016/J.ASOC.2021.107630.
[2] 陈兵,蒋行国.卷积神经网络用于人脸特征提取[J].现代电子技术,2022,45(18): 182-186. DOI: 10.16652/j.issn.1004-373x.2022.18.035.
[3] 田晟, 宋霖.基于CNN和Bagging集成的交通标志识别[J].广西师范大学学报(自然科学版),2022, 40(4): 35-46.DOI: 10.16088/j.issn.1001-6600.2021102203.
[4] ZHAO Z J, LI X W, LIU H Z, et al. Improved target detection algorithm based on libra R-CNN[J]. IEEE Access, 2020,8: 114044-114056.DOI: 10.1109/ACCESS.2020.3002860.
[5] 王连云,陶洪峰,徐琛, 等.基于多层训练干扰的CNN轴承故障诊断[J].控制工程,2022,29(9): 1652-1657. DOI: 10.14107/j.cnki.kzgc.20200786.
[6] LI G Q, WU J, DENG C, et al. Convolutional neural network-based Bayesian Gaussian mixture for intelligent fault diagnosis of rotating machinery[J]. IEEE Transactions on Instrumentation and Measurement, 2021,70: 3517410. DOI: 10.1109/TIM.2021.3080402.
[7] GUO S,YANG T,HUA H C,et al. Coupling fault diagnosis of wind turbine gearbox based on multitask parallel convolutional neural networks with overall information[J]. Renewable Energy,2021,178: 639-650.DOI: 10.1016/j.renene.2021.06.088.
[8] 郭文娟,冯全,李相周.基于农作物病害检测与识别的卷积神经网络模型研究进展[J].中国农机化学报,2022,43(10): 157-166.DOI: 10.13733/j.jcam.issn.2095-5553.2022.10.023.
[9] 杜菲,马天兵,胡伟康,等.基于小波变换和改进卷积神经网络的刚性罐道故障诊断[J].工矿自动化,2022,48(9): 42-48,62.DOI: 10.13272/j.issn.1671-251x.17964.
[10] SABOUR S,FROSST N,HINTON G E.Dynamic routing between capsules[C]// Advances in Neural Information Processing Systems 30 (NIPS 2017). Red Hook, NY: Curran Associates, Inc., 2017: 3859-3869.
[11] HINTON G E, SABOUR S, FROSST N. Matrix capsules with EM routing[C]// International Conference on Learning Representations 2018. Vancouver: ICLR,2018: 1-15.
[12] ZHANG L H, EDRAKI M, QI G J.CapProNet: deep feature learning via orthogonal projections onto capsule subspace[C]// Advances in Neural Information Processing Systems 31 (NeurIPS 2018). Red Hook, NY: Curran Associates, Inc., 2018: 5819-5828.
[13] SHAHROUDNEJAD A, AFSHAR P, PLATANIOTIS K N. et al. Improved explainability of capsule networks: relevance path by agreement[C]// 2018 IEEE Global Conference on Signal and Information Processing(GlobalSIP). Piscataway, NJ: IEEE, 2018: 549-553. DOI: 10.1109/GlobalSIP.2018.8646474.
[14] DELIÈGE A, CIOPPA A, VAN DROOGENBROECK M. Hitnet: a neural network with capsules embedded in a Hit-or-Miss layer, extended with hybrid data augmentation and ghost capsules[EB/OL].(2018-06-18)[2022-09-11]. https://arxiv.org/abs/1806.06519. DOI: 10.48550/arXiv.1806.06519.
[15] DUARTE K, RAWAT Y S,SHAH M. VideoCapsuleNet: a simplified network for action detection[C]// Advances in Neural Information Processing Systems 31 (NeurIPS 2018). Red Hook, NY: Curran Associates, Inc., 2018: 7621-7630.
[16] GUGGLBERGER J,PEER D,RODRÍGUEZ-SÁNCHEZ A.Training deep capsule networks with residual connections[C]// Artificial Neural Networks and Machine Learning - ICANN 2021: LNCS Volume 12891. Cham: Springer Nature Switzerland AG, 2021: 541-552. DOI: 10.1007/978-3-030-86362-3_44.
[17] 曾岚蔚,许青林.基于深度卷积注意胶囊网络的微表情识别方法[J].计算机工程与设计,2022,43(9): 2631-2637. DOI: 10.16208/j.issn1000-7024.2022.09.030.
[18] 张凌慷,仝明磊.基于谱聚类胶囊网络的文本分类方法[J].科技创新与应用,2022,12(4): 16-19. DOI: 10.19981/j.CN23-1581/G3.2022.04.004.
[19] 暴雨轩. 基于深度学习的伪造人脸视频检测方法研究[D].北京: 中国人民公安大学,2022. DOI: 10.27634/d.cnki.gzrgu.2022.000196.
[20] SAHU S K, KUMAR P, SINGH A P. Dynamic routing using inter capsule routing protocol between capsules[C]// 2018 UKSim-AMSS 20th International Conference on Computer Modelling and Simulation(UKSim).Los Alamitos, CA: IEEE Computer Society, 2018: 1-5.DOI: 10.1109/UKSim.2018.00012.
[21] 林凯迪,杜洪波,朱立军.基于DPC优化动态路由的胶囊网络算法[J].沈阳工程学院学报(自然科学版),2021,17(2): 61-67.DOI: 10.13888/j.cnki.jsie(ns).2021.02.012.
[22] 李冰,崔艳荣.基于动态路由规则的胶囊网络模型研究[J].电脑编程技巧与维护,2022(8): 165-167, 175. DOI: 10.16184/j.cnki.comprg.2022.08.010.
[23] 雷建云,陈楚,郑禄,等.基于改进残差网络的水稻害虫识别[J].江苏农业科学,2022,50(14): 190-198. DOI: 10.15889/j.issn.1002-1302.2022.14.027.
[24] 张贤坤,陶健伟,董梅,等.多尺度自路由胶囊网络的构建方法[J].天津科技大学学报,2022,37(3): 59-66. DOI: 10.13364/j.issn.1672-6510.20210267.
[25] KOSIOREK A R, SABOUR S, TEH Y W, et al. Stacked capsule autoencoders[C]// Advances in Neural Information Processing Systems 32 (NeurIPS 2019).Red Hook, NY: Curran Associates, Inc., 2019: 15512-15522.
[26] DAI J Z, XIONG S W. An evasion attack against stacked capsule autoencoder[J]. Algorithms, 2022, 15(2): 32.DOI: 10.3390/a15020032.
[27] XIANG C Q,WANG Z N,ZOU W B,et al.DPR-CAE: capsule autoencoder with dynamic part represention for image parsing[EB/OL].(2021-09-07)[2022-09-11].https://arxiv.org/abs/2104.14735. DOI: 10.48550/arXiv.2104.14735.
[28] HONG C Q,CHEN L,LIANG Y X,et al.Stacked capsule graph autoencoders for geometry-aware 3D head pose estimation[J].Computer Vision and Image Understanding, 2021, 208/209: 103224.DOI: 10.1016/j.cviu.2021.103224.
[29] TANG C, BIAN M R, LIU X W, et al.Unsupervised feature selection via latent representation learning and manifold regularization[J]. Neural Networks, 2019,117: 163-178.DOI: 10.1016/j.neunet.2019.04.015.
[30] LIU X W, WANG L, ZHANG J, et al. Global and local structure preservation for feature selection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(6): 1083-1095.DOI: 10.1109/TNNLS.2013.2287275.
[31] PEÑA J M,LOZANO J A,LARRAÑAGA P. An empirical comparison of four initialization methods for the K-Means algorithm[J]. Pattern Recognition Letters,1999,20(10): 1027-1040. DOI: 10.1016/S0167-8655(99)00069-0.
[32] 袁非牛,章琳,史劲亭,等.自编码神经网络理论及应用综述[J].计算机学报,2019,42(1): 203-230.DOI: 10.11897/SP.J.1016.2019.00203.
[33] YANG B,FU X, SIDIROPOULOS N D,et al.Towards K-means-friendly spaces: simultaneous deep learning and clustering[C]// Proceedings of the 34th International Conference on Machine Learning: PMLR Volume 70. Sydney: PMLR, 2017: 3861-3870.
[34] JIANG Z X, ZHENG Y, TAN H C, et al. Variational deep embedding: an unsupervised and generative approach to clustering[C]// Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17).Melbourne: International Joint Conferences on Artifical Intelligence,2017: 1965-1972. DOI: 10.24963/ijcai.2017/273.
[35] 容培盛.基于无监督深度学习的图像聚类研究[D].杭州: 杭州电子科技大学,2020. DOI: 10.27075/d.cnki.ghzdc.2020.000288.
[1] YANG Shuozhen, ZHANG Long, WANG Jianhua, ZHANG Hengyuan. Review of Sound Event Detection [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 1-18.
[2] ZHANG Weijian, BING Qichun, SHEN Fuxin, HU Yanran, GAO Peng. Travel Time Estimation Method of Urban Expressway Section [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 49-57.
[3] ZHAO Zhonghua, YAN Xiaofeng, TONG Youwei. SOC Estimation of Lithium Ion Battery Based on Adaptive Fading Extended Kalman Filter [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(1): 58-66.
[4] MA Jun. Dynamics and Model Approach for Functional Neurons [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(5): 307-323.
[5] ZHONG Hui, SONG Shuxiang, CEN Mingcan, CAI Chaobo, JIANG Pinqun, LIU Zhenyu. Differential N-path Filter Based on Sampling Calculation [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(4): 58-67.
[6] ZHANG Ping, XU Qiaozhi. Segmentation of Lung Nodules Based on Multi-receptive Field and Grouping Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 76-87.
[7] LI Yongjie, ZHOU Guihong, LIU Bo. Fusion Algorithm of Face Detection and Head Pose Estimation Based on YOLOv3 Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 95-103.
[8] MA Xinna, ZHAO Men, QI Lin. Fault Diagnosis Based on Spiking Convolution Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 112-120.
[9] WU Jun, OUYANG Aijia, ZHANG Lin. Phosphorylation Site Prediction Model Based on Multi-head Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 161-171.
[10] YAN Longchuan, LI Yan, SONG Hu, ZOU Haodong, WANG Lijun. Web Traffic Prediction Based on Prophet-DeepAR [J]. Journal of Guangxi Normal University(Natural Science Edition), 2022, 40(3): 172-184.
[11] LU Kaifeng, YANG Yilong, LI Zhi. A Web Service Classification Method Using BERT and DPCNN [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(6): 87-98.
[12] WU Kangkang, ZHU Xufei, LU Ye, ZHOU Peng, DONG Cui, DAI Qinxuan, ZHOU Runchang. LS-FIR Filter Based on Least Square Method [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 89-99.
[13] WU Lingyu, LAN Yang, XIA Haiying. Retinal Image Registration Using Convolutional Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 122-133.
[14] CHEN Wenkang, LU Shenglian, LIU Binghao, LI Guo, LIU Xiaoyu, CHEN Ming. Real-time Citrus Recognition under Orchard Environment by Improved YOLOv4 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 134-146.
[15] WU Kangkang, ZHOU Peng, LU Ye, JIANG Dan, YAN Jianghong, QIAN Zhengcheng, GONG Chuang. FIR Equalizer Based on Mini-batch Gradient Descent Method [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(4): 9-20.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] ZHOU Zhengchun. Research Progress of Complementary Sequences[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(1): 1 -16 .
[2] YANG Shuozhen, ZHANG Long, WANG Jianhua, ZHANG Hengyuan. Review of Sound Event Detection[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 1 -18 .
[3] YANG Shenglong, MU Qingchuang, ZHANG Zhihua, LIU Kui. Technical Progress in Recovery and Utilization of Spent Lithium-ion Batteries[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 19 -26 .
[4] LI Kangliang, QIU Caixiong, HE Shuang, HUANG Chunhua, WU Guanyi. Research Progress of IL-31 in Itch[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 27 -35 .
[5] LU Xumeng, NAN Xinyuan, XIA Sibo. Trajectory Tracking Control Based on Model-Free Coordinate Compensation Integral Sliding Mode Constraints[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 36 -48 .
[6] ZHANG Weijian, BING Qichun, SHEN Fuxin, HU Yanran, GAO Peng. Travel Time Estimation Method of Urban Expressway Section[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 49 -57 .
[7] YANG Xiu, WEI Duqu. Chaos Tracking Control of Permanent Magnet Synchronous Motor Based on Single State Variable[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 58 -66 .
[8] ZHAO Yuan, SONG Shuxiang, LIU Zhenyu, CEN Mingcan, CAI Chaobo, JIANG Pinqun. Design of a Novel Current-Mirror Operational Transconductance Amplifier[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 67 -75 .
[9] ZHAO Ming, LUO Qiulian, CHEN Weimeng, CHEN Jiani. Influence of Control Timing and Strength on the Spreading of Epidemic[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 86 -97 .
[10] YANG Xiufeng, FAN Jianghua. Connectedness of the Strong Efficient Solution Set for Vector Equilibrium Problems[J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 98 -105 .