基于复数协方差卷积神经网络的运动想象脑电信号解码方法

doi:10.16088/j.issn.1001-6600.2024092401

Abstract

Abstract: To improve the classification performance of motor imagery (MI) tasks by deeply mining and using the characteristic information of electroencephalogram (EEG) signals has always been the focus of brain-computer interfaces (BCI) research. Because EEG feature space is highly dimensional and directly related to both amplitude and phase of EEG signals, how to simultaneously represent and utilize the information contained in amplitude and phase has become a difficult issue. To address this issue, a three-dimensional complex convolutional neural network based on complex-value covariance features is proposed. Firstly, complex-value covariance matrices related to different frequencies as MI-based EEG features is constructed. As a result, complex value can combine the amplitude and phase information of EEG signals together. Moreover, the covariance matrices can preserve multivariate information such as amplitude, phase, spatial locations, frequency, etc. required for classification. Secondly, a full complex convolutional neural network is designed for learning the covariance features and thus achieving high performance classification. Experimental results on two publicly available datasets show that the proposed method can achieve mean accuracies that are at least 2.49 and 1.85 percentage points higher than state-of-the-art methods.

Key words: electroencephalogram, brain-computer interface, fusion of amplitude and phase information, complex covariance features, complex-valued convolutional neural network, information interactive

CLC Number: TN911.7

HUANG Renhui, ZHANG Ruifeng, WEN Xiaohao, BI Jinjie, HUANG Shoulin, LI Tinghui. Complex-value Covariance-based Convolutional Neural Network for Decoding Motor Imagery-based EEG Signals[J].Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 43-56.

References

[1] LAZAROU I, NIKOLOPOULOS S, PETRANTONAKIS P C, et al. EEG-based brain-computer interfaces for communication and rehabilitation of people with motor impairment: a novel approach of the 21st Century[J]. Frontiers in Human Neuroscience, 2018, 12: 14. DOI: 10.3389/fnhum.2018.00014.
[2]邱爽,杨帮华,陈小刚,等.非侵入式脑—机接口编解码技术研究进展[J].中国图象图形学报,2023,28(6):1543-1566. DOI: 10.11834/jig.230031.
[3]ZHANG J C, WANG M. A survey on robots controlled by motor imagery brain-computer interfaces[J]. Cognitive Robotics, 2021, 1: 12-24. DOI: 10.1016/j.cogr.2021.02.001.
[4]PFURTSCHELLER G, LOPES DA SILVA F H. Event-related EEG/MEG synchronization and desynchronization: basic principles[J]. Clinical Neurophysiology, 1999, 110(11): 1842-1857. DOI: 10.1016/S1388-2457(99)00141-8.
[5]ALTAHERI H, MUHAMMAD G, ALSULAIMAN M, et al. Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: a review[J]. Neural Computing and Applications, 2023, 35(20): 14681-14722. DOI: 10.1007/s00521-021-06352-5.
[6]BLANKERTZ B, TOMIOKA R, LEMM S, et al. Optimizing spatial filters for robust EEG single-trial analysis[J]. IEEE Signal Processing Magazine, 2008, 25(1): 41-56. DOI: 10.1109/MSP.2008.4408441.
[7]潘林聪,王坤,许敏鹏,等.面向运动意图解码的共空间模式及其扩展算法研究综述[J].中国生物医学工程学报,2022,41(5):577-588. DOI: 10.3969/j.issn.0258-8021.2022.05.007.
[8]ANG K K, CHIN Z Y, ZHANG H H, et al. Filter bank common spatial pattern (FBCSP) in brain-computer interface[C]// 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). Los Alamitos, CA: IEEE Computer Society, 2008: 2390-2397. DOI: 10.1109/IJCNN.2008.4634130.
[9]KOLDA T G, BADER B W. Tensor decompositions and applications[J]. SIAM Review, 2009, 51(3): 455-500. DOI: 10.1137/07070111X.
[10]BARACHANT A, BONNET S, CONGEDO M, et al. Riemannian geometry applied to BCI classification[C]// Latent Variable Analysis and Signal Separation. Berlin: Springer, 2010: 629-636. DOI: 10.1007/978-3-642-15995-4_78.
[11]徐慧,何宏,张慧敏,等.黎曼流形切平面空间中运动想象脑电信号的迁移学习[J].中国生物医学工程学报,2023,42(6):659-667. DOI: 10.3969/j.issn.0258-8021.2023.06.003.
[12]ROSIPAL R, ROŠTŤÁKOVÁ Z, TREJO L J. Tensor decomposition of human narrowband oscillatory brain activity in frequency, space and time[J]. Biological Psychology, 2022, 169: 108287. DOI: 10.1016/j.biopsycho.2022.108287.
[13]黄守麟.面向运动想象脑电信号的张量空频分析方法研究[D].哈尔滨:哈尔滨工业大学,2021. DOI: 10.27061/d.cnki.ghgdu.2021.004962.
[14]TANG X L, YANG C Q, SUN X, et al. Motor imagery EEG decoding based on multi-scale hybrid networks and feature enhancement[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 1208-1218. DOI: 10.1109/TNSRE.2023.3242280.
[15]SCHIRRMEISTER R T, SPRINGENBERG J T, FIEDERER L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization[J]. Human Brain Mapping, 2017, 38(11): 5391-5420. DOI: 10.1002/hbm.23730.
[16]LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[J]. Journal of Neural Engineering, 2018, 15(5): 056013. DOI: 10.1088/1741-2552/aace8c.
[17]MANE R, ROBINSON N, VINOD A P, et al. A multi-view CNN with novel variance layer for motor imagery brain computer interface[C]// 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Los Alamitos, CA: IEEE Computer Society, 2020: 2950-2953. DOI: 10.1109/EMBC44109.2020.9175874.
[18]BANG J S, LEE M H, FAZLI S, et al. Spatio-spectral feature representation for motor imagery classification using convolutional neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022, 33(7): 3038-3049. DOI: 10.1109/TNNLS.2020.3048385.
[19]SAUSENG P, KLIMESCH W. What does phase information of oscillatory brain activity tell us about cognitive processes?[J]. Neuroscience & Biobehavioral Reviews, 2008, 32(5): 1001-1013. DOI: 10.1016/j.neubiorev.2008.03.014.
[20]HELFRICH R F, KNEPPER H, NOLTE G, et al. Spectral fingerprints of large-scale cortical dynamics during ambiguous motion perception[J]. Human Brain Mapping, 2016, 37(11): 4099-4111. DOI: 10.1002/hbm.23298.
[21]POURALI H, OMRANPOUR H. CSP-Ph-PS: learning CSP-phase space and poincare sections based on evolutionary algorithm for EEG signals recognition[J]. Expert Systems with Applications, 2023, 211: 118621. DOI: 10.1016/j.eswa.2022.118621.
[22]LACHAUX J P, RODRIGUEZ E, MARTINERIE J, et al. Measuring phase synchrony in brain signals[J]. Human Brain Mapping, 1999, 8(4): 194-208. DOI: 10.1002/(SICI)1097-0193(1999)8:4<194::AID-HBM4>3.0.CO;2-C.
[23]CHEN J C, CUI Y G, WANG H, et al. Deep learning approach for detection of unfavorable driving state based on multiple phase synchronization between multi-channel EEG signals[J]. Information Sciences, 2024, 658: 120070. DOI: 10.1016/j.ins.2023.120070.
[24]钟文潇,安兴伟,狄洋,等.基于脑电信号的身份特征提取方法研究综述[J].生物医学工程学杂志,2021,38(6):1203-1210. DOI: 10.7507/1001-5515.202102057.
[25]XU G Y, WANG Z Y, XU T H, et al. Engagement recognition using a multi-domain feature extraction method based on correlation-based common spatial patterns[J]. Applied Sciences, 2023, 13(21): 11924. DOI: 10.3390/app132111924.
[26]FAN C C, YANG B H, LI X O, et al. Temporal-frequency-phase feature classification using 3D-convolutional neural networks for motor imagery and movement[J]. Frontiers in Neuroscience, 2023, 17: 1250991. DOI: 10.3389/fnins.2023.1250991.
[27]CHAKRABORTY B, GHOSH L, KONAR A. Designing phase-sensitive common spatial pattern filter to improve brain-computer interfacing[J]. IEEE Transactions on Bio-Medical Engineering, 2020, 67(7): 2064-2072. DOI: 10.1109/tbme.2019.2954470.
[28]HUANG S L, CAI G Q, WANG T, et al. Amplitude-phase information measurement on Riemannian manifold for motor imagery-based BCI[J]. IEEE Signal Processing Letters, 2021, 28: 1310-1314. DOI: 10.1109/LSP.2021.3087099.
[29]TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3D convolutional networks[C]// 2015 IEEE International Conference on Computer Vision (ICCV). Los Alamitos, CA: IEEE Computer Society, 2015: 4489-4497. DOI: 10.1109/ICCV.2015.510.
[30]HO J, SALIMANS T, GRITSENKO A, et al. Video diffusion models[C]// Advances in Neural Information Processing Systems 35 (NeurIPS 2022). Red Hook: Curran Associates, Inc., 2022: 8633-8646.
[31]JAMIOŁKOWSKI A. Linear transformations which preserve trace and positive semidefiniteness of operators[J]. Reports on Mathematical Physics, 1972, 3(4): 275-278. DOI: 10.1016/0034-4877(72)90011-0.
[32]LEE C Y, HASEGAWA H, GAO S C. Complex-valued neural networks: a comprehensive survey[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(8): 1406-1426. DOI: 10.1109/JAS.2022.105743.
[33]GOLIK P, DOETSCH P, NEY H. Cross-entropy vs. squared error training: a theoretical and experimental comparison[C]// Proceedings of the Interspeech 2013. Baixas: International Speech Communications Association, 2013: 1756-1760. DOI: 10.21437/Interspeech.2013-436.
[34]SHI G J, SHANECHI M M, AARABI P. On the importance of phase in human speech recognition[J]. IEEE Transactions on Audio Speech and Language Processing, 2006, 14(5): 1867-1874. DOI: 10.1109/TSA.2005.858512.
[35]ARJOVSKY M, SHAH A, BENGIO Y. Unitary evolution recurrent neural networks[C]// Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York: JMLR, 2016: 1120-1128. DOI: 10.5555/3045390.3045509.
[36]TRABELSI C, BILANIUK O, ZHANG Y, et al. Deep complex networks[EB/OL]. (2018-02-25)[2024-09-01]. https://arxiv.org/abs/1705.09792. DOI: 10.48550/arXiv.1705.09792.
[37]HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2016: 770-778. DOI: 10.1109/CVPR.2016.90.
[38]KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. (2017-01-30)[2024-09-01]. https://arxiv.org/abs/1412.6980. DOI: 10.48550/arXiv.1412.6980.
[39]TANGERMANN M, MÜLLER K R, AERTSEN A, et al. Review of the BCI competition IV[J]. Frontiers in Neuroscience, 2012, 6: 55. DOI: 10.3389/fnins.2012.00055.
[40]LEE M H, KWON O Y, KIM Y J, et al. EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy[J]. GigaScience, 2019, 8(5): giz002. DOI: 10.1093/gigascience/giz002.
[41]BLANKERTZ B, LEMM S, TREDER M, et al. Single-trial analysis and classification of ERP components: a tutorial[J]. NeuroImage, 2011, 56(2): 814-825. DOI: 10.1016/j.neuroimage.2010.06.048.
[42]JAMIESON A R, GIGER M L, DRUKKER K, et al. Exploring nonlinear feature space dimension reduction and data representation in breast CADx with Laplacian eigenmaps and t-SNE[J]. Medical Physics, 2010, 37(1): 339-351. DOI: 10.1118/1.3267037.
[43]DAVIES D L, BOULDIN D W. A cluster separation measure[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1979, PAMI-1(2): 224-227. DOI: 10.1109/TPAMI.1979.4766909.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Complex-value Covariance-based Convolutional Neural Network for Decoding Motor Imagery-based EEG Signals

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 4

Metrics

Comments

Recommended 0

[1]	TANG Liang, CHEN Bowen, NIU Yisen, MA Ronggeng. A YOLOv8-based Real-time Object Detection Method for Vehicles and Pedestrians in Foggy Weather [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 72-83.
[2]	LIANG Yinjie, NAN Xinyuan, CAI Xin, LI Yunpeng, GOU Haiguang. Bridge Defect Detection Based on Data Augmentation and Improved YOLOv8 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2025, 43(3): 84-97.
[3]	WANG Kai-ming, ZHOU Hai-yan, GUO Jia-liang, YANG Xiao-jing, WANG Gang, ZHONG Ning. Analysis of Depression Electroencephalogram Basedon Statistics Distribution Entropy [J]. Journal of Guangxi Normal University(Natural Science Edition), 2015, 33(2): 29-35.
[4]	YANG Hong, XIA Bin, XIE Hong, WANG Cong, XIAO Dian-yun. A Grouping Strategy of Motor Imagery for Multi-commands Output [J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(3): 179-182.