Journal of Guangxi Normal University(Natural Science Edition) ›› 2024, Vol. 42 ›› Issue (6): 138-148.doi: 10.16088/j.issn.1001-6600.2023112501

Previous Articles     Next Articles

Multi-scale Attention Learning for Abdomen Multi-organ Image Segmentation

LU Jiahui1, CHEN Qingfeng1*, WANG Wenguang2, YU Qian1, HE Naixu1, HAN Zongzhao1   

  1. 1. School of Computer, Electronics and Information, Guangxi University, Nanning Guangxi 530004, China;
    2. Guilin Branch of Guangxi Zhuang Autonomous Region Tobacco Company, Guilin Guangxi 541004, China
  • Received:2023-11-25 Revised:2024-04-07 Online:2024-12-30 Published:2024-12-30

Abstract: Image segmentation technology is an important branch in the field of medical image research, and this technology helps doctors diagnose and treat cancer. In order to further improve the accuracy of image segmentation, a multi-scale axial attention model MAU-Net (multi-scale axial attention U-Net) is proposed in this paper for organ segmentation. Firstly, the model uses a deep residual network to extract image features in the encoder stage to improve the model’s generalization ability. Secondly, a pixel fusion module (PFM) is added to the decoder to enhance the ability to extract feature position information by re-encoding and linearly enhancing the feature information of the encoder. Finally, a multi-branch axial attention module (MAM) is added between the decoders to capture contextual information and enhance the ability to identify key feature information. Experimental results on multiple multi-organ image data sets such as Synapse, ACDC, and SegTHOR show that MAU-Net can achieve better results in both organ recognition and edge prediction.

Key words: image segmentation, organ segmentation, attention mechanism, thoracic and abdominal organs, deep learning

CLC Number:  TP391.41
[1] 郭雯, 鞠忠建, 吴青南, 等. 基于深度学习的器官自动分割研究进展[J]. 医疗卫生装备, 2020, 41(1): 85-94. DOI: 10.19745/j.1003-8868.2020021.
[2] 施俊, 汪琳琳, 王珊珊, 等. 深度学习在医学影像中的应用综述[J]. 中国图象图形学报, 2020, 25(10): 1953-1981. DOI: 10.11834/jig.200255.
[3] 李欣, 梁森, 黄正南, 等. 乳腺X线AI智能病灶检测[J]. 放射学实践, 2018, 33(10): 1029-1032. DOI: 10.13609/j.cnki.1000-0313.2018.10.010.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTONG E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. DOI: 10.1145/3065386.
[5] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10)[2023-11-25]. https://arxiv.org/abs/1409.1556. DOI: 10.48550/arXiv.1409.1556.
[6] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2016: 770-778. DOI: 10.1109/cvpr.2016.90.
[7] SHELHAMER E,LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651. DOI: 10.1109/TPAMI.2016.2572683.
[8] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C] // Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: LNCS Volume 9351. Cham: Springer International Publishing Switzerland, 2015: 234-241. DOI: 10.1007/978-3-319-24574-4_28.
[9] ZHANG J W, ZHANG Y C, JIN Y Z, et al. MDU-Net: multi-scale densely connected U-Net for biomedical image segmentation[J]. Health Information Science and Systems, 2023, 11(1): 13. DOI: 10.1007/s13755-022-00204-9.
[10] ALOM M Z, HASAN M, YAKOPCIC C, et al. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation[EB/OL]. (2018-05-29)[2023-11-25]. https://arxiv.org/abs/1802.06955. DOI: 10.48550/arXiv.1802.06955.
[11] 彭璟, 罗浩宇, 赵淦森, 等. 深度学习下的医学影像分割算法综述[J]. 计算机工程与应用, 2021, 57(3): 44-57. DOI: 10.3778/j.issn.1002-8331.2010-0335.
[12] 曹玉红, 徐海, 刘荪傲, 等. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287. DOI: 10.11772/j.issn.1001-9081.2020101638.
[13] 周涛, 董雅丽, 霍兵强, 等. U-Net网络医学图像分割应用综述[J]. 中国图象图形学报, 2021, 26(9): 2058-2077. DOI: 10.11834/jig.200704.
[14] CHEN W L, ZHANG Y, HE J J, et al. Prostate segmentation using 2D bridged U-net[C] // 2019 International Joint Conference on Neural Networks (IJCNN). Piscataway, NJ: IEEE, 2019: 1-7. DOI: 10.1109/ijcnn.2019.8851908.
[15] 吴玲玉, 蓝洋, 夏海英. 基于卷积神经网络的眼底图像配准研究[J]. 广西师范大学学报(自然科学版), 2021, 39(5): 122-133. DOI: 10.16088/j.issn.1001-6600.2020122801.
[16] FU H Z, CHENG J, XU Y W, et al. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation[J]. IEEE Transactions on Medical Imaging, 2018, 37(7): 1597-1605. DOI: 10.1109/tmi.2018.2791488.
[17] MILLETARI F, NAVAB N, AHMADI S A. V-Net: fully convolutional neural networks for volumetric medical image segmentation[C] // 2016 Fourth International Conference on 3D Vision (3DV). Los Alamitos, CA: IEEE Computer Society, 2016: 565-571. DOI: 10.1109/3dv.2016.79.
[18] ZHOU Z W, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[C] // Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: LNCS Volume 11045. Cham: Springer, 2018: 3-11. DOI: 10.1007/978-3-030-00889-5_1.
[19] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: learning where to look for the pancreas[EB/OL]. (2018-05-20)[2023-11-25]. https://arxiv.org/abs/1804.03999. DOI:10.48550/arXiv.1804.03999.
[20] 万黎明, 张小乾, 刘知贵, 等. 基于高效通道注意力的UNet肺结节CT图像分割[J]. 广西师范大学学报(自然科学版), 2022, 40(3): 66-75. DOI: 10.16088/j.issn.1001-6600.2021071202.
[21] 彭宝玉, 曹立佳. 多尺度融合注意力机制改进U-Net实现肺部感染区域分割[J]. 国外电子测量技术, 2023, 42(10): 177-183. DOI: 10.19652/j.cnki.femt.2305144.
[22] 谢飞, 权媚阳, 管子玉, 等. 基于深度学习的腹部多器官图像分割[J]. 西北大学学报(自然科学版), 2021, 51(1): 1-7. DOI: 10.16152/j.cnki.xdxbzr.2021-01-001.
[23] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C] // Advances in Neural Information Processing Systems 30 (NIPS 2017). Red Hook, NY: Curran Associates Inc., 2017: 6000-6010.
[24] DOSOVITSKIY A, BEYER L, KOLESNIKOVA, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2021-06-03)[2023-11-25]. https://arxiv.org/abs/2010.11929. DOI: 10.48550/arXiv.2010.11929.
[25] CHEN J N, LU Y Y, YU Q H, et al. TransUNet: transformers make strong encoders for medical image segmentation[EB/OL]. (2021-02-08)[2023-11-25]. https://arxiv.org/abs/2102.04306. DOI: 10.48550/arXiv.2102.04306.
[26] XIE E Z, WANG W H, YU Z D, et al. SegFormer: simple and efficient design for semantic segmentation with transformers[C] // Advances in Neural Information Processing Systems 34 (NeurIPS 2021). Red Hook, NY: Curran Associates Inc., 2021: 12077-12090.
[27] HUANG X H, DENG Z F, LI D D, et al. MISSFormer: an effective medical image segmentation transformer[EB/OL]. (2021-12-19)[2023-11-25]. https://arxiv.org/abs/2109.07162. DOI: 10.48550/arXiv.2109.07162.
[28] CAO H, WANG Y Y, CHEN J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation[C] // Computer Vision-ECCV 2022 Workshops: LNCS Volume 13803. Cham: Springer Nature Switzerland AG, 2023: 205-218. DOI: 10.1007/978-3-031-25066-8_9.
[29] WANG H Y, XIE S A, LIN L F, et al. Mixed transformer U-Net for medical image segmentation[C] // ICASSP 2022: 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2022: 2390-2394. DOI: 10.1109/icassp43922.2022.9746172.
[30] 李鸿, 邹俊颖, 谭茜成, 等. 面向医学图像分割的多注意力融合网络[J]. 计算机应用, 2022, 42(12): 3891-3899. DOI: 10.11772/j.issn.1001-9081.2021101737.
[31] 卢玲, 漆为民. 基于Transformer的脊椎CT图像分割[J]. 中国图象图形学报, 2023, 28(11): 3618-3628. DOI: 10.11834/jig.221084.
[32] Sage Bionetworks. Multi-atlas labeling beyond the cranial vault: workshop and challenge[EB/OL]. (2016-09-16)[2023-11-25]. https://www.synapse.org/Synapse:syn3193805/wiki/89480. DOI: 10.7303/syn3193805.
[33] BERNARD O, LALANDE A, ZOTTI C, et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?[J]. IEEE Transactions on Medical Imaging, 2018, 37(11): 2514-2525. DOI: 10.1109/TMI.2018.2837502.
[34] LAMBERT Z, PETITJEAN C, DUBRAY B, et al. SegTHOR: segmentation of thoracic organs at risk in CT images[C] // 2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA). Piscataway, NJ: IEEE, 2020: 1-6. DOI: 10.1109/IPTA50016.2020.9286453.
[35] JI Y F, BAI H T, GE C J, et al. AMOS: a large-scale abdominal multi-organ benchmark for versatile medical image segmentation[C] // Advances in Neural Information Processing Systems 35 (NeurIPS 2022). Red Hook, NY: Curran Associates Inc., 2022: 36722-36732.
[36] HENDRYCKS D, GIMPEL K. Gaussian error linear units (GELUs)[EB/OL]. (2023-06-06)[2023-11-25]. https:// arxiv.org/abs/1606.08415. DOI: 10.48550/arXiv.1606.08415.
[37] ZHAO X Y, ZHANG P, SONG F, et al. Prior attention network for multi-lesion segmentation in medical images[J]. IEEE Transactions on Medical Imaging, 2022, 41(12): 3812-3823. DOI: 10.1109/tmi.2022.3197180.
[38] 李夏, 胡巍, 王子民. 基于Non-local的腰椎间盘突出患者多裂肌分割方法[J].桂林电子科技大学学报,2023,43(3): 217-222. DOI: 10.16725/j.cnki.cn45-1351/tn.2023.03.008.
[1] LI Xin, NING Jing. Online Assessment of Transient Stability in Power Systems Based on Spatiotemporal Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 89-100.
[2] HOU Haiyan, TAN Yumei, SONG Shuxiang, XIA Haiying. Head Pose-Robust Facial Expression Recognition [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 126-137.
[3] DU Shuaiwen, JIN Ting. A Deep Hybrid Recommendation Algorithm Based on User Behavior Characteristics [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(5): 91-100.
[4] TIAN Sheng, HU Xiao. Vehicle Trajectory Prediction Based on Transformer Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 47-58.
[5] YI Jianbing, PENG Xin, CAO Feng, LI Jun, XIE Weijia. Research on Point Cloud Registration Algorithm Based on Multi-scale Feature Fusion [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(3): 108-120.
[6] XIAO Yuting, LÜ Xiaoqi, GU Yu, LIU Chuanqiang. Classification of Diabetic Retinopathy Based on Split Residual Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 91-101.
[7] XI Lingfei, Yilihamu Yaermaimaiti, LIU Yajie. Surface Defect Detection Method for Aluminum Profile Based on Improved YOLOv5 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(1): 111-119.
[8] GAO Fei, GUO Xiaobin, YUAN Dongfang, CAO Fujun. Improved PINNs Method for Solving the Convective Dominant Diffusion Equation with Boundary Layer [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 33-50.
[9] SONG Guanwu, CHEN Zhiming, LI Jianjun. Remote Sensing Image Classification with Cascade Attention Based on ResNet-50 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(6): 80-91.
[10] WU Zhengqing, CAO Hui, LIU Baokai. Chinese Fake Review Detection Based on Attention Convolutional Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(5): 26-36.
[11] JIANG Yibo, LIU Huijia, WU Tian. Research on Identification of Lightning Overvoltage in Transmission Line by Improved Residual Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(4): 74-83.
[12] HUANG Yeqi, WANG Mingwei, YAN Rui, LEI Tao. Surface Quality Detection of Diamond Wire Based on Improved YOLOv5 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(4): 123-134.
[13] QIAN Youwei, HE Fuyun, WEI Yan, FENG Huiling, HU Cong. Neuron Image Segmentation Based on Dual Coding Path Fusion and Bidirectional ConvLSTM [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(3): 67-79.
[14] DENG Xizhen, JIANG Ming, CEN Mingcan, LUO Yuling. Ransomware Classification Based on Entropy Image Static Analysis Technology [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(3): 91-104.
[15] YANG Shuozhen, ZHANG Long, WANG Jianhua, ZHANG Hengyuan. Review of Sound Event Detection [J]. Journal of Guangxi Normal University(Natural Science Edition), 2023, 41(2): 1-18.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] ZHU Gege, HUANG Anshu, QIN Yingying. Analysis of Development Trend of International Mangrove Research Based on Web of Science[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(5): 1 -12 .
[2] HE Jing, FENG Yuanliu, SHAO Jingwen. Research Progress on Multi-source Data Fusion Based on CiteSpace[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(5): 13 -27 .
[3] WANG Shuying, LU Yuxiang, DONG Shutong, CHEN Mo, KANG Bingya, JIANG Zhanglan, SU Chengyuan. Research Progress on the Propagation Process and Control Technology of ARGs in Wastewater[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 1 -15 .
[4] ZHONG Qiao, CHEN Shenglong, TANG Congcong. Hydrogel Technology for Microalgae Collection: Status Overview, Challenges and Development Analysis[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 16 -29 .
[5] ZHAI Siqi, CAI Wenjun, ZHU Su, LI Hanlong, SONG Hailiang, YANG Xiaoli, YANG Yuli. Dynamic Relationship Between Reverse Solute Flux and Membrane Fouling in Forward Osmosis[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 30 -39 .
[6] ZHENG Guoquan, QIN Yongli, WANG Chenxiang, GE Shijia, WEN Qianmin, JIANG Yongrong. Stepwise Precipitation of Heavy Metals from Acid Mine Drainage and Mineral Formation in Sulfate-Reducing Anaerobic Baffled Reactor System[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 40 -52 .
[7] LIU Yang, ZHANG Yijie, ZHANG Yan, LI Ling, KONG Xiangming, LI Hong. Current Status and Trends of Algal Coagulation Elimination Technology in Drinking Water Treatment: a Visual Analysis Based on CiteSpace[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 53 -66 .
[8] TIAN Sheng, CHEN Dong. A Joint Eco-driving Optimization Research for Connected Fuel Cell Hybrid Vehicle via Deep Reinforcement Learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 67 -80 .
[9] CHEN Xiufeng, WANG Chengxin, ZHAO Fengyang, YANG Kai, GU Kexin. A Single Intersection Signal Control Method Based on Improved DQN Algorithm[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 81 -88 .
[10] LI Xin, NING Jing. Online Assessment of Transient Stability in Power Systems Based on Spatiotemporal Feature Fusion[J]. Journal of Guangxi Normal University(Natural Science Edition), 2024, 42(6): 89 -100 .