Journal of Guangxi Normal University(Natural Science Edition) ›› 2021, Vol. 39 ›› Issue (5): 122-133.doi: 10.16088/j.issn.1001-6600.2020122801

Previous Articles     Next Articles

Retinal Image Registration Using Convolutional Neural Network

WU Lingyu, LAN Yang, XIA Haiying*   

  1. College of Electronics Engineering, Guangxi Normal University, Guilin Guangxi 541004, China
  • Received:2020-12-28 Revised:2021-03-09 Online:2021-09-25 Published:2021-10-19

Abstract: The distribution of feature points extracted by traditional fundus image registration methods is too dense, which leads to the inaccurate alignment of the images to be registered images. And the retinal vessel bifurcation feature points have sparse distribution and stable features to improve the accuracy and speed of image registration. Therefore, this paper proposed a deep learning-based fundus image registration framework for vessel segmentation and bifurcation feature point extraction. This framework is composed of two deep convolutional neural networks. The first is the fundus blood vessel segmentation network SR-UNet, which combines channel attention (SE) and residual blocks on the basis of U-Net to segment retinal vessels to assist feature points extraction. The second is the feature point detection network FD-Net, which is used to extract bifurcation feature points from the vessel segmentation map. The proposed registration model is tested on the public fundus registration data set FIRE. The correct matching rate of the feature points is 90.03%. Compared with more advanced retinal image registration algorithms, the algorithm proposed has better performance and strong robustness both in registration quantitative and visual analysis.

Key words: retinal image registration, deep learning, vessel segmentation network, feature point detection network, U-Net

CLC Number: 

  • TP18
[1] MITCHELL P, LIEW G, GOPINATH B, et al. Age-related macular degeneration[J]. The Lancet, 2018, 392(10153): 1147-1159. DOI:10.1016/S0140-6736(18)31550-2.
[2] FU H Z, WANG B Y, SHEN J B, et al. Evaluation of retinal image quality assessment networks in different color-spaces[C] // Medical Image Computing and Computer Assisted Intervention-MICCAI 2019: LNCS Volume 11764. Cham: Springer Nature Switzerland AG, 2019: 48-56. DOI:10.1007/978-3-030-32239-7_6.
[3] 陈丹华. 彩色眼底视网膜图像配准算法研究[D]. 桂林: 广西师范大学, 2019.
[4] 张娟. 医学图像配准中相似性测度的研究[D]. 广州: 南方医科大学, 2014.
[5] LEGG P A, ROSIN P L, MARSHALL D, et al. Improving accuracy and efficiency of mutual information for multi-modal retinal image registration using adaptive probability density estimation[J]. Computerized Medical Imaging and Graphics, 2013, 37(7/8): 597-606. DOI:10.1016/j.compmedimag.2013.08.004.
[6] 朱明, 姚强, 唐俊, 等. 超图约束和改进归一化互相关方法相结合的图像配准算法[J]. 国防科技大学学报, 2019, 41(3): 50-55. DOI:10.11887/j.cn.201903009.
[7] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. DOI:10.1023/B:VISI.0000029664.99615.94.
[8] 张欣鹏, 杜伟强, 肖志涛, 等. 结合SIFT变换与Otsu匹配的彩色眼底图像拼接方法[J]. 计算机工程与应用, 2017, 53(18): 176-181, 249. DOI:10.3778/j.issn.1002-8331.1604-0243.
[9] 孙华魁. SIFT方法在医学图像配准中的应用研究[D]. 济南: 山东大学, 2015.
[10] BAY H, TUYTELAARS T, Van GOOL L. SURF: speeded up robust features[C] // Computer Vision - ECCV 2006: LNCS Volume 3951. Berlin: Springer, 2006: 404-417. DOI:10.1007/11744023_32.
[11] 罗天健, 刘秉瀚. 融合特征的快速SURF配准算法[J]. 中国图象图形学报, 2015, 20(1): 95-103. DOI:10.11834/jig.20150110.
[12] CHEN J, TIAN J, LEE N, et al. A partial intensity invariant feature descriptor for multimodal retinal image registration[J]. IEEE Transactions on Biomedical Engineering, 2010, 57(7): 1707-1718. DOI:10.1109/TBME.2010.2042169.
[13] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C] // 2011 IEEE International Conference on Computer Vision (ICCV 2011). Los Alamitos, CA: IEEE Computer Society, 2011: 2564-2571. DOI:10.1109/ICCV.2011.6126544.
[14] 杨炳坤, 程树英, 郑茜颖. 改进的ORB 特征匹配算法[J]. 传感器与微系统, 2020, 39(2): 136-139. DOI:10.13873/j.1000-9787(2020)02-0136-04.
[15] LEUTENEGGER S, CHLI M, SIEGWART R Y. BRISK: binary robust invariant scalable keypoints[C] // 2011 IEEE International Conference on Computer Vision (ICCV 2011). Los Alamitos, CA: IEEE Computer Society, 2011: 2548-2555. DOI:10.1109/ICCV.2011.6126542.
[16] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C] // Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: LNCS Volume 9351. Cham: Springer International Publishing AG Switzerland, 2015: 234-241. DOI:10.1007/978-3-319-24574-4_28.
[17] YI K M, TRULLS E, LEPETIT V, et al. LIFT: learned invariant feature transform[C] // Computer Vision - ECCV 2016: LNCS Volume 9910. Cham: Springer International Publishing AG Switzerland, 2016: 467-483. DOI:10.1007/978-3-319-46466-4_28.
[18] MISHCHUK A, MISHKIN D, RADENOVIC F, et al. Working hard to know your neighbor′s margins: local descriptor learning loss[EB/OL]. (2017-05-30)[2020-12-28]. https:// arxiv.org/pdf/1705.10872.
[19] 陈向前, 郭小青, 周钢, 等. 基于深度学习的2D/3D医学图像配准研究[J]. 中国生物医学工程学报, 2020, 39(4): 394-403. DOI:10.3969/j.issn.0258-8021.2020.04.002.
[20] 姚明青, 胡靖. 基于深度强化学习的多模态医学图像配准[J]. 计算机辅助设计与图形学学报, 2020, 32(8): 1236-1247.
[21] DETONE D, MALISIEWICZ T, RABINOVICH A. SuperPoint: self-supervised interest point detection and description [EB/OL]. (2017-12-20)[2020-12-28]. https:// arxiv.org/pdf/1712.07629.
[22] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2016: 770-778. DOI:10.1109/CVPR.2016.90.
[23] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. DOI:10.1109/TPAMI.2019.2913372.
[24] FRAZ M M, REMAGNINO P, HOPPE A, et al. Blood vessel segmentation methodologies in retinal images: a survey[J]. Computer Methods and Programs in Biomedicine, 2012, 108(1): 407-433. DOI:10.1016/j.cmpb.2012.03.009.
[25] TAN M X, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. (2019-05-28)[2020-12-28]. https:// arxiv.org/pdf/1905.11946.
[26] LU Z, PU H M, WANG F C, et al. The expressive power of neural networks: a view from the width[EB/OL]. (2017-09-08)[2020-12-28]. https:// arxiv.org/pdf/1709.02540.
[27] ZAGORUYKO S, KOMODAKIS N. Wide residual networks[EB/OL]. (2016-05-23)[2020-12-28]. https:// arxiv.org/pdf/1605.07146.
[28] HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17)[2020-12-27]. https:// arxiv.org/pdf/1704.04861.
[29] HERNANDEZ-MATAS C, ZABULIS X, TRIANTAFYLLOU A, et al. FIRE: fundus image registration dataset[J]. Journal for Modeling in Ophthalmology, 2017, 1(4): 16-28.
[30] DECENCIÈRE E, ZHANG X W, CAZUGUEL G, et al. Feedback on a publicly distributed database: the Messidor databas[J]. Image Analysis & Stereology, 2014, 33(3): 231-234. DOI:10.5566/ias.1155.
[31] WANG G, WANG Z C, CHEN Y F, et al. Robust point matching method for multimodal retinal image registration[J]. Biomedical Signal Processing and Control, 2015, 19: 68-76. DOI:10.1016/j.bspc.2015.03.004.
[1] CHEN Wenkang, LU Shenglian, LIU Binghao, LI Guo, LIU Xiaoyu, CHEN Ming. Real-time Citrus Recognition under Orchard Environment by Improved YOLOv4 [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 134-146.
[2] YANG Zhou, FAN Yixing, ZHU Xiaofei, GUO Jiafeng, WANG Yue. Survey on Modeling Factors of Neural Information Retrieval Model [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 1-12.
[3] DENG Wenxuan, YANG Hang, JIN Ting. A Dimensionality-reduction Method Based on Attention Mechanismon Image Classification [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 32-40.
[4] XUE Tao, QIU Senhui, LU Hao, QIN Xingsheng. Exchange Rate Prediction Based on Empirical Mode Decomposition and Multi-branch LSTM Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(2): 41-50.
[5] TANG Rongchai, WU Xiru. Real-time Detection of Passion Fruit Based on Improved YOLO-V3 Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(6): 32-39.
[6] ZHANG Mingyu,ZHAO Meng,CAI Fuhong,LIANG Yu,WANG Xinhong. Wave Power Prediction Based on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(3): 25-32.
[7] LI Weiyong, LIU Bin, ZHANG Wei, CHEN Yunfang. An Automatic Summarization Model Based on Deep Learning for Chinese [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 51-63.
[8] LIU Yingxuan, WU Xiru, XUE Ganggang. Multi-target Real-time Detection for Road Traffic SignsBased on Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 96-106.
[9] ZHANG Jinlei, LUO Yuling, FU Qiang. Predicting Financial Time Series Based on Gated Recurrent Unit Neural Network [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(2): 82-89.
[10] HUANG Liming,CHEN Weizheng,YAN Hongfei,CHEN Chong. A Stock Prediction Method Based on Recurrent Neural Network and Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 13-22.
[11] YUE Tianchi, ZHANG Shaowu, YANG Liang, LIN Hongfei, YU Kai. Stance Detection Method Based on Two-Stage Attention Mechanism [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 42-49.
[12] YU Chuanming,LI Haonan,AN Lu. Analysis of Text Emotion Cause Based on Multi-task Deep Learning [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 50-61.
[13] WANG Qi,QIU Jiahui,RUAN Tong,GAO Daqi,GAO Ju. Recurrent Capsule Network for Clinical Relation Extraction [J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(1): 80-88.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] CHEN Chun-qiang, DENG Hua, HUANG Fang-fang. Concentration and Speciation Analysis of Heavy Metals in Soils in Guiping Manganese Mine, Guangxi, China[J]. Journal of Guangxi Normal University(Natural Science Edition), 2014, 32(4): 108 -114 .
[2] HUANG Jia-yu, TAN Yu, LIAO Yu-jie, WANG Wei-sheng, WANG Ying-hui. Effects of Arbuscular Mycorrhizal Fungi on the Uptake of Copper and Zinc by Eucalyptus[J]. Journal of Guangxi Normal University(Natural Science Edition), 2013, 31(2): 118 -122 .
[3] LI Hui, YIN Hui, JIANG Zhong-cheng, YANG Qi-yong, WANG Yue. Remote Sensing of Rock Desertification and Soil Erosion in Typical Karst Area and the Key Problems[J]. Journal of Guangxi Normal University(Natural Science Edition), 2013, 31(2): 133 -139 .
[4] CHEN You-ying, ZHENG Zhi, KONG Xiang-zeng, ZHANG Sheng-yuan. Classification of Colon Cancer Data Based on Bayesian Classifier[J]. Journal of Guangxi Normal University(Natural Science Edition), 2011, 29(3): 187 -191 .
[5] LI Zhaomei, LI Wenlin, MENG Anxin, ZHAO Zhenchi, QIN Yongfu, LAN Haihui, LU Huijin, CHEN Lisha, LIANG Weigang, SHEN Hongtao. Study of 14C-AMS in Dating of the Ancient Painting[J]. Journal of Guangxi Normal University(Natural Science Edition), 2019, 37(2): 38 -43 .
[6] DUAN Huajuan, WEI Yongqing, LIU Peiyu, ZHOU Peng. An Improved Multi-decision Tree Algorithm for Imbalanced Classification[J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(2): 72 -80 .
[7] ZHANG Mingyu,ZHAO Meng,CAI Fuhong,LIANG Yu,WANG Xinhong. Wave Power Prediction Based on Deep Learning[J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(3): 25 -32 .
[8] XU Peng, LI Chaofeng, CHANG Xinwei, ZHANG Yanwu. Synthesis of Water-Soluble Ketone-Aldehyde Resins by Catalysis of Solid Base[J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(3): 70 -79 .
[9] XU Jianmin, WEI Jia, SHOU Yanfang. Comprehensive Evaluation of Urban Road Traffic Operation StatusBased on Game Theory-Cloud Model[J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(4): 1 -10 .
[10] HU Jinming, WEI Duqu. Research on Generalized Sychronization of Fractional-order PMSM[J]. Journal of Guangxi Normal University(Natural Science Edition), 2020, 38(6): 14 -20 .