基于卷积神经网络的眼底图像配准研究

doi:10.16088/j.issn.1001-6600.2020122801

摘要/Abstract

摘要： 传统眼底图像配准方法提取的特征点分布过于密集,导致配准图像无法准确对齐,而眼底血管分叉特征点具有分布稀疏和特征稳定等特性,能提高图像的配准精度和速度。因此,本文针对血管分割和分叉特征点提取,提出一个基于深度学习的眼底图像配准框架。这个框架由2个深度卷积神经网络组成:第一个是眼底血管分割网络SR-UNet, 其在U-Net的基础上融合通道注意力(SE)和残差块,用于分割血管去辅助提取特征点;第二个是特征点检测网络FD-Net,用于从血管分割图中提取分叉特征点。提出的配准模型在公共眼底配准数据集FIRE上进行实验,其特征点正确匹配率为90.03%,与较先进的算法进行对比,本文提出的算法在配准定量和视觉分析上都有较好的性能提升,具有较强的鲁棒性。

关键词: 眼底图像配准, 深度学习, 血管分割网络, 特征提取网络, U-Net

Abstract: The distribution of feature points extracted by traditional fundus image registration methods is too dense, which leads to the inaccurate alignment of the images to be registered images. And the retinal vessel bifurcation feature points have sparse distribution and stable features to improve the accuracy and speed of image registration. Therefore, this paper proposed a deep learning-based fundus image registration framework for vessel segmentation and bifurcation feature point extraction. This framework is composed of two deep convolutional neural networks. The first is the fundus blood vessel segmentation network SR-UNet, which combines channel attention (SE) and residual blocks on the basis of U-Net to segment retinal vessels to assist feature points extraction. The second is the feature point detection network FD-Net, which is used to extract bifurcation feature points from the vessel segmentation map. The proposed registration model is tested on the public fundus registration data set FIRE. The correct matching rate of the feature points is 90.03%. Compared with more advanced retinal image registration algorithms, the algorithm proposed has better performance and strong robustness both in registration quantitative and visual analysis.

Key words: retinal image registration, deep learning, vessel segmentation network, feature point detection network, U-Net

中图分类号:

TP18

吴玲玉, 蓝洋, 夏海英. 基于卷积神经网络的眼底图像配准研究[J]. 广西师范大学学报（自然科学版）, 2021, 39(5): 122-133.

WU Lingyu, LAN Yang, XIA Haiying. Retinal Image Registration Using Convolutional Neural Network[J]. Journal of Guangxi Normal University(Natural Science Edition), 2021, 39(5): 122-133.

参考文献

[1] MITCHELL P, LIEW G, GOPINATH B, et al. Age-related macular degeneration[J]. The Lancet, 2018, 392(10153): 1147-1159. DOI:10.1016/S0140-6736(18)31550-2.
[2] FU H Z, WANG B Y, SHEN J B, et al. Evaluation of retinal image quality assessment networks in different color-spaces[C] // Medical Image Computing and Computer Assisted Intervention-MICCAI 2019: LNCS Volume 11764. Cham: Springer Nature Switzerland AG, 2019: 48-56. DOI:10.1007/978-3-030-32239-7_6.
[3] 陈丹华. 彩色眼底视网膜图像配准算法研究[D]. 桂林: 广西师范大学, 2019.
[4] 张娟. 医学图像配准中相似性测度的研究[D]. 广州: 南方医科大学, 2014.
[5] LEGG P A, ROSIN P L, MARSHALL D, et al. Improving accuracy and efficiency of mutual information for multi-modal retinal image registration using adaptive probability density estimation[J]. Computerized Medical Imaging and Graphics, 2013, 37(7/8): 597-606. DOI:10.1016/j.compmedimag.2013.08.004.
[6] 朱明, 姚强, 唐俊, 等. 超图约束和改进归一化互相关方法相结合的图像配准算法[J]. 国防科技大学学报, 2019, 41(3): 50-55. DOI:10.11887/j.cn.201903009.
[7] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. DOI:10.1023/B:VISI.0000029664.99615.94.
[8] 张欣鹏, 杜伟强, 肖志涛, 等. 结合SIFT变换与Otsu匹配的彩色眼底图像拼接方法[J]. 计算机工程与应用, 2017, 53(18): 176-181, 249. DOI:10.3778/j.issn.1002-8331.1604-0243.
[9] 孙华魁. SIFT方法在医学图像配准中的应用研究[D]. 济南: 山东大学, 2015.
[10] BAY H, TUYTELAARS T, Van GOOL L. SURF: speeded up robust features[C] // Computer Vision - ECCV 2006: LNCS Volume 3951. Berlin: Springer, 2006: 404-417. DOI:10.1007/11744023_32.
[11] 罗天健, 刘秉瀚. 融合特征的快速SURF配准算法[J]. 中国图象图形学报, 2015, 20(1): 95-103. DOI:10.11834/jig.20150110.
[12] CHEN J, TIAN J, LEE N, et al. A partial intensity invariant feature descriptor for multimodal retinal image registration[J]. IEEE Transactions on Biomedical Engineering, 2010, 57(7): 1707-1718. DOI:10.1109/TBME.2010.2042169.
[13] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C] // 2011 IEEE International Conference on Computer Vision (ICCV 2011). Los Alamitos, CA: IEEE Computer Society, 2011: 2564-2571. DOI:10.1109/ICCV.2011.6126544.
[14] 杨炳坤, 程树英, 郑茜颖. 改进的ORB 特征匹配算法[J]. 传感器与微系统, 2020, 39(2): 136-139. DOI:10.13873/j.1000-9787(2020)02-0136-04.
[15] LEUTENEGGER S, CHLI M, SIEGWART R Y. BRISK: binary robust invariant scalable keypoints[C] // 2011 IEEE International Conference on Computer Vision (ICCV 2011). Los Alamitos, CA: IEEE Computer Society, 2011: 2548-2555. DOI:10.1109/ICCV.2011.6126542.
[16] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C] // Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: LNCS Volume 9351. Cham: Springer International Publishing AG Switzerland, 2015: 234-241. DOI:10.1007/978-3-319-24574-4_28.
[17] YI K M, TRULLS E, LEPETIT V, et al. LIFT: learned invariant feature transform[C] // Computer Vision - ECCV 2016: LNCS Volume 9910. Cham: Springer International Publishing AG Switzerland, 2016: 467-483. DOI:10.1007/978-3-319-46466-4_28.
[18] MISHCHUK A, MISHKIN D, RADENOVIC F, et al. Working hard to know your neighbor′s margins: local descriptor learning loss[EB/OL]. (2017-05-30)[2020-12-28]. https:// arxiv.org/pdf/1705.10872.
[19] 陈向前, 郭小青, 周钢, 等. 基于深度学习的2D/3D医学图像配准研究[J]. 中国生物医学工程学报, 2020, 39(4): 394-403. DOI:10.3969/j.issn.0258-8021.2020.04.002.
[20] 姚明青, 胡靖. 基于深度强化学习的多模态医学图像配准[J]. 计算机辅助设计与图形学学报, 2020, 32(8): 1236-1247.
[21] DETONE D, MALISIEWICZ T, RABINOVICH A. SuperPoint: self-supervised interest point detection and description [EB/OL]. (2017-12-20)[2020-12-28]. https:// arxiv.org/pdf/1712.07629.
[22] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2016: 770-778. DOI:10.1109/CVPR.2016.90.
[23] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023. DOI:10.1109/TPAMI.2019.2913372.
[24] FRAZ M M, REMAGNINO P, HOPPE A, et al. Blood vessel segmentation methodologies in retinal images: a survey[J]. Computer Methods and Programs in Biomedicine, 2012, 108(1): 407-433. DOI:10.1016/j.cmpb.2012.03.009.
[25] TAN M X, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. (2019-05-28)[2020-12-28]. https:// arxiv.org/pdf/1905.11946.
[26] LU Z, PU H M, WANG F C, et al. The expressive power of neural networks: a view from the width[EB/OL]. (2017-09-08)[2020-12-28]. https:// arxiv.org/pdf/1709.02540.
[27] ZAGORUYKO S, KOMODAKIS N. Wide residual networks[EB/OL]. (2016-05-23)[2020-12-28]. https:// arxiv.org/pdf/1605.07146.
[28] HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[EB/OL]. (2017-04-17)[2020-12-27]. https:// arxiv.org/pdf/1704.04861.
[29] HERNANDEZ-MATAS C, ZABULIS X, TRIANTAFYLLOU A, et al. FIRE: fundus image registration dataset[J]. Journal for Modeling in Ophthalmology, 2017, 1(4): 16-28.
[30] DECENCIÈRE E, ZHANG X W, CAZUGUEL G, et al. Feedback on a publicly distributed database: the Messidor databas[J]. Image Analysis & Stereology, 2014, 33(3): 231-234. DOI:10.5566/ias.1155.
[31] WANG G, WANG Z C, CHEN Y F, et al. Robust point matching method for multimodal retinal image registration[J]. Biomedical Signal Processing and Control, 2015, 19: 68-76. DOI:10.1016/j.bspc.2015.03.004.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed