E-mail Alert
RSS
-
摘要
OCT视网膜图像中存在着噪声和散斑，单一的提取空间特征往往容易遗漏一些重要信息，导致不能准确地分割目标区域。而OCT图像本身存在光谱频域特征，针对OCT图像的频域特征，本文基于U-Net和快速傅立叶卷积提出一种新的双编码器模型以提高对OCT图像视网膜层、液体的分割性能，提出的频域编码器可以提取图像频域信息并通过快速傅里叶卷积转换为空间信息，将很好地弥补单一空间编码器遗漏特征信息的不足。经过与其他经典模型的对比和消融实验，结果表明，随着频域编码器的添加，该模型能有效提升对视网膜层和液体的分割性能，平均Dice系数和mIoU相较于U-Net均提高2%，相较于ReLayNet分别提高8%和4%，其中对液体的分割提升尤为明显，相较于U-Net 模型Dice系数提高了10%。
Abstract
There are noises and speckles in OCT retinal images, and a single extraction of spatial features is often easy to miss some important information. Therefore, the target region cannot be accurately segmented. OCT images themselves have spectral frequency domain characteristics. Aiming at the frequency domain characteristics of OCT images, this paper proposes a new dual encoder model based on U-Net and fast Fourier convolution to improve the segmentation performance of the retinal layer and liquid in OCT images. The proposed frequency encoder can extract image frequency domain information and convert it into spatial information through fast Fourier convolution. The lack of feature information that can be omitted by a single space encoder will be well-complemented. After comparison with other classical models and ablation experiments, the results show that with the addition of a frequency domain encoder, the model can effectively improve the segmentation performance of the retinal layer and liquid. Both average Dice coefficient and mIoU are increased by 2% compared with U-Net. They are increased by 8% and 4% compared with ReLayNet, respectively. Among them, the improvement of liquid segszmentation is particularly obvious, and the Dice coefficient is increased by 10% compared with the U-Net model.
-
Overview
Overview: Deep learning methods have already had a profound impact on medical image processing. However, some noises and speckles contained in OCT images affect the quality of the images, coupled with the elongated and complex retinal layer and the irregular distribution of pathological fluid in it, which brings great challenges to the automatic segmentation task. At the same time, depending on the limited manpower and time, it is also difficult to sketch a large number of existing images by relying on the professional knowledge of doctors. For the above reasons, automatic medical image segmentation, a scientific medical auxiliary support, is of great clinical significance.
The research content of this paper mainly focuses on the analysis and processing of retinal OCT images. When it comes to monitoring the state of the patient's retina layer, due to the noise and speckle in OCT images and the subtle and complex structure of the retina layer itself, the performance of the model is limited by a single extraction space feature, and the target region cannot be accurately segmented. Aiming at the frequency domain characteristics of OCT images, this paper proposes a dual encoder model based on U-Net and fast Fourier convolution to improve the segmentation performance of the retinal layer and liquid in OCT images. The frequency domain encoder extracts the image frequency domain information and converts it into spatial information by fast Fourier convolution to complete the feature extraction of a single spatial encoder. The experimental results show that the model can effectively improve the segmentation performance of the retinal layer and liquid, both average Dice coefficient and mIoU are increased by 2% compared with U-Net, and the Dice coefficient of liquid is increased by 10%.
-
-
图 6 本文模型与U-Net的分割结果比较。(a)小部分聚集液体区域的分割对比； (b)长形相连接液体区域的分割对比； (c)分散无规则分布液体区域的分割对比
Figure 6. Some qualitative results of ours compared to U-Net. (a) Segmentation and comparison of small areas of liquid accumulation; (b) Segmentation and comparison of long forms and connected liquid regions; (c) Segmentation and comparison of randomly distributed liquid regions
表 1 不同方法在数据集上的分割结果
Table 1. Results of each method on dataset
Model Dice mIoU ILM NFL-IPL INL OPL ONL-ISM ISE OS-RPE Fluid Mean U-Net 0.84 0.89 0.77 0.76 0.89 0.89 0.89 0.79 0.84 0.74 ReLayNet 0.84 0.85 0.70 0.71 0.87 0.88 0.84 0.55 0.78 0.72 Dual-encoder 0.87 0.89 0.77 0.75 0.90 0.90 0.90 0.89 0.86 0.76 
下载: 导出CSV
表 2 关于FFC-DC和α的消融实验
Table 2. Ablation study on the FFC-DC blocks and α
Method α Dice ILM NFL-IPL INL OPL ONL-ISM ISE OS-RPE Fluid Mean - - 0.86 0.88 0.76 0.77 0.89 0.89 0.89 0.80 0.843 +FFC-DC 0.25 0.86 0.89 0.77 0.76 0.89 0.89 0.89 0.90 0.856 +FFC-DC 0.5 0.87 0.89 0.77 0.75 0.90 0.90 0.90 0.89 0.860 +FFC-DC 0.75 0.86 0.89 0.77 0.76 0.87 0.89 0.89 0.87 0.850 +FFC-DC 1 0.84 0.88 0.76 0.76 0.88 0.89 0.88 0.88 0.846
下载: 导出CSV
-
参考文献
[1] Hayreh S S, Zimmerman M B. Fundus changes in branch retinal arteriolar occlusion[J]. Retina, 2015, 35(10): 2060−2066. doi: 10.1097/IAE.0000000000000585
[2] Fraz M M, Remagnino P, Hoppe A, et al. Blood vessel segmentation methodologies in retinal images—A survey[J]. Comput Methods Programs Biomed, 2012, 108(1): 407−433. doi: 10.1016/j.cmpb.2012.03.009
[3] 李云耀, 樊金宇, 蒋天亮, 等. 光学相干层析技术在眼科手术导航方面的研究进展[J]. 光电工程, 2023, 50(1): 220027. doi: 10.12086/oee.2023.220027
Li Y Y, Fan J Y, Jiang T L, et al. Review of the development of optical coherence tomography imaging navigation technology in ophthalmic surgery[J]. Opto-Electron Eng, 2023, 50(1): 220027. doi: 10.12086/oee.2023.220027
[4] 姜盼秋, 汪平河. 谱域光学相干层析系统的色散补偿技术研究[J]. 光电工程, 2021, 48(10): 210184. doi: 10.12086/oee.2021.210184
Jiang P Q, Wang P H. Research on dispersion compensation technology for SDOCT system[J]. Opto-Electron Eng, 2021, 48(10): 210184. doi: 10.12086/oee.2021.210184
[5] Huang D, Swanson E A, Lin C P, et al. Optical coherence tomography[J]. Science, 1991, 254(5035): 1178−1181. doi: 10.1126/science.1957169
[6] Tranos P G, Wickremasinghe S S, Stangos N T, et al. Macular edema[J]. Surv Ophthalmol, 2004, 49(5): 470−490. doi: 10.1016/j.survophthal.2004.06.002
[7] Monemian M, Rabbani H. Mathematical analysis of texture indicators for the segmentation of optical coherence tomography images[J]. Optik, 2020, 219: 165227. doi: 10.1016/j.ijleo.2020.16522
[8] Liu Y H, Carass A, He Y F, et al. Layer boundary evolution method for macular OCT layer segmentation[J]. Biomed Opt Express, 2019, 10(3): 1064−1080. doi: 10.1364/BOE.10.001064
[9] 杨平, 彭清, 刘维平, 等. 一种眼底黄斑水肿OCT图像分割方法[J]. 生物医学工程学杂志, 2011, 28(5): 1001−1006. doi: 10.13741/j.cnki.11-1879/o4.2019.06.015
Yang P, Peng Q, Liu W P, et al. A segmentation algorithm of OCT image for macula edema[J]. J Biomed Eng, 2011, 28(5): 1001−1006. doi: 10.13741/j.cnki.11-1879/o4.2019.06.015
[10] 何锦涛, 陈明惠, 贾文宇, 等. 眼底OCT图像中糖尿病性黄斑水肿的分割[J]. 光电工程, 2018, 45(7): 170605. doi: 10.12086/oee.2018.170605
He J T, Chen M H, Jia W Y, et al. Segmentation of diabetic macular edema in OCT retinal images[J]. Opto-Electron Eng, 2018, 45(7): 170605. doi: 10.12086/oee.2018.170605
[11] Hussain A, Bhuiyan A, Turpin A, et al. Automatic identification of pathology-distorted retinal layer boundaries using SD-OCT imaging[J]. IEEE Trans Biomed Eng, 2017, 64(7): 1638−1649. doi: 10.1109/TBME.2016.2619120
[12] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, 2016: 770–778. https://doi.org/10.1109/CVPR.2016.90.
[13] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, inception-ResNet and the impact of residual connections on learning[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, 2017. https://doi.org/10.1609/aaai.v31i1.11231.
[14] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the 3rd International Conference on Learning Representations, San Diego, 2015.
[15] 吴玉超, 林岚, 王婧璇. 基于卷积神经网络的语义分割在医学图像中的应用[J]. 生物医学工程学杂志, 2020, 37(3): 533−540. doi: 10.7507/1001-5515.201906067
Wu Y C, Lin L, Wang J X. Application of semantic segmentation based on convolutional neural network in medical images[J]. J Biomed Eng, 2020, 37(3): 533−540. doi: 10.7507/1001-5515.201906067
[16] 秦传波, 宋子玉, 曾军英, 等. 联合多尺度和注意力-残差的深度监督乳腺癌分割[J]. 光学 精密工程, 2021, 29(4): 877−895. doi: 10.37188/OPE.20212904.0877
Qin C B, Song Z Y, Zeng J Y, et al. Deeply supervised breast cancer segmentation combined with multi-scale and attention-residuals[J]. Opt Precis Eng, 2021, 29(4): 877−895. doi: 10.37188/OPE.20212904.0877
[17] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical Image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, 2015: 234–241. https://doi.org/10.1007/978-3-319-24574-4_28.
[18] Roy A G, Conjeti S, Karri S P K, et al. ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks[J]. Biomed Opt Express, 2017, 8(8): 3627−3642. doi: 10.1364/BOE.8.003627
[19] 袁坤, 霍力. 用于视网膜OCT图像分层的多尺度补全卷积神经网络[J]. 中国激光, 2021, 48(15): 1507004. doi: 10.3788/CJL202148.1507004
Yuan K, Huo L. Multiple-scale inpainting convolutional neural network for retinal OCT image segmentation[J]. Chin J Lasers, 2021, 48(15): 1507004. doi: 10.3788/CJL202148.1507004
[20] Liu W, Sun Y K, Ji Q G. MDAN-UNet: multi-scale and dual attention enhanced nested U-net architecture for segmentation of optical coherence tomography images[J]. Algorithms, 2020, 13(3): 60. doi: 10.3390/a13030060
[21] Ngo L, Cha J, Han J H. Deep neural network regression for automated retinal layer segmentation in optical coherence tomography images[J]. IEEE Trans Image Process, 2020, 29: 303−312. doi: 10.1109/TIP.2019.2931461
[22] Chi L, Jiang B R, Mu Y D. Fast Fourier convolution[C]//Proceedings of the 34th International Conference on Neural Information Processing System, Vancouver, 2020: 4479-4488. https://dl.acm.org/doi/abs/10.5555/3495724.3496100
[23] Nair V, Chatterjee M, Tavakoli N, et al. Optimizing CNN using fast Fourier transformation for object recognition[C]//Proceedings of the 19th IEEE International Conference on Machine Learning and Applications, Miami, 2020: 234–239. https://doi.org/10.1109/ICMLA51294.2020.00046.
[24] Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[Z]. arXiv: 1511.07122, 2016. https://arxiv.org/abs/1511.07122.
[25] Chiu S J, Allingham M J, Mettu P S, et al. Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema[J]. Biomed Opt Express, 2015, 6(4): 1172−1194. doi: 10.1364/BOE.6.001172
[26] Mondal P K, Prodhan U K, AI Mamun S, et al. Segmentation of white blood cells using fuzzy C means segmentation algorithm[J]. IOSR J Comput Eng, 2014, 16(3): 1−5. doi: 10.9790/0661-16390105
[27] Singh V K, Romani S, Rashwan H A, et al. Conditional generative adversarial and convolutional networks for X-ray breast mass segmentation and shape classification[C]//Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, 2018: 833–840. https://doi.org/10.1007/978-3-030-00934-2_92.
-
访问统计
点击扫一扫
图(7)
表(2)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
