E-mail Alert
RSS
Retinal lesions graded algorithm that integrates coordinate perception and hybrid extraction
-
摘要:
针对糖尿病视网膜病变中存在样本分布不平衡和病灶区域特征识别困难等问题，提出一种融合坐标感知与混合提取的视网膜病变分级算法。该算法首先对视网膜输入图像进行裁剪、高斯滤波等预处理操作，以增强图像病变前景与噪声背景之间的差异度；然后由Res2Net-50和Densenet-121骨干网络组成的混合双模型将增强后的图像进行特征逐层提取，实现多尺度特征纹理的充分捕捉；再在混合双模型连接处融入多层坐标感知模块和注意力特征融合模块，达到剔除聚焦病灶特征干扰的目的，实现不同病灶语义间的权重重塑；最后利用组合损失函数缓解样本分布不均匀问题，进一步监督模型的训练与测试。该文算法在IDRID和APTOS 2019数据集上进行实验，二次加权系数分别为88.76%和90.29%；准确率分别为81.55%和84.42%，为视网膜病变分级智能辅助诊断提供了新窗口。
Abstract:Aiming at the problems of unbalanced sample distribution and difficulty identification of the lesion area in diabetic retinopathy, we propose a retinal lesions grading algorithm that integrates coordinate perception and hybrid extraction. This algorithm first processes the retinal input image and the Gaussian filtering to enhance the difference between the image lesions and the background of the noise, and then the hybrid dual models composed of the backbone network of Res2Net-50 and Densenet-121 will be enhanced. The image is extracted layer by layer to achieve the full capture of the multi-scale feature texture, then the multi-layer coordinate perception module and the attention characteristics fusion module are integrated at the mixed dual model connection to achieve the purpose of eliminating the characteristics of the lesions and the realization of different lesions. The weight of semantics is reshaped, finally uses the combined loss function to relieve the uneven distribution of samples to further supervise the training and test of the model. This article is experimented on the IDRID and Aptos 2019 data sets, with the secondary weighted coefficients of 88.76% and 90.29%, respectively. Accuracy rates were 81.55% and 84.42%, which provides a new window for the diagnosis of retinopathy grades and intelligent auxiliary diagnosis.
-
Overview: Diabetic Retinopathy (DR) is a kind of eye disease caused by diabetes. Patients have been in a high blood sugar environment for a long time. It is very easy to damage the retina. If the disease fails to find the disease in time, it is easy to cause blindness. In recent years, with the development of deep learning technology, it has been widely used in DR intelligent grading diagnosis, but there are still deficiencies: there are small lesions such as micromiel tumors, yellow and white hard exudation or a small number of bleeding spots around the retinal images. The differences between the comparison with normal areas are not obvious. The difference between each child class is difficult to distinguish the difficulty of identification. At the same time, due to the impact of the diabetic disease stage and cure, the number of patients in different stages of different lesions is inconsistent. The sample distribution is unbalanced. In order to solve the above problems, this article proposes a hometown lesion classification algorithm that integrates coordinate perception and mixed extraction. This algorithm first conducts pre-processing operations such as IDRID and APTOS 2019 dataset images, Gaussian filtering and other pre-processing to enhance the differences between image lesions and noise background, then a hybrid dual model composed of the backbone network composed of Res2Net-50 and Densenet-121 backbone network extracts the enhanced images layer by layer, realizes the full capture of multi-scale feature texture, and enhances the robustness and generalization of the algorithm, then integrates the multi-layer coordinate perception module and attention characteristics fusion at the connection between the hybrid dual model connection. The module achieves the purpose of eliminating the characteristics of the lesion, realizing the weight of the semantics of different lesions, ensuring that the small lesion area can also obtain sufficient weight, finally using the focus loss function and cross-entropy. The combination loss function composed of the loss function relieves uneven distribution of sample distribution, weakens the accuracy of the hierarchy of the retinal lesions due to the sample, and further monitors the training and test of the model, thereby improving the accuracy of DR grading.
Experiments on IDRID and APTOS 2019 data sets, the accuracy, secondary weighted coefficients, sensitivity and specificity in IDRID are 81.55%, 88.76%, 94.20%, and 97.05%; The area of the second weighted coefficient, sensitivity and ROC curve was 84.42%, 90.29%, 87.40%, and 93.60%, respectively. The experimental results show that the comparison of the algorithm in this article still has a certain degree of superiority compared with the current mainstream algorithm, which provides a new window for the diagnosis of the hierarchical meta-cooked intelligent auxiliary diagnosis.
-
-
图 4 不同DR分级图像预处理对比。(a)原始图像；(b)预处理后图像
Figure 4. Different DR hierarchical images pre -processing comparison. (a) Primitive images; (b) Pre-processing images
图 5 本文算法在 (a) IDRID数据集和 (b) APTOS 2019数据集上的训练损失曲线
Figure 5. The training loss curves of the proposed algorithm on (a) the IDRID dataset and (b) the APTOS 2019 dataset
图 6 网络特征热图。(a) 初始图像；(b) 热力图像
Figure 6. Network feature hot pictures. (a) Initial images; (b) Thermal images
表 1 在IDRID数据集的消融结果
Table 1. The ablation results of the IDRID dataset
Model Acc/% QWK/% Se/% Sp/% M1 78.64 88.30 94.20 94.11 M2 78.64 87.13 91.30 94.11 M3 79.61 86.13 94.20 91.17 M4 80.58 88.54 86.95 97.05 M5 79.61 88.44 91.30 97.05 M6 81.55 88.76 94.20 97.05 
下载: 导出CSV
表 2 不同算法在IDRID数据集的结果表现
Table 2. The results of different algorithms in IDRID data sets
Methods Model QWK/% Acc/% Se/% Sp/% Shi et al.[7] Efficientnet-b5 87.63 79.06 — — Bhardwaj et al.[19] ResNet-50 — 79.46 82.85 76.98 Wu et al.[20] ResNet-18 — 56.19 64.21 87.39 Liang et al.[21] ResNet-50+FCFM 88.70 80.58 94.20 94.10 Song et al.[22] ResNet-50+FDM 89.11 80.58 92.75 94.10 Liu et al.[23] ResNet-50 84.42 75.78 92.75 91.17 Ours Res2Net-50+ Densenet-121 88.76 81.55 94.20 97.05 注：Re和Se计算公式一致，能够相互替换。
下载: 导出CSV
表 3 不同模型在APTOS 2019数据集的结果表现
Table 3. The results of different models in the APTOS 2019 data sets
Methods Model QWK/% Acc/% Re/% AUC/% Shaik et al.[8] LA-NSVM 75.64 84.31 66.16 — Bodapati et al.[24] Xception+VGG16 — 82.54 83.00 79.00 Bodapati et al.[25] VGG16-fc2+Xception 70.90 80.96 — — Kobat et al.[26] DenseNet201 78.37 85.93 69.72 — Song et al.[22] ResNet-50+FDM 86.34 84.28 85.32 92.78 Liu et al.[23] ResNet-50 86.08 81.96 86.43 92.46 Ours Res2Net-50+ Densenet-121 90.29 84.42 87.40 93.60 注：Re和Se计算公式一致，能够相互替换。
下载: 导出CSV
-
[1] 梁礼明, 卢宝贺, 龙鹏威, 等. 自适应特征融合级联Transformer视网膜血管分割算法[J]. 光电工程, 2023, 50(10): 230161. doi: 10.12086/oee.2023.230161
Liang L M, Lu B H, Long P W, et al. Adaptive feature fusion cascade Transformer retinal vessel segmentation algorithm[J]. Opto-Electron Eng, 2023, 50(10): 230161. doi: 10.12086/oee.2023.230161
[2] 吕佳, 王泽宇, 梁浩城. 边界注意力辅助的动态图卷积视网膜血管分割[J]. 光电工程, 2023, 50(1): 220116. doi: 10.12086/oee.2023.220116
Lv J, Wang Z Y, Liang H C. Boundary attention assisted dynamic graph convolution for retinal vascular segmentation[J]. Opto-Electron Eng, 2023, 50(1): 220116. doi: 10.12086/oee.2023.220116
[3] 陈明惠, 王腾, 袁媛, 等. 引入双编码器模型的OCT视网膜图像分割[J]. 光电工程, 2023, 50(10): 230146. doi: 10.12086/oee.2023.230146
Chen M H, Wang T, Yuan Y, et al. Study on retinal OCT segmentation with dual-encoder[J]. Opto-Electron Eng, 2023, 50(10): 230146. doi: 10.12086/oee.2023.230146
[4] He A L, Li T, Li N, et al. CABNet: category attention block for imbalanced diabetic retinopathy grading[J]. IEEE Trans Med Imaging, 2021, 40(1): 143−153. doi: 10.1109/TMI.2020.3023463
[5] Ashwini K, Dash R. Grading diabetic retinopathy using multiresolution based CNN[J]. Biomed Signal Process Control, 2023, 86: 105210. doi: 10.1016/j.bspc.2023.105210
[6] Zhou K, Gu Z W, Liu W, et al. Multi-cell multi-task convolutional neural networks for diabetic retinopathy grading[C]//Proceedings of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018: 2724–2727. https://doi.org/10.1109/EMBC.2018.8512828.
[7] Shi L, Zhang J X. Few-shot learning based on multi-stage transfer and class-balanced loss for diabetic retinopathy grading[Z]. arXiv: 2109.11806, 2023. https://doi.org/10.48550/arXiv.2109.11806.
[8] Shaik N S, Cherukuri T K. Lesion-aware attention with neural support vector machine for retinopathy diagnosis[J]. Mach Vision Appl, 2021, 32(6): 126. doi: 10.1007/s00138-021-01253-y
[9] 张文轩, 吴秦. 基于多分支注意力增强的细粒度图像分类[J]. 计算机科学, 2022, 49(5): 105−112. doi: 10.11896/jsjkx.210100108
Zhang W X, Wu Q. Fine-grained image classification based on multi-branch attention-augmentation[J]. Comput Sci, 2022, 49(5): 105−112. doi: 10.11896/jsjkx.210100108
[10] 程小辉, 李贺军, 邓昀, 等. 基于ME-ANet模型的糖尿病视网膜病变分级[J]. 广西科学, 2022, 29(2): 249−259. doi: 10.13656/j.cnki.gxkx.20220526.004
Cheng X H, Li H J, Deng Y, et al. Study on grading of diabetic retinopathy based on Me-ANet model[J]. Guangxi Sci, 2022, 29(2): 249−259. doi: 10.13656/j.cnki.gxkx.20220526.004
[11] Gao S H, Cheng M M, Zhao K, et al. Res2Net: a new multi-scale backbone architecture[J]. IEEE Trans Pattern Analy Mach Intell, 2019, 43(2): 652−662. doi: 10.1109/TPAMI.2019.2938758
[12] Vellaichamy A S, Swaminathan A, Varun C, et al. Multiple plant leaf disease classification using densenet-121 architecture[J]. Int J Electr Eng Technol, 2021, 12(5): 38−57. doi: 10.34218/IJEET.12.5.2021.005
[13] Hou Q B, Zhou D Q, Feng J S. Coordinate attention for efficient mobile network design[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 13708–13717. https://doi.org/10.1109/CVPR46437.2021.01350.
[14] Dai Y M, Gieseke F, Oehmcke S, et al. Attentional feature fusion[C]//Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision, 2021: 3559–3568. https://doi.org/10.1109/WACV48630.2021.00360.
[15] Mukhoti J, Kulharia V, Sanyal A, et al. Calibrating deep neural networks using focal loss[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems, 2020: 1282.
[16] Rezaei-Dastjerdehei M R, Mijani A, Fatemizadeh E. Addressing imbalance in multi-label classification using weighted cross entropy loss function[C]//Proceedings of the 27th National and 5th International Iranian Conference on Biomedical Engineering (ICBME), 2020: 333–338. https://doi.org/10.1109/ICBME51989.2020.9319440.
[17] 郑雯, 沈琪浩, 任佳. 基于Improved DR-Net算法的糖尿病视网膜病变识别与分级[J]. 光学学报, 2021, 41(22): 2210002. doi: 10.3788/AOS202141.2210002
Zheng W, Shen Q H, Ren J. Recognition and classification of diabetic retinopathy based on improved DR-Net algorithm[J]. Acta Opt Sin, 2021, 41(22): 2210002. doi: 10.3788/AOS202141.2210002
[18] Adak C, Karkera T, Chattopadhyay S, et al. Detecting severity of diabetic retinopathy from fundus images using ensembled transformers[Z]. arXiv: 2301.00973, 2023. https://doi.org/10.48550/arXiv.2301.00973.
[19] Bhardwaj C, Jain S, Sood M. Transfer learning based robust automatic detection system for diabetic retinopathy grading[J]. Neural Comput Appl, 2021, 33(20): 13999−14019. doi: 10.1007/s00521-021-06042-2
[20] Wu Z, Shi G L, Chen Y, et al. Coarse-to-fine classification for diabetic retinopathy grading using convolutional neural network[J]. Artif Intell Med, 2020, 108: 101936. doi: 10.1016/j.artmed.2020.101936
[21] 梁礼明, 雷坤, 詹涛, 等. 特征自适应过滤的视网膜病变分级算法[J]. 图学学报, 2022, 43(5): 815−824. doi: 10.11996/JG.j.2095-302X.2022050815
Liang L M, Lei K, Zhan T, et al. A feature-adaptive filtering algorithm for grading retinopathy[J]. J Graph, 2022, 43(5): 815−824. doi: 10.11996/JG.j.2095-302X.2022050815
[22] Song J W, Yang R Y. Feature boosting, suppression, and diversification for fine-grained visual classification[C]//Proceedings of 2021 International Joint Conference on Neural Networks (IJCNN), 2021: 1–8. https://doi.org/10.1109/IJCNN52387.2021.9534004.
[23] Du R Y, Chang D L, Bhunia A K, et al. Fine-grained visual classification via progressive multi-granularity training of jigsaw patches[C]//Proceedings of the 16th European Conference on Computer Vision, 2020: 153–168. https://doi.org/10.1007/978-3-030-58565-5_10.
[24] Bodapati J D, Shareef N S, Naralasetti V. Composite deep neural network with gated-attention mechanism for diabetic retinopathy severity classification[J]. J Ambient Intell Human Comput, 2021, 12(10): 9825−9839. doi: 10.1007/s12652-020-02727-z
[25] Bodapati J D, Naralasetti V, Shareef S N, et al. Blended multi-modal deep ConvNet features for diabetic retinopathy severity prediction[J]. Electronics, 2020, 9(6): 914. doi: 10.3390/electronics9060914
[26] Kobat S G, Baygin N, Yusufoglu E, et al. Automated diabetic retinopathy detection using horizontal and vertical patch division-based pre-trained DenseNET with digital fundus images[J]. Diagnostics, 2022, 12(8): 1975. doi: 10.3390/diagnostics12081975
-
点击扫一扫
图(9)
表(3)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
