E-mail Alert
RSS
An automatic object detection method for microscopic images based on
attention mechanism
-
摘要:
显微图像普遍具有背景复杂、细胞多重叠的特点，传统的图像处理方法由于其技术局限性，无法实时准确地完成识别任务。针对上述问题，本文提出一种采用注意力机制的显微图像智能检测方法，该方法对目标检测模型DETR进行改进，利用分组卷积机制对输入特征进行降维并分别训练不同卷积核实现特征提取，从而提高模型对于显微图像中目标物体的注意力，并增加了检测准确率。本文以阴道炎致病微生物检测为应用场景，实验结果表明，改进的模型平均查准率均值mAP为96.3%，比原模型提高约10%。同时，本方法在细胞重叠、粘连以及背景杂质繁多等场景下，仍具有高鲁棒性和强检测能力。每张显微图像检测耗时约为88.8 ms，可以满足实时检测的需求。因此，本文提出的模型可实时准确地对常见的阴道炎致病微生物进行实时高效自动检测，可满足显微镜检临床使用的实际需求。
Abstract:The microscopic image has the characteristics of complex background and overlapping cells. Due to the technical limitations, traditional image processing methods cannot accurately complete the real-time recognition task. To address the above-mentioned problems, we propose an automatic detection method for microscopic images using attention mechanism. This method improves the original DETR architecture by introducing a split-transform-merge mechanism, which reduces the dimensionality of input features and trains multiple groups of convolution kernels for feature extraction, thereby effectively improving the model's feature extraction ability for the targets and increasing the accuracy of model detection rate. The experimental results show that the mAP of the improved model was 96.3%, which is 10% higher than that of the original model DETR. Meanwhile, the proposed method has superior detection capabilities for scenarios such as cell overlap, adhesion, and complex background. Moreover, the detection time for each leucorrhea image was about 88.8 ms, which can satisfy the requirement of real-time microscopy examination.
-
Key words:
- microscopic image /
- attention mechanism /
- deep learning /
- vaginitis /
- automatic detection
-
Overview: The microscopic image has the characteristics of complex background and overlapping cells. The traditional microscopy examination is performed by experienced doctors, and usually takes long time. Vaginitis is a common and potentially harmful gynecological disease. It can significantly increase the risk of women being infected with pelvic inflammatory disease, Human Immunodeficiency Virus (HIV), and premature birth. Routine leucorrhea examination is a common early screening method for vaginitis, but traditional microscopic examination takes a long time, and the test results are largely affected by the subjectivity of the doctor. Due to the technical limitations, traditional image processing methods cannot accurately complete the real-time recognition task. To address the above-mentioned problems, we propose an automatic object detection method for microscopic images based on attention mechanism and perform it on the scenario of vaginitis screen. This method can achieve real-time and efficient detection of three common cells, which are mildews, trichomonas and clue cells. The proposed framework uses DETR architecture based on the self-attention mechanism for detection. Different from the commonly used object detection deep learning models, DETR employs Hungarian bipartite match algorithm to assign the prediction and ground truth based on the least cost, which saves the procedure of generating anchor boxes and region of interest proposal. The self-attention mechanism is also used in DETR to generate the global connection of the targets and the whole image, which enables the model to detect the overlapped targets more accurately. In our scenario of microscopic images with complex background, to achieve better performance in image feature extraction section, we replaced the original CNN backbone with a CNN architecture with a split-transform-merge mechanism. It reduces the dimensionality of input features and trains multiple groups of convolution kernels for feature extraction, thereby effectively improving the model's feature extraction ability for the targets and increasing the accuracy of model detection rate. The experimental results show that the mAP of the improved model was 96.3%, which is 10% higher than that of the original model DETR. Meanwhile, the proposed method has superior detection capabilities for scenarios such as cell overlap, adhesion, and complex background. Moreover, the detection time for each leucorrhea image was about 88.8 ms, which can satisfy the requirement of real-time detection. Therefore, this model can accurately identify pathogenic microorganisms in real time and meet the needs of actual clinical use for leucorrhea routine examination. In addition, this method has high transferability and robustness, which can be quickly migrated to microscopic clinical applications such as routine fecal microscopy, urine composition analysis, blood routine analysis, etc.
-
-
图 3 三种常见的阴道炎致病菌显微图像。
Figure 3. Microscopic images of the three common pathogenic cells that cause vaginitis.
图 4 验证集表现结果对比
Figure 4. The performance of the original DETR and the improved DETR on validation dataset
图 5 原始模型和改进的模型的PR曲线对比。
Figure 5. The comparison of PR curves computed from the original model and the improved model.
图 6 三种常见阴道炎致病微生物检测结果图。
Figure 6. The detection results of the three common pathogenic cells.
图 7 模型检测结果和注意力权重可视化图对比。
Figure 7. Comparison of detection results and attention weights visualization map.
表 1 数据集具体划分情况
Table 1. The details of dataset split
数据集类别 图像张数 标注细胞总数 霉菌个数 滴虫个数 线索细胞个数 训练集 653 1844 1757 63 24 验证集 218 593 562 21 10 测试集 218 620 582 30 8 
下载: 导出CSV
表 2 原始模型和改进模型的mAP和AP指标对比
Table 2. The comparison of mAP and AP results of the original model and the proposed model
模型 mAP/% 霉菌AP/% 滴虫AP/% 线索细胞AP/% 原始DETR 87.5 87.0 85.9 89.7 本文改进模型 96.3 93.0 95.8 100.0
下载: 导出CSV
表 3 原始模型和改进模型的查准率和召回率指标对比
Table 3. The comparison of precision and recall results of the original DETR and the proposed improved DETR
评价指标 模型 霉菌 滴虫 线索细胞 查准率(precision)/% 原始DETR 85.3 90.0 100 本文改进模型 88.0 100 100 召回率(recall)/% 原始DETR 85.0 90.0 75.0 本文改进模型 91.0 93.0 100
下载: 导出CSV
-
[1] Paczkowska M, Chanaj-Kaczmarek J, Romaniuk-Drapała A, et al. Mucoadhesive chitosan delivery system with Chelidonii herba lyophilized extract as a promising strategy for vaginitis treatment[J]. J Clin Med, 2020, 9(4): 1208. doi: 10.3390/jcm9041208
[2] Abdul-Aziz M, Mahdy M A K, Abdul-Ghani R, et al. Bacterial vaginosis, vulvovaginal candidiasis and trichomonal vaginitis among reproductive-aged women seeking primary healthcare in Sana'a city, Yemen[J]. BMC Infect Dis, 2019, 19(1): 879. doi: 10.1186/s12879-019-4549-3
[3] Mylonas I, Bergauer F. Diagnosis of vaginal discharge by wet mount microscopy: a simple and underrated method[J]. Obstet Gynecol Surv, 2011, 66(6): 359−368. doi: 10.1097/OGX.0b013e31822bdf31
[4] 田鹏, 严伟, 李凡星, 等. 均匀球面波数字同轴全息生物显微方法[J]. 光电工程, 2019, 46(1): 180110.
Tian P, Yan W, Li F X, et al. Biology microscopy using well-distributed sphere digital in-line holography[J]. Opto-Electron Eng, 2019, 46(1): 180110.
[5] Zhang S L, Liu L W, Ren S, et al. Recent advances in nonlinear optics for bio-imaging applications[J]. Opto-Electron Adv, 2020, 3(10): 200003. doi: 10.29026/oea.2020.200003
[6] 顾鑫, 黄伟, 杨立梅, 等. 微流体衍射相位显微成像及其在寄生虫测量中的应用[J]. 光电工程, 2019, 46(12): 190046.
Gu X, Huang W, Yang L M, et al. Microfluidic diffraction phase microscopy and its application in parasites measurement[J]. Opto-Electron Eng, 2019, 46(12): 190046.
[7] 崔光茫, 张克奇, 毛磊, 等. 结合多尺度分解和梯度绝对值算子的显微图像清晰度评价方法[J]. 光电工程, 2019, 46(6): 180531.
Cui G M, Zhang K Q, Mao L, et al. Micro-image definition evaluation using multi-scale decomposition and gradient absolute value[J]. Opto-Electron Eng, 2019, 46(6): 180531.
[8] 杜晓辉, 刘霖, 张静, 等. 基于LBP纹理特征的白带显微图像中上皮细胞检测方法[J]. 液晶与显示, 2019, 34(9): 871−878.
Du X H, Liu L, Zhang J, et al. Detection of epithelial cells in leucorrhea microscopic images based on LBP texture features[J]. Chin J Liquid Crys Dis, 2019, 34(9): 871−878.
[9] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems, 2012.
[10] Wen T X, Wu H X, Du Y, et al. Faster R-CNN with improved anchor box for cell recognition[J]. Math Biosci Eng, 2020, 17(6): 7772−7786. doi: 10.3934/mbe.2020395
[11] Jia D Y, Zhou J L, Zhang C W. Detection of cervical cells based on improved SSD network[J]. Multimed Tools Appl (2021). doi: 10.1007s11042-021-11015-7.
[12] Jiang Z F, Liu X, Yan Z Z, et al. Improved detection performance in blood cell count by an attention-guided deep learning method[J]. OSA Continuum, 2021, 4(2): 323−333. doi: 10.1364/OSAC.413787
[13] Carion N, Massa F, Synnaeve G, et al. 2020. End-to-end object detection with transformers[C]//Proceedings of the 16th European Conference on Computer Science, 2020: 213–229.
[14] Prangemeier T, Reich C, Koeppl H. Attention-based transformers for instance segmentation of cells in microstructures[C]//Proceedings of 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2020: 700–707.
[15] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770–778.
[16] Kuhn H W. The Hungarian method for the assignment problem[J]. Nav Res Logist Quart, 1955, 2(1-2): 83−97. doi: 10.1002/nav.3800020109
[17] Xie S N, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5987–5995.
-
点击扫一扫
图(8)
表(3)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
