E-mail Alert
RSS
-
摘要:
针对金属表面缺陷检测效率低以及检测算法参数量大、精度低的问题，本文提出了一种改进YOLOv8n的金属表面缺陷检测轻量化方法。首先，设计局部卷积倒置交叉融合 (partial inverted bottleneck cross stage partial fusion, PIC2f)模块，该模块通过构造的局部卷积倒置瓶颈 (partial IRMB bottleneck, PIBN)模块替换BottleNeck模块，将部分卷积和倒置残差块组合，从而减少算法的参数量并提升模型的特征提取能力。然后，采用基于注意力尺寸内特征交互 (attention-based intra-scale feature interaction, AIFI)模块，该模块结合位置嵌入和多头注意力机制，增强了模型对小目标的检测能力。最后，使用平均池化下采样 (average pooling down sampling, ADown)模块替换传统卷积作为模型特征缩减模块，通过池化和卷积操作，在不降低检测精度情况下，进一步减少模型的参数量和计算复杂度。实验结果表明，与YOLOv8n算法相比，在NEU-DET钢材缺陷数据集上的PIC2f-YOLO方法的mAP50增加了2.7%，参数量减少了0.403 M。在铝片表面工业缺陷、PASCAL VOC2012和带状合金功能材料表面缺陷数据集上的泛化性实验也验证了PIC2f-YOLO方法的有效性。
Abstract:To address the low efficiency in metal surface defect detection, and the problems related to numerous model parameters and low precision, a lightweight detection method based on an improved YOLOv8n was proposed. The partially inverted bottleneck cross-stage partial fusion (PIC2f) module was introduced, replacing the bottleneck module with a partial IRMB bottleneck (PIBN) module. This combination of partial convolution and inverted residual blocks reduced the algorithm’s parameters and enhanced the model’s feature extraction ability. The attention-based intra-scale feature interaction (AIFI) module was applied, integrating location embedding and multi-head attention to improve the model’s small-target detection performance. Lastly, the average pooling down sampling (ADown) module replaced traditional convolution as the feature reduction module, reducing parameters and computational complexity while maintaining detection accuracy. The experimental results show that, compared to YOLOv8n, the PIC2f-YOLO method improves mAP50 by 2.7% on the NEU-DET steel defect dataset and reduces parameters by 0.403 M. Generalization experiments on aluminum sheet surface industrial defects, PASCAL VOC2012 and surface defects of strip alloy functional material datasets also confirm the method’s effectiveness.
-
Key words:
- surface defect detection /
- partial convolution /
- inverted residual /
- attention mechanism
-
Overview: Metallic materials, especially steel, are widely used in industry due to their mature manufacturing processes, excellent durability, and significant economic benefits. As a core material in various manufacturing sectors, steel is critical in advancing global industrialization and information technology. In the early stages of industrial metal production, surface defect detection relied primarily on manual visual inspections. However, as production scales expanded and quality demands increased, the limitations of manual inspection methods, such as inefficiency, susceptibility to human error, and high subjectivity, became evident. Thus, quickly and reliably identifying surface defects on metal has become a critical challenge in modern manufacturing processes. Traditional nondestructive testing techniques, such as eddy current testing and infrared inspection, were initially applied to surface defect detection on metal materials. However, these techniques often struggle to detect small defects effectively. Later, machine learning methods were introduced to improve detection accuracy, but limitations in processing power and generalization hindered their performance in large-scale industrial applications. With the rapid development of computing hardware, deep-learning-based methods have become the primary solution for metal surface defect detection. These methods are usually divided into two-stage and one-stage object detection frameworks. Two-stage methods, such as Faster RCNN, are recognized for their high accuracy but have slower processing speeds, limiting their real-time applications. Conversely, one-stage methods, such as YOLO, provide faster detection speeds but often compromise accuracy. To address the low efficiency in metal surface defect detection, as well as problems related to large algorithm parameters and low precision, this paper proposes a lightweight detection method based on the improved YOLOv8n. First, the local convolution inverted cross-stage partial fusion (PIC2f) module was designed. It replaces the BottleNeck module with the constructed local convolution inverted residual bottleneck (PIBN) module, which combines partial convolution and an inverted residual block to reduce algorithm parameters and enhance the model's feature extraction ability. Next, an attention-based intra-scale feature interaction (AIFI) module was adopted. It combines position embedding and multi-head attention to enhance the model's detection capability for small targets. Lastly, average pooling downsampling (ADown) replaced traditional convolution as the feature reduction module, effectively reducing parameters and computational complexity without compromising detection accuracy. Experimental results show that, compared with YOLOv8n, the PIC2f-YOLO method increases mAP50 by 2.7 % on the NEU-DET steel defect dataset and reduces parameters by 0.403 M. Experiments on aluminum sheet surface industrial defects, PASCAL VOC2012, and surface defects of strip alloy functional material datasets also confirm the effectiveness of the PIC2f-YOLO method.
-
-
图 4 Conv和PConv结构。(a) Conv；(b) PConv
Figure 4. Conv and PConv structure. (a) Conv; (b) PConv
图 10 数据集增强前后缺陷mAP50-95值对比
Figure 10. Comparison of defect mAP50-95 values before and after dataset augmentation
图 11 不同方法在NEU-DET数据集下的检测实例。(a)标注的缺陷图像；(b) YOLOv8n；(c) PIC2f-YOLO
Figure 11. Detection instance of different methods under the NEU-DET dataset. (a) Annotated defect images; (b) YOLOv8n; (c) PIC2f-YOLO
图 12 不同方法在带状合金功能材料表面缺陷数据集下的检测实例。(a)标注的缺陷图像；(b)YOLOv8n；(c)PIC2f-YOLO
Figure 12. Detection instance of different methods under the surface defects of strip alloy functional materials datasets. (a) Annotated defect images; (b) YOLOv8n; (c) PIC2f-YOLO
表 1 数据增强前后数据集中各类缺陷标签数
Table 1. Number of defect labels for each class in the dataset before and after data augmentation
Defects Training set Validation set Testing set Swelling 337/643 38/104 43/73 Dent 493/981 62/120 65/118 Scratch 1752/3006 189/375 228/405 Peeling 1145/1981 141/244 180/237 Gap 417/632 52/71 39/86 Perforation 131/239 11/20 17/27 Weld 285/432 29/51 30/47 Snake 119/254 27/33 19/30 
下载: 导出CSV
表 2 消融实验结果
Table 2. Ablation results
Methods mAP50/% Param/M FLOPs/G FPS/(f/s) Baseline 75.3 3.006 8.1 163 +M1 76.9 2.788 11.5 98 +M2 77.1 3.238 8.1 128 +M3 76.2 2.591 7.2 153 +M1M2 77.6 3.019 11.5 84 +M1M2M3 78.0 2.603 10.6 82
下载: 导出CSV
表 3 PIC2f 实验结果
Table 3. PIC2f experiment results
PConv IRMB mAP50/% Param/M 75.3 3.006 √ 75.6 2.589 √ k=1 75.8 2.783 √ k=3 76.9 2.788 √ k=5 75.6 2.797 √ k=7 74.9 2.811 √ k=9 75.3 2.829
下载: 导出CSV
表 4 铝片表面工业缺陷数据集泛化实验
Table 4. Generalization experiment results on the aluminum sheet surface industrial defect dataset
Methods mAP50/% Param/M FPS/(f/s) YOLOv8n 93.8 3.006 190 PIC2f-YOLO 95.0 2.603 88
下载: 导出CSV
表 5 PASCAL VOC2012数据集泛化实验
Table 5. Generalization experiment results on the PASCAL VOC2012 dataset
Methods mAP50/% Param/M FPS/(f/s) YOLOv8n 58.8 3.009 71 PIC2f-YOLO 59.2 2.606 45
下载: 导出CSV
表 6 NEU-DET数据集实验效果对比
Table 6. Comparison experiment results on the NEU-DET dataset
Methods mAP50/% mAP50-95 /% Param/M FLOPs /G FPS /(f/s) Faster-RCNN 76.7 36.3 41.37 134.0 32 YOLOv3-tiny 69.6 35.4 12.13 18.9 235 YOLOv5n 74.1 40.6 2.50 7.1 160 YOLOv6n 70.0 36.6 4.23 11.8 180 YOLOv7-tiny 67.8 31.2 6.03 13.2 102 YOLOXs 74.6 39.5 8.94 26.8 97 YOLOv9-tiny 75.2 41.9 2.62 10.7 130 YOLOv10n 70.9 38.9 2.69 6.7 140 YOLOv8n 75.3 40.7 3.01 8.1 163 PIC2f-YOLO 78.0 43.6 2.60 10.6 82
下载: 导出CSV
表 7 PASCAL VOC2012数据集实验效果对比
Table 7. Comparison experiment results on the PASCAL VOC2012 dataset
Methods mAP50/% mAP50-95/% Param/M FLOPs/G FPS/(f/s) Faster-RCNN 56.0 40.7 41.43 134.0 13 YOLOv3-tiny 52.6 31.8 12.13 18.9 107 YOLOv5n 58.0 40.0 2.51 7.1 71 YOLOv6n 58.9 43.0 4.24 11.8 72 YOLOv7-tiny 57.9 40.4 6.06 13.2 58 YOLOXs 57.6 39.5 8.96 26.8 52 YOLOv9-tiny 59.1 44.8 2.62 10.7 92 YOLOv10n 58.9 43.1 2.70 6.7 95 YOLOv8n 58.8 40.9 3.01 8.1 71 PIC2f-YOLO 59.2 41.6 2.61 10.6 45
下载: 导出CSV
表 8 带状合金功能材料表面缺陷数据集实验效果对比
Table 8. Comparison experiment results on the surface defects of strip alloy functional material dataset
Methods mAP50/% mAP50-95/% Param/M FLOPs/G FPS/(f/s) Faster-RCNN 57.2 30.6 41.43 134.0 20 YOLOv3-tiny 63.5 33.7 12.13 18.9 151 YOLOv5n 72.3 38.6 2.50 7.1 108 YOLOv6n 68.9 37.4 4.23 11.8 117 YOLOv7-tiny 69.7 37.4 6.02 13.1 98 YOLOXs 66.7 34.5 8.94 26.8 95 YOLOv9-tiny 71.3 37.9 2.62 10.7 105 YOLOv10n 64.3 36.8 2.69 6.7 110 YOLOv8n 74.1 42.1 3.01 8.1 113 PIC2f-YOLO 75.6 42.8 2.61 10.6 60
下载: 导出CSV
-
[1] 李国权. 高炉无料钟炉顶设备维护发展现状分析[J]. 设备管理与维修, 2019, (6): 35−36. doi: 10.16621/j.cnki.issn1001-0599.2019.03D.17
Li G Q. Analysis on the development status of bell-less top equipment maintenance for blast furnaces[J]. Plant Maintenance Eng, 2019, (6): 35−36. doi: 10.16621/j.cnki.issn1001-0599.2019.03D.17
[2] 潘晨燕. 基于改进SPIHT算法的钢板表面缺陷漏磁检测海量数据压缩方法研究[D]. 沈阳: 东北大学, 2013.
Pan C Y. Research on data compression method of massive magnetic flux leakage detection data of surface defects on steel plate based on improved SPIHT algorithm[D]. Shenyang: Northeastern University, 2013.
[3] 徐长航, 陈国明, 谢静. 红外图像处理技术在金属表面缺陷检测中的应用[J]. 制造业自动化, 2009, 31(10): 51−54. doi: 10.3969/j.issn.1009-0134.2009.10.015
Xu C H, Chen G M, Xie J. Application of infrared thermography technology in surface defects detection of products in metal[J]. Manuf Autom Eng, 2009, 31(10): 51−54. doi: 10.3969/j.issn.1009-0134.2009.10.015
[4] Li X D, Mao W J, Jiang W. Image recognition for steel ball's surface quality detecting based on kernel extreme learning machine[C]//Proceedings of the 34th Chinese Control Conference, 2015: 3727–3731. https://doi.org/10.1109/ChiCC.2015.7260217.
[5] Lu J L, Lin M X, Huang Y, et al. A high-accuracy algorithm for surface defect detection of steel based on DAG-SVM[J]. Sens Transducers, 2013, 157(10): 412−418.
[6] Zhang Z B, Lv G H, Zhao G X, et al. BS-YOLOv5s: insulator defect detection with attention mechanism and multi-scale fusion[C]//Proceedings of 2023 IEEE International Conference on Image Processing, 2023: 2365–2369. https://doi.org/10.1109/ICIP49359.2023.10222163.
[7] Ding K Y, Ding Z H, Zhang Z B, et al. SCD-YOLO: a novel object detection method for efficient road crack detection[J]. Multimedia Syst, 2024, 30(6): 351. doi: 10.21203/rs.3.rs-4350707/v1
[8] 朱克佳. 基于深度学习的目标检测研究[J]. 现代信息科技, 2024, 8(13): 76−83. doi: 10.19850/j.cnki.2096-4706.2024.13.016
Zhu K J. Research on object detection based on deep learning[J]. Mod Inf Technol, 2024, 8(13): 76−83. doi: 10.19850/j.cnki.2096-4706.2024.13.016
[9] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39(6): 1137−1149. doi: 10.1109/TPAMI.2016.2577031
[10] 刘琪, 雷景生. 基于改进深度网络的钢材表面缺陷检测[J]. 计算机工程与设计, 2022, 43(9): 2654−2661. doi: 10.16208/j.issn1000-7024.2022.09.033
Liu Q, Lei J S. Surface defect detection of steel based on improved deep network[J]. Comput Eng Design, 2022, 43(9): 2654−2661. doi: 10.16208/j.issn1000-7024.2022.09.033
[11] Shi X C, Zhou S K, Tai Y C, et al. An improved Faster R-CNN for steel surface defect detection[C]//Proceedings of 2022 IEEE 24th International Workshop on Multimedia Signal Processing, 2022: 1–5. https://doi.org/10.1109/MMSP55362.2022.9949350.
[12] 姜菲菲, 李宁, 邱翠翠, 等. 基于改进Faster RCNN的金属丝网缺陷检测方法[J]. 中国科技论文, 2024, 19(2): 153−159. doi: 10.3969/j.issn.2095-2783.2024.02.003
Jiang F F, Li N, Qiu C C, et al. Defect detection method of wire mesh based on improved Faster RCNN[J]. China Sciencepaper, 2024, 19(2): 153−159. doi: 10.3969/j.issn.2095-2783.2024.02.003
[13] 马燕婷, 赵红东, 阎超, 等. 改进YOLOv5网络的带钢表面缺陷检测方法[J]. 电子测量与仪器学报, 2022, 36(8): 150−157. doi: 10.13382/j.jemi.B2205354
Ma Y T, Zhao H D, Yan C, et al. Strip steel surface defect detection method by improved YOLOv5 network[J]. J Electron Meas Instrum, 2022, 36(8): 150−157. doi: 10.13382/j.jemi.B2205354
[14] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000–6010.
[15] 李鑫, 汪诚, 李彬, 等. 改进YOLOv5的钢材表面缺陷检测算法[J]. 空军工程大学学报, 2022, 23(2): 26−33. doi: 10.3969/j.issn.1009-3516.2022.02.005
Li X, Wang C, Li B, et al. Steel surface defect detection algorithm based on Improved YOLOv 5[J]. J Air Force Eng Univ, 2022, 23(2): 26−33. doi: 10.3969/j.issn.1009-3516.2022.02.005
[16] 熊聪, 于安宁, 高兴华, 等. 基于改进YOLOX的钢材表面缺陷检测算法[J]. 电子测量技术, 2023, 46(9): 151−157. doi: 10.19651/j.cnki.emt.2211012
Xiong C, Yu A N, Gao X H, et al. Steel surface defect detection algorithm based on improved YOLOX[J]. Electron Meas Technol, 2023, 46(9): 151−157. doi: 10.19651/j.cnki.emt.2211012
[17] 梁礼明, 龙鹏威, 卢宝贺, 等. 改进GBS-YOLOv7t的钢材表面缺陷检测[J]. 光电工程, 2024, 51(5): 240044. doi: 10.12086/oee.2024.240044
Liang L M, Long P W, Lu B H, et al. Improvement of GBS-YOLOv7t for steel surface defect detection[J]. Opto-Electron Eng, 2024, 51(5): 240044. doi: 10.12086/oee.2024.240044
[18] 杨本臣, 李世熙, 李依泽, 等. 改进YOLOv8n的钢材表面缺陷检测算法[J/OL]. 机械科学与技术, 2024: 1–10. [2024-09-20].https://doi.org/10.13433/j.cnki.1003-8728.20240135.
Yang B C, Li S X, Li Y Z, et al. Improved YOLOv8n algorithm for steel surface defect detection[J/OL]. Mech Sci Technol Aerosp Eng, 2024: 1–10. [2024-09-20].https://doi.org/10.13433/j.cnki.1003-8728.20240135.
[19] Ultralytics. YOLOv8: the latest version of YOLO object detection algorithm[EB/OL]. GitHub, 2023. [2024-08-22].https://github.com/ultralytics/ultralytics.
[20] Chen J R, Kao S H, He H, et al. Run, don't walk: chasing higher FLOPS for faster neural networks[C]//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023: 12021–12031. https://doi.org/10.1109/CVPR52729.2023.01157.
[21] Zhang J N, Li X T, Li J, et al. Rethinking mobile block for efficient attention-based models[C]//Proceedings of 2023 IEEE/CVF International Conference on Computer Vision, 2023: 1389–1400. https://doi.org/10.1109/ICCV51070.2023.00134.
[22] Zhao Y, Lv W Y, Xu S J, et al. DETRs beat YOLOs on real-time object detection[C]//Proceedings of 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024: 16965–16974. https://doi.org/10.1109/CVPR52733.2024.01605.
[23] Wang C Y, Yeh I H, Liao H Y M, et al. YOLOv9: learning what you want to learn using programmable gradient information[C]//Proceedings of the 18th European Conference on Computer Vision, 2024: 1–21. https://doi.org/10.1007/978-3-031-72751-1_1.
[24] He Y, Song K C, Meng Q G, et al. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features[J]. IEEE Trans Instrum Meas, 2020, 69(4): 1493−1504. doi: 10.1109/TIM.2019.2915404
[25] 铝片表面工业缺陷目标检测[EB/OL]. Baidu AI Studio, 2022. [2024-09-25].https://aistudio.baidu.com/datasetdetail/135640.
Lion C. Surface defect detection for aluminum sheets in iIndustrial applications[EB/OL]. Baidu AI Studio, 2022. [2024-09-25].https://aistudio.baidu.com/datasetdetail/135640. Lion C.
[26] Everingham M, Eslami S M A, Van Gool L, et al. The PASCAL visual object classes challenge: a retrospective[J]. Int J Comput Vision, 2015, 111(1): 98−136. doi: 10.1007/s11263-014-0733-5
[27] 带状合金功能材料表面缺陷数据集[EB/OL]. Baidu AI Studio, 2024. [2024-10-22].https://aistudio.baidu.com/datasetdetail/299785.
Hu Y L. Dataset of surface defects of strip alloy functional materials[EB/OL]. Baidu AI Studio, 2024. [2024-10-22].https://aistudio.baidu.com/datasetdetail/299785. Hu Y L.
[28] 易遵辉, 蒋朝辉, 陈晓方, 等. 基于成像区域模型的工业相机视场角优化配置方法[J/OL]. 控制理论与应用, 2023: 1–9. [2024-11-29].http://kns.cnki.net/kcms/detail/44.1240.TP.20231214.0848.038.html.
Yi Z H, Jiang C H, Chen X F, et al. Optimal configuration method for field angle of industrial cameras based on imaging area model[J/OL]. Control Theory Appl, 2023: 1–9. [2024-11-29].http://kns.cnki.net/kcms/detail/44.1240.TP.20231214.0848.038.html.
[29] 帅金晓. 单芯片CMOS图像传感器数字系统的设计与实现[D]. 湖南: 湖南大学, 2009.
Shuai J X. The design and implementation of the digital system of a System-on-a-Chip CMOS image sensor[D]. Hunan: Hunan University, 2009.
[30] Redmon J, Farhadi A. YOLOv3: an incremental improvement[Z]. arXiv: 1804.02767, 2018. https://arxiv.org/abs/1804.02767.
[31] Li C Y, Li L L, Jiag H L, et al. YOLOv6: a single-stage object detection framework for industrial applications[Z]. arXiv: 2209.02976, 2022. https://arxiv.org/abs/2209.02976.
[32] Wang C Y, Bochkovskiy A, Liao H Y M. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[C]//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023: 7464–7475. https://doi.org/10.1109/CVPR52729.2023.00721.
[33] Ge Z, Liu S T, Wang F, et al. YOLOX: exceeding YOLO series in 2021[Z]. arXiv: 2107.08430, 2021. https://arxiv.org/abs/2107.08430.
[34] Wang A, Chen H, Liu L H, et al. YOLOv10: real-time end-to-end object detection[Z]. arXiv: 2405.14458, 2024. https://arxiv.org/abs/2405.14458.
-
点击扫一扫
图(13)
表(8)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
