E-mail Alert
RSS
Lightweight YOLOv5 sonar image object detection algorithm and implementation based on ZYNQ
-
摘要:
针对声呐图像存在的模糊、样本量不足的现象，本文提出一种基于YOLOv5的声呐图像目标检测改进算法。利用几何滤波、垂直翻转等方法，对声呐图像数据集进行数据增强。添加融合注意力机制模块，使算法更好地关注声呐图像小目标的特征。同时，针对目前大多数目标检测算法运行在云端，无法做到实时性声呐图像检测的问题，本文利用替换轻量级网络和NCNN边端移植技术，同时在颈部网络中采用GSConv模块，将算法成功移植到ZYNQ平台，实现声呐图像的嵌入式端实时检测。实验表明，本文提出的算法在降低了56%参数量的同时，在map50和map50-95上分别提高2.2%和2.5%。改进后的算法性能提升明显，证明所提出的方法在轻量化声呐图像目标检测任务上具有一定的可行性与有效性。
Abstract:To address the problems of blurring and insufficient sample size in sonar images, an improved sonar image target detection algorithm is proposed based on YOLOv5. The algorithm uses geometric filtering, vertical flipping, and other methods to enhance the sonar image dataset. The fusion attention mechanism module is added to make the algorithm better focus on the features of small targets in sonar images. At the same time, in response to the problem that most target detection algorithms currently run on the cloud and cannot achieve real-time sonar image detection, this paper uses lightweight network replacement and NCNN edge porting technology. It adopts the GSConv module in the neck network to successfully transplant the algorithm to the ZYNQ platform, realizing real-time detection of sonar images on the embedded end. After experiments, the algorithm proposed in this paper reduced the parameter quantity by 56%, increasing map50 and map50-95 by 2.2% and 2.5%, respectively. The algorithm’s performance has significantly improved, proving that the method proposed has certain feasibility and effectiveness in lightweight sonar image target detection tasks.
-
Key words:
- underwater target detection /
- YOLO /
- ZYNQ /
- sonar image /
- deep learning /
- lightweight
-
Overview: In the 21st century, mankind has entered a period of large-scale development and utilization of the ocean. With the strategic mission of becoming a maritime power, we need to improve our ability to develop and utilize marine resources, which requires a large amount of accurate marine environment data to carry out tasks such as underwater target detection and seabed resource detection. Underwater target detection technology is crucial for underwater target positioning and underwater resource exploration. Underwater target detection can be achieved through different imaging technologies, but sonar is currently the most commonly used detection method because it can operate reliably in low visibility conditions. Due to the complexity of underwater acoustic channels, as well as the loss and scattering of sound waves themselves, the image quality and resolution obtained by imaging sonar are low, accompanied by a large amount of speckle noise and unclear edges, which also seriously affects the follow-up of sonar images. Currently, most sonar image target detection algorithms run on the cloud and cannot achieve real-time sonar image detection. This paper proposes a lightweight YOLOv5 sonar image target detection algorithm based on ZYNQ to achieve real-time detection of small target images on the embedded side of sonar equipment. First, geometric filtering, vertical flipping, and other methods are used to perform data enhancement on the sonar image dataset, adding a fusion attention mechanism module allows the algorithm to better focus on the characteristics of small targets in sonar images. At the same time, in order to solve the problem that most target detection algorithms currently run in the cloud and cannot achieve real-time sonar image detection, this paper uses replacement lightweight networks and NCNN edge-end transplantation technology, and uses the GSConv module in the neck network to convert the algorithm successfully ported to ZYNQ platform. The sonar image detection system is independently designed. The PL side uses the wave velocity formation algorithm to generate images, and the PS side realizes the embedded side real-time detection of sonar images. After experiments, the algorithm proposed in this article reduced the calculation amount by 56%, while map50 and map50-95 increased by 2.2% and 2.5%, respectively. The performance of the improved algorithm has been significantly improved, proving that the method proposed in this article has certain feasibility and effectiveness in lightweight sonar image target detection tasks.
-
-
图 1 轻量化声呐图像YOLOv5目标检测网络模型
Figure 1. Lightweight sonar image YOLOv5 object detection network model
表 1 不同设备运行结果
Table 1. Running results of different equipments
Equipment FLOPs/G Memory footprint/MB Frame rate/FPS Power/W Frame rate/power/(FPS/W) GPU 6.0 1401 36.4 250 0.146 CPU 6.0 2103 8.8 65 0.135 ZYNQ7020 5.8 90 0.6 3.7 0.162 
下载: 导出CSV
表 2 对比试验
Table 2. Comparative experiments
Algorithm Map50 Params(M) YOLOv5s 0.912 7.2 EfficientDet 0.834 3.9 YOLOv7 0.890 37 Faster-RCNN 0.896 25.6 SSD 0.912 51 This paper 0.933 3.2
下载: 导出CSV
表 3 消融实验
Table 3. Ablation experiments
Algorithm Precision Recall Map50 Map50-95 Params(M) YOLOv5s 0.896 0.905 0.912 0.685 7.2 YOLOv5s+SimAM 0.892 0.911 0.922 0.700 7.0 YOLOv5s+CBAM 0.896 0.927 0.918 0.697 7.1 YOLOv5s+SimAM+CBAM 0.892 0.912 0.925 0.701 7.1 YOLOv5s+Focal-CIOU 0.899 0.891 0.930 0.700 7.2 YOLOv5s+DWConv 0.880 0.911 0.919 0.694 3.6 YOLOv5s+GSConv 0.897 0.906 0.923 0.699 6.2 This paper 0.907 0.894 0.933 0.712 3.2
下载: 导出CSV
表 4 经典数据集对比实验
Table 4. Comparison experiment of classic datasets
Algorithm Precision Recall Map50 Map50-95 YOLOv5s 0.678 0.53 0.583 0.353 This paper 0.661 0.519 0.59 0.339
下载: 导出CSV
-
[1] 郭戈, 王兴凯, 徐慧朴. 基于声呐图像的水下目标检测、识别与跟踪研究综述[J]. 控制与决策, 2018, 33(5): 906−922. doi: 10.13195/j.kzyjc.2017.1678
Guo G, Wang X K, Xu H P. Review on underwater target detection, recognition and tracking based on sonar image[J]. Control Decis, 2018, 33(5): 906−922. doi: 10.13195/j.kzyjc.2017.1678
[2] Petrich J, Brown M F, Pentzer J L, et al. Side scan sonar based self-localization for small autonomous underwater vehicles[J]. Ocean Eng, 2018, 161: 221−226. doi: 10.1016/j.oceaneng.2018.04.095
[3] Huang S W, Chen E, Guo J. Efficient seafloor classification and submarine cable route design using an autonomous underwater vehicle[J]. IEEE J Oceanic Eng, 2018, 43(1): 7−18. doi: 10.1109/JOE.2017.2686558
[4] Park C, Kim Y, Lee H, et al. Development of a 2 MHz sonar sensor for inspection of bridge substructures[J]. Sensors (Basel), 2018, 18(4): 1222. doi: 10.3390/s18041222
[5] 陈朋, 蔡烜伟, 赵冬冬, 等. 基于自适应BM3D的侧扫声纳图像散斑降噪[J]. 光电工程, 2020, 47(7): 190580. doi: 10.12086/oee.2020.190580
Chen P, Cai X W, Zhao D D, et al. Despeckling for side-scan sonar images based on adaptive block-matching and 3D filtering[J]. Opto-Electron Eng, 2020, 47(7): 190580. doi: 10.12086/oee.2020.190580
[6] Abu A, Diamant R. A statistically-based method for the detection of underwater objects in sonar imagery[J]. IEEE Sensors J, 2019, 19(16): 6858−6871. doi: 10.1109/JSEN.2019.2912325
[7] Negahdaripour S. Application of forward-scan sonar stereo for 3-D scene reconstruction[J]. IEEE J Oceanic Eng, 2020, 45(2): 547−562. doi: 10.1109/JOE.2018.2875574
[8] Shang Z G, Zhao C H, Wan J. Application of multi-resolution analysis in sonar image denoising[J]. J Sys Eng Electron, 2008, 19(6): 1082−1089. doi: 10.1016/S1004-4132(08)60201-7
[9] Jin Y, Ku B, Ahn J, et al. Nonhomogeneous noise removal from side-scan sonar images using structural sparsity[J]. IEEE Geosci Remote Sens Lett, 2019, 16(8): 1215−1219. doi: 10.1109/LGRS.2019.2895843
[10] Wang Z, Zhang S W, Huang W Z, et al. Sonar image target detection based on adaptive global feature enhancement network[J]. IEEE Sensors J, 2022, 22(2): 1509−1530. doi: 10.1109/JSEN.2021.3131645
[11] Mignotte M, Collet C, Pérez P, et al. Markov random field and fuzzy logic modeling in sonar imagery: application to the classification of underwater floor[J]. Comput Vis Image Underst, 2000, 79(1): 4−24. doi: 10.1006/cviu.2000.0844
[12] Köhntopp D, Lehmann B, Kraus D, et al. Classification and localization of naval mines with superellipse active contours[J]. IEEE J Ocean Eng, 2019, 44(3): 767−782. doi: 10.1109/JOE.2018.2835218
[13] Valdenegro-Toro M. End-to-end object detection and recognition in forward-looking sonar images with convolutional neural networks[C]//2016 IEEE/OES Autonomous Underwater Vehicles (AUV), 2016: 144–150. https://doi.org/10.1109/AUV.2016.7778662
[14] Williams D P. Underwater target classification in synthetic aperture sonar imagery using deep convolutional neural networks[C]//2016 23rd International Conference on Pattern Recognition (ICPR), 2016: 2497–2502. https://doi.org/10.1109/ICPR.2016.7900011.
[15] Kim J, Yu S C. Convolutional neural network-based real-time ROV detection using forward-looking sonar image[C]//2016 IEEE/OES Autonomous Underwater Vehicles (AUV), 2016: 396–400. https://doi.org/10.1109/AUV.2016.7778702.
[16] 刘桂雄, 刘思洋, 吴俊芳, 等. 基于深度学习的机器视觉目标检测算法及在票据检测中应用[J]. 中国测试, 2019, 45(5): 1−9. doi: 10.11857/j.issn.1674-5124.2019030130
Liu G X, Liu S Y, Wu J F, et al. Machine vision object detection algorithm based on deep learning and application in banknote detection[J]. China Measur Test Technol, 2019, 45(5): 1−9. doi: 10.11857/j.issn.1674-5124.2019030130
[17] Harikrishnan J, Sudarsan A, Sadashiv A, et al. Vision-face recognition attendance monitoring system for surveillance using deep learning technology and computer vision[C]//2019 International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN), 2019: 1–5. https://doi.org/10.1109/ViTECoN.2019.8899418.
[18] 李宝奇, 黄海宁, 刘纪元, 等. 基于改进SSD的合成孔径声呐图像水下多尺度目标轻量化检测模型[J]. 电子与信息学报, 2021, 43(10): 2854−2862. doi: 10.11999/JEIT201042
Li B Q, Huang H N, Liu J Y, et al. Synthetic aperture sonar underwater multi-scale target efficient detection model based on improved single shot detector[J]. J Electron Inf Technol, 2021, 43(10): 2854−2862. doi: 10.11999/JEIT201042
[19] 陈龙, 张建林, 彭昊, 等. 多尺度注意力与领域自适应的小样本图像识别[J]. 光电工程, 2023, 50(4): 220232. doi: 10.12086/oee.2023.220232
Chen L, Zhang J L, Peng H, et al. Few-shot image classification via multi-scale attention and domain adaptation[J]. Opto-Electron Eng, 2023, 50(4): 220232. doi: 10.12086/oee.2023.220232
[20] 葛锡云, 魏柠阳, 周宏坤, 等. 基于侧扫声呐的水下小目标检测技术研究[J]. 数字海洋与水下攻防, 2023, 6(2): 155−161. doi: 10.19838/j.issn.2096-5753.2023.02.004
Ge X Y, Wei N Y, Zhou H K, et al. Research on small underwater target detection technology based on side-scan sonar[J]. Digit Ocean Underwater Warf, 2023, 6(2): 155−161. doi: 10.19838/j.issn.2096-5753.2023.02.004
[21] Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: optimal speed and accuracy of object detection[Z]. arXiv: 2004.10934, 2020. https://doi.org/10.48550/arXiv.2004.10934.
[22] Singh A, Kalaichelvi V, Karthikeyan R. Performance analysis of object detection algorithms for robotic welding applications in planar environment[J]. Int J Comput Integr Manuf, 2023, 36(7): 1083−1108. doi: 10.1080/0951192X.2022.2162601
[23] 陈旭, 彭冬亮, 谷雨. 基于改进YOLOv5s的无人机图像实时目标检测[J]. 光电工程, 2022, 49(3): 210372. doi: 10.12086/oee.2022.210372
Chen X, Peng D L, Gu Y. Real-time object detection for UAV images based on improved YOLOv5s[J]. Opto-Electron Eng, 2022, 49(3): 210372. doi: 10.12086/oee.2022.210372
[24] Ge Z, Liu S T, Wang F, et al. YOLOX: exceeding YOLO series in 2021[Z]. arXiv: 2107.08430, 2021. https://doi.org/10.48550/arXiv.2107.08430.
[25] Ying B Y, Xu Y C, Zhang S, et al. Weed detection in images of carrot fields based on improved YOLO v4[J]. Traitem Signal, 2021, 38(2): 341−348. doi: 10.18280/ts.380211
[26] 黄磊, 杨媛, 杨成煜, 等. FS-YOLOv5: 轻量化红外目标检测方法[J]. 计算机工程与应用, 2023, 59(9): 215−224. doi: 10.3778/j.issn.1002-8331.2210-0487
Huang L, Yang Y, Yang C Y, et al. FS-YOLOv5: lightweight infrared rode target detection method[J]. Comput Eng Appl, 2023, 59(9): 215−224. doi: 10.3778/j.issn.1002-8331.2210-0487
[27] Li H L, Li J, Wei H B, et al. Slim-neck by GSConv: a better design paradigm of detector architectures for autonomous vehicles[Z]. arXiv: 2206.02424, 2022. https://doi.org/10.48550/arXiv.2206.02424.
[28] 奉志强, 谢志军, 包正伟, 等. 基于改进YOLOv5的无人机实时密集小目标检测算法[J]. 航空学报, 2023, 44(7): 327106. doi: 10.7527/s1000-6893.2022.27106
Feng Z Q, Xie Z J, Bao Z W, et al. Real-time dense small object detection algorithm for UAV based on improved YOLOv5[J]. Acta Aeronaut Astronaut Sin, 2023, 44(7): 327106. doi: 10.7527/s1000-6893.2022.27106
[29] Zhu X K, Lyu S C, Wang X, et al. TPH-YOLOv5: improved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops, 2021: 2778–2788. https://doi.org/10.1109/ICCVW54120.2021.00312.
[30] Niu Z Y, Zhong G Q, Yu H. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48−62. doi: 10.1016/j.neucom.2021.03.091
[31] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 7132–7141. https://doi.org/10.1109/CVPR.2018.00745.
[32] Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision, 2018: 3–19. https://doi.org/10.1007/978-3-030-01234-2_1.
[33] Yang L X, Zhang R Y, Li L D, et al. SimAM: a simple, parameter-free attention module for convolutional neural networks[C]//Proceedings of the 38th International Conference on Machine Learning, 2021: 11863–11874.
[34] Zheng Z H, Wang P, Liu W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020: 12993–13000. https://doi.org/10.1609/aaai.v34i07.6999.
[35] 袁明阳, 宋亚林, 张潮, 等. 基于GA-RetinaNet的水下目标检测[J]. 计算机系统应用, 2023, 32(6): 80−90. doi: 10.15888/j.cnki.csa.009134
Yuan M Y, Song Y L, Zhang C, et al. Underwater object detection based on GA-RetinaNet[J]. Comput Syst Appl, 2023, 32(6): 80−90. doi: 10.15888/j.cnki.csa.009134
[36] Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, 2017: 2999–3007. https://doi.org/10.1109/ICCV.2017.324.
-
点击扫一扫
图(12)
表(4)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
