E-mail Alert
RSS
Edge feature and detail-aware network integrated YOLOv8s algorithm for hip joint keypoint detection
-
摘要
髋关节关键点的准确识别对于提高发育性髋关节发育不良诊断精度具有重要意义。然而，在儿童髋关节X射线图像中，关键点所在的骨骼区域通常对比度低和边缘模糊，导致边缘特征不明显。同时，在特征提取过程中，下采样操作会进一步弱化边缘信息。此外，关键点邻域内的关键结构易受背景干扰，这些因素均限制了关键点的精确定位。为此，本文提出了一种融合边缘特征与细节感知网络的YOLOv8s髋关节关键点检测算法。该算法在网络中设计了边缘特征强化模块，以捕获关键点周围空间信息并增强其所在的边缘特征；同时，提出细节感知网络，对多层级特征进行融合与优化，增强对图像中细微结构的感知能力。本文使用重庆医科大学附属儿童医院影像科提供的髋关节X射线图像数据集进行实验，结果显示，关键点的平均定位误差和平均角度误差降低至4.2090 pixel和1.4872°，相较于YOLOv8s降低了6.8%和9.9%，显著优于现有方法。实验证明，本文算法有效提升了关键点的检测精度，为临床诊断提供了重要参考。
-
关键词:
- 发育性髋关节发育不良 /
- 关键点检测 /
- YOLOv8s /
- 边缘特征强化 /
- 细节感知网络
Abstract
The accurate identification of the hip joint keypoint is vital for diagnosing developmental dysplasia of the hip. However, in pediatric hip X-ray images, bone regions around key points often exhibit low contrast and blurred edges, resulting in unclear edge features. Furthermore, down-sampling operations during feature extraction further weaken edge information. Key structures surrounding the keypoint are highly susceptible to background interference. Such factors hinder the precise localization of key points. An edge feature and detail-aware integrated YOLOv8s algorithm was proposed for hip joint key point detection. The algorithm designs an edge feature enhancement module to capture spatial information around key points and strengthen edge features. A detail-aware network was designed to integrate and refine multi-level features, enhancing image perception of fine structures. Experiments used a hip X-ray dataset from the Department of Radiology, Children's Hospital of Chongqing Medical University. Results showed reductions in average keypoint localization and angular errors to 4.2090 pixel and 1.4872°, respectively. These reductions, which are 6.8% and 9.9% compared to those of YOLOv8s, highlight significant improvements in detection accuracy. The algorithm enhances keypoint detection precision and provides valuable support for clinical diagnosis.
-
Overview
Overview: Hip dysplasia is a common orthopedic disease in newborns, and timely and precise diagnosis is critical for optimal patient outcomes. In clinical diagnosis, specific key points on hip joint X-ray images are typically annotated, followed by the calculation of the acetabular index through angular measurements based on these points using goniometric tools. The diagnosis is then determined in combination with the age of the patient. However, manual annotation of key points in the hip joint not only demands that clinicians possess robust professional expertise and extensive clinical experience but also renders the process highly time-consuming and susceptible to subjective bias. Therefore, there is an urgent need for precise and automated key points detection technology to assist doctors in diagnosis. However, traditional template matching methods exhibit poor robustness and generalization when processing complex hip X-ray images, especially when faced challenges such as illumination changes, occlusions, and image rotations. To address these issues, researchers have enhanced the attention mechanism and extracted detailed information around key points using deep learning techniques, thereby improving the accuracy of key points localization. Nonetheless, these methods overlook the significance of bone edge information in assisting recognition and struggle with identifying local neighborhood key structural features, which limits further improvements in localization accuracy. To resolve these problems, an edge feature and detail-aware network integrated with the YOLOv8s algorithm for hip joint key points detection is proposed in this paper. This algorithm introduces an edge feature enhancement module to capture spatial features around key points and enhance the edge features of the bones where they are located. The module is applied multiple times during the feature extraction process of the network to progressively strengthen edge features and guide the network to focus on the key points edge areas. In addition, a detail-aware network is proposed to perform feature fusion and optimization on feature maps at different levels, enhancing the network's ability to capture important fine structures within the local neighborhood of key points. The algorithm was experimentally tested on the hip joint X-ray image dataset provided by the Department of Imaging of the Children's Hospital Affiliated to Chongqing Medical University. The results demonstrate that the average localization error and average angular error for key points have been reduced to 4.2090 pixel and 1.4872°, respectively, representing reductions of 6.8% and 9.9% compared with YOLOv8s, and significantly outperforms existing methods. The experimental findings confirm that the proposed algorithm effectively enhances the accuracy of key point detection, offering valuable insights for clinical diagnosis.
-
-
图 4 YOLOv8s中三种Neck网络结构对比图。(a) Neck；(b) Small-neck；(c) DAN
Figure 4. Comparison diagram of three neck network structures in YOLOv8s. (a) Neck; (b) Small-neck; (c) DAN
图 6 CSP-OKM模块及其子模块结构图。(a) CSP-OKM；(b) FSAM；(c) DCAM；(d) OKM
Figure 6. Structure diagram of CSP-OKM module and associated submodules. (a) CSP-OKM; (b) FSAM; (c) DCAM; (d) OKM
图 7 不同改进模块后的髋关节X射线图像热力图。(a) YOLOv8s；(b) YOLOv8s+EFEM；(c) YOLOv8s+DAN；(d) EDA-YOLOv8s
Figure 7. Heatmaps of hip X-ray image after different improved modules. (a) YOLOv8s; (b) YOLOv8s+EFEM; (c) YOLOv8s+DAN; (d) EDA-YOLOv8s
图 8 使用不同改进模块后的APE与AAE变化关系图
Figure 8. The relationship diagram between APE and AAE after using different improvement modules
图 9 关键点成功检测率实验结果
Figure 9. Experimental results of successful detection rate of keypoint
图 10 关键点定位误差分布对比。(a) YOLOv8s；(b) EDA-YOLOv8s
Figure 10. Comparison of keypoint localization error distribution. (a) YOLOv8s; (b) EDA-YOLOv8s
图 11 EDA-YOLOv8s与YOLOv8s检测结果可视化对比
Figure 11. Visualization comparison of detection results between EDA-YOLOv8s and YOLOv8s
图 12 EDA-YOLOv8s与YOLOv8s关键点角度可视化对比。(a) YOLOv8s；(b) EDA-YOLOv8s
Figure 12. Visualization comparison of keypoint angle between EDA-YOLOv8s and YOLOv8s. (a) YOLOv8s; (b) EDA-YOLOv8s
表 1 评价指标与计算公式
Table 1. Evaluation index and computational formula
Index Calculation formula PELl $ \left( {\displaystyle\sum\limits_{i = 1}^n {\sqrt {{T_{li}} - {P_{li}}} } } \right)/n $ APE $ \left(PEL_{\text{RASM}}+PEL_{\text{RTCC}}+PEL_{\text{LTCC}}+PEL_{\text{LASM}}\right)/4 $ EAk $ \left( {\displaystyle\sum\limits_{i = 1}^n {|{T_{ki}} - {P_{ki}}|} } \right)/n $ AAE $ \left(EA_{\mathrm{R}}+EA_{\mathrm{L}}\right)/2 $ SDR $ \left\{i:\sqrt{\left(x_{T_{li}}-x_{P_{li}}\right)^2+\left(y_{T_{li}}-y_{P_{li}}\right)^2}\leqslant{\textit{z}}\right\}\times100\%/n $ MDR $\dfrac{m}{n} \times 100\% $ 
下载: 导出CSV
表 2 EFEM模块内消融和边缘检测算子对比实验结果
Table 2. The ablation and edge detection operator comparison experiment results in the EFEM module
EFEM (edge detection operator) BN APE/pixel AAE/(°) × × 4.5150 1.6504 Sobel × 4.3652 1.5842 Sobel √ 4.3459 1.5565 Prewitt √ 4.3470 1.5555 Laplacian √ 4.4130 1.6934
下载: 导出CSV
表 3 三种Neck网络算法对比实验结果
Table 3. Comparison experiment results of three neck network algorithms
Neck network APE/pixel AAE/(°) FLOPs/G YOLOv8s
(neck)4.5150 1.6504 28.8 YOLOv8s+
(small-neck)4.7150 1.7322 36.9 YOLOv8s+
(DAN)4.3281 1.5451 34.8
下载: 导出CSV
表 4 不同改进模块对算法的影响
Table 4. The influence of different improved modules on the algorithm
Detection algorithm PELRASM/pixel PELRTCC/pixel PELLTCC/pixel PELLASM/pixel APE/pixel EAR/(°) EAL/(°) AAE/(°) YOLOV8s 4.8476 3.7404 3.4674 6.0045 4.5150 1.6250 1.6757 1.6504 YOLOv8s+EFEM 4.6133 3.7692 3.2258 5.7752 4.3459 1.4886 1.6244 1.5565 YOLOv8s+DAN 4.1284 3.6947 3.3784 6.1110 4.3281 1.2587 1.8315 1.5451 EDA-YOLOv8s 4.2851 3.4870 3.2530 5.8107 4.2090 1.3420 1.6324 1.4872
下载: 导出CSV
表 5 不同关键点检测算法对比实验结果
Table 5. Comparison experiment results of different keypoint detection algorithms
Detection algorithm PELRASM/pixel PELRTCC/pixel PELLTCC/pixel PELLASM/pixel APE/pixel EAR/(°) EAL/(°) AAE/(°) MDR/% YOLOv8n 5.5012 3.8100 3.2056 5.9099 4.6067 1.4698 1.7188 1.5943 0 YOLOv8s 4.8476 3.7404 3.4674 6.0045 4.5150 1.6250 1.6757 1.6504 0 YOLOv9s 4.6463 3.7893 3.4716 6.2252 4.5331 1.4878 1.8033 1.6456 0 YOLOv10s 4.6806 3.7820 3.3007 5.9486 4.4280 1.4594 1.8313 1.6454 6.0 PN-UNet 4.5411 3.7115 3.6881 6.2045 4.5363 1.4384 1.6533 1.5458 0 CircleNet 4.8348 3.7234 3.6113 5.8375 4.5017 1.4988 1.7996 1.6492 0 CBA-YOLOv5s 4.3828 3.6464 3.5506 6.3773 4.4893 1.3235 1.8488 1.5861 0 EDA-YOLOv8s 4.2851 3.4870 3.2530 5.8107 4.2090 1.3420 1.6324 1.4872 0
下载: 导出CSV
-
参考文献
[1] Bradley C S, Verma Y, Maddock C L, et al. A comprehensive nonoperative treatment protocol for developmental dysplasia of the hip in infants: a prospective longitudinal cohort study[J]. Bone Joint J, 2023, 105-B(8): 935−942. doi: 10.1302/0301-620X.105B8.BJJ-2023-0149.R1
[2] Alrashdi N, Alotaibi M, Alharthi M, et al. Incidence, prevalence, risk factors, and clinical treatment for children with developmental dysplasia of the hip in Saudi Arabia. a systematic review[J]. J Epidemiol Glob Health, 2024, 14(3): 549−560. doi: 10.1007/s44197-024-00217-5
[3] Fujii M, Kawano S, Ueno M, et al. Clinical results of periacetabular osteotomy with structural bone allograft for the treatment of severe hip dysplasia[J]. Bone Joint J, 2023, 105-B(7): 743−750. doi: 10.1302/0301-620X.105B7.BJJ-2023-0056.R1
[4] Dornacher D, Lutz B, Fuchs M, et al. Acetabular deficiency in borderline hip dysplasia is underestimated by lateral center edge angle alone[J]. Arch Orthop Trauma Surg, 2023, 143(7): 3937−3944. doi: 10.1007/s00402-022-04652-6
[5] Iyengar K P, Fitzpatrick J D, Michalos M, et al. Birmingham royal orthopaedic hospital (BROH) femoral offset—an ancillary measure of adult dysplasia of the hip[J]. Indian J Radiol Imaging, 2023, 33(4): 471−477. doi: 10.1055/s-0043-1769501
[6] McQuivey K S, Secretov E, Domb B G, et al. A multicenter study of radiographic measures predicting failure of arthroscopy in borderline hip dysplasia: beware of the Tönnis angle[J]. Am J Sports Med, 2020, 48(7): 1608−1615. doi: 10.1177/0363546520914942
[7] Li Q, Zhong L, Huang H N, et al. Auxiliary diagnosis of developmental dysplasia of the hip by automated detection of sharp's angle on standardized anteroposterior pelvic radiographs[J]. Medicine, 2019, 98(52): e18500. doi: 10.1097/MD.0000000000018500
[8] Liu C B, Xie H T, Zhang S C, et al. Misshapen pelvis landmark detection by spatial local correlation mining for diagnosing developmental dysplasia of the hip[C]//Proceedings of the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, 2019: 441–449. https://doi.org/10.1007/978-3-030-32226-7_49.
[9] Al-Bashir A K, Al-Abed M, Sharkh F M A, et al. Algorithm for automatic angles measurement and screening for developmental dysplasia of the hip (DDH)[C]//Proceedings of 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2015: 6386–6389. https://doi.org/10.1109/EMBC.2015.7319854.
[10] Rana M, Bhushan M. Machine learning and deep learning approach for medical image analysis: diagnosis to detection[J]. Multimed Tools Appl, 2023, 82(17): 26731−26769. doi: 10.1007/s11042-022-14305-w
[11] Liu C B, Xie H T, Zhang S C, et al. Misshapen pelvis landmark detection with local-global feature learning for diagnosing developmental dysplasia of the hip[J]. IEEE Trans Med Imaging, 2020, 39(12): 3944−3954. doi: 10.1109/TMI.2020.3008382
[12] Wu H, Xie H T, Liu C B, et al. CircleNet for hip landmark detection[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020, 34 : 12370–12377. https://doi.org/10.1609/aaai.v34i07.6922.
[13] Lv J, Che J L, Chen X. CBA-YOLOv5s: a hip dysplasia detection algorithm based on YOLOv5s using angle consistency and bi-level routing attention[J]. Biomed Signal Process Control, 2024, 95: 106482. doi: 10.1016/j.bspc.2024.106482
[14] Xu W Z, Shu L Q, Gong P, et al. A deep-learning aided diagnostic system in assessing developmental dysplasia of the hip on pediatric pelvic radiographs[J]. Front Pediatr, 2022, 9: 785480. doi: 10.3389/FPED.2021.785480
[15] Xu J Y, Xie H T, Tan Q F, et al. Multi-task hourglass network for online automatic diagnosis of developmental dysplasia of the hip[J]. World Wide Web, 2023, 26(2): 539−559. doi: 10.1007/s11280-022-01051-0
[16] Kim N. Computational modelling for surgical planning of hip dysplasia[D]. Auckland: the University of Auckland, 2023.
[17] Rahmawati S, Devita R, Zain R H, et al. Prewitt and canny methods on inversion image edge detection: an evaluation[J]. J Phys: Conf Ser, 2021, 1933(1): 012039. doi: 10.1088/1742-6596/1933/1/012039
[18] Tian B, Wei W. Research overview on edge detection algorithms based on deep learning and image fusion[J]. Secur Commun Networks, 2022, 2022(1): 1155814. doi: 10.1155/2022/1155814
[19] Xie S N, Tu Z W. Holistically-nested edge detection[C]//Proceedings of 2015 IEEE International Conference on Computer Vision, 2015: 1395–1403. https://doi.org/10.1109/ICCV.2015.164.
[20] Kong L Y, Wang F B, Yang F Y, et al. FISRCN: a single small-sized image super-resolution convolutional neural network by using edge detection[J]. Multimed Tools Appl, 2024, 83(7): 19609−19627. doi: 10.1007/s11042-023-15380-3
[21] Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 779–788. https://doi.org/10.1109/CVPR.2016.91.
[22] Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 936–944. https://doi.org/10.1109/CVPR.2017.106.
[23] Liu S, Qi L, Qin H F, et al. Path aggregation network for instance segmentation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 8759–8768. https://doi.org/10.1109/CVPR.2018.00913.
[24] Cui Y N, Ren W Q, Knoll A. Omni-kernel network for image restoration[C]//Proceedings of the 38th AAAI Conference on Artificial Intelligence, 2024, 38 : 1426–1434. https://doi.org/10.1609/aaai.v38i2.27907.
[25] Zheng J W, Shao A H, Yan Y D, et al. Remote sensing semantic segmentation via boundary supervision-aided multiscale channelwise cross attention network[J]. IEEE Trans Geosci Remote Sens, 2023, 61: 4405814. doi: 10.1109/TGRS.2023.3292112
[26] Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning, 2015: 448–456.
[27] Pan J Y, Zhang Y J. Small object detection in aerial drone imagery based on YOLOv8[J]. IAENG Int J Comput Sci, 2024, 51(9): 1346−1354.
[28] Xu L Y, Zhao Y F, Zhai Y H, et al. Small object detection in UAV images based on YOLOv8n[J]. Int J Comput Intell Syst, 2024, 17(1): 223. doi: 10.1007/s44196-024-00632-3
[29] 谢竞, 邓月明, 王润民. 改进YOLOv8s的交通标志检测算法[J]. 计算机工程, 2024, 50(11): 338−349. doi: 10.19678/j.issn.1000-3428.0068742
Xie J, Deng Y M, Wang R M. Improved traffic sign detection algorithm based on YOLOv8s[J]. Comput Eng, 2024, 50(11): 338−349. doi: 10.19678/j.issn.1000-3428.0068742
[30] Wang C Y, Yeh I H, Mark Liao H Y. YOLOv9: learning what you want to learn using programmable gradient information[C]//Proceedings of the 18th European Conference on Computer Vision, 2024: 1–21. https://doi.org/10.1007/978-3-031-72751-1_1.
[31] Muhammad W, Aramvith S, Onoye T. Multi-scale Xception based depthwise separable convolution for single image super-resolution[J]. PLoS One, 2021, 16(8): e0249278. doi: 10.1371/journal.pone.0249278
[32] 王燕, 王宏辉, 刘树东, 等. 多级特征筛选和任务动态对齐的声呐图像小目标检测[J]. 光电工程, 2024, 51(10): 240196. doi: 10.12086/oee.2024.240196
Wang Y, Wang H H, Liu S D, et al. Small target detection in sonar images with multilevel feature screening and task dynamic alignment[J]. Opto-Electron Eng, 2024, 51(10): 240196. doi: 10.12086/oee.2024.240196
-
访问统计
点击扫一扫
图(13)
表(5)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
