结合极坐标建模与神经网络的IVUS图像分割

刘靖雨,蔡怀宇,郝文月,等. 结合极坐标建模与神经网络的IVUS图像分割[J]. 光电工程,2023,50(1): 220118. doi: 10.12086/oee.2023.220118
引用本文: 刘靖雨,蔡怀宇,郝文月,等. 结合极坐标建模与神经网络的IVUS图像分割[J]. 光电工程,2023,50(1): 220118. doi: 10.12086/oee.2023.220118
Liu J Y, Cai H Y, Hao W Y, et al. Intravascular ultrasound image segmentation combining polar coordinate modeling and a neural network[J]. Opto-Electron Eng, 2023, 50(1): 220118. doi: 10.12086/oee.2023.220118
Citation: Liu J Y, Cai H Y, Hao W Y, et al. Intravascular ultrasound image segmentation combining polar coordinate modeling and a neural network[J]. Opto-Electron Eng, 2023, 50(1): 220118. doi: 10.12086/oee.2023.220118

结合极坐标建模与神经网络的IVUS图像分割

  • 基金项目:
    血管超声三维实时成像算法研究及软件平台设计 (2021GKF-0422)
详细信息
    作者简介:
    *通讯作者: 蔡怀宇,hycai@tju.edu.cn
  • 中图分类号: R54;TP391.41

Intravascular ultrasound image segmentation combining polar coordinate modeling and a neural network

  • Fund Project: Research on Vascular Ultrasound 3D Real-time Imaging Algorithm and Software Platform Design (2021GKF-0422)
More Information
  • 针对现有血管内超声(IVUS)图像分割网络不能保证分割结果之间的拓扑关系符合医学先验知识,影响后续临床参数计算的问题,提出了一种基于极坐标建模和密集距离回归网络的IVUS图像分割方法。首先通过极坐标建模将含有先验知识的二维掩膜编码为一维距离向量;然后构建一个结合残差网络和语义嵌入模块的密集距离回归网络,用于学习IVUS图像和一维距离向量之间的映射关系。同时提出联合损失函数约束网络的学习方向。预测结果最终通过样条曲线拟合被重建为二维掩模。实验结果表明,所提方法在血管、管腔和斑块区域的分割结果拓扑关系100%符合先验知识,Jaccard测量值分别达到0.89、0.87和0.74。该算法适用于一般的IVUS图像分割,分割结果中血管结构定位准确,拓扑关系正确,可提供可靠的临床参数。

  • 加载中
  • 图 1  理想假设示意图。(a) 符合理想假设;(b) 不符合理想假设

    Figure 1.  Ideal hypothesis diagrams. (a) The mask image that meets the ideal hypothesis; (b) The situation that does not meet the ideal hypothesis

    图 2  建模示意图。 (a) IVUS 原图;(b) 建模结果示意图。内膜边界和中-外膜边界分别用红色和绿色曲线标记,管腔区域和斑块区域建模结果分别使用红色和绿色线段标记

    Figure 2.  Modeling schematics. (a) Original image of IVUS; (b) Schematic diagram of modeling result. The intima contour and media contour are marked with red and green curves, respectively. The modeling results of the lumen area and plaque area are marked with red and green line segments, respectively

    图 3  密集距离回归网络结构

    Figure 3.  The proposed dense distance of regression network

    图 4  真值与预测值斑块区域交并情况示意图。注:为便于观察,将真值射线与预测值射线错开一定角度表示,实际二者在同一射线上

    Figure 4.  Schematic diagram of the intersection of the true value and the predicted value patch area. Note: For the convenience of observation, the true value ray and the predicted value ray are staggered by a certain angle, and the two are actually on the same ray

    图 5  JM随射线数量变化曲线图

    Figure 5.  The graph of JM changing with the number of rays

    图 6  不同建模方式分割结果可视化

    Figure 6.  Visualization of segmentation results of different modeling methods

    图 7  分割结果视觉效果对比

    Figure 7.  Comparison of the visual effects of the segmentation results

    图 8  关键临床参数线性回归分析

    Figure 8.  Linear regression analysis of key clinical parameters

    图 9  关键临床参数 Bland-Altman 分析

    Figure 9.  Bland-Altman analysis of key clinical parameters

    表 1  IVUS数据集信息

    Table 1.  Information of the IVUS dataset

    患者标号1234
    图像数量21839318168
    下载: 导出CSV

    表 2  不同深度的骨架网络与不同数量SEB模块组合实验结果

    Table 2.  The performance of the proposed method under different depths of backbone and different numbers of SEB modules

    BackboneSEB numJMHD/mmPADTER
    MedLumPlaqueMedLumMedLum-
    ResNet1800.86300.85890.69350.23610.15010.10770.10380
    10.86580.85200.69470.22520.16030.10480.11740
    20.86590.86340.69790.21670.15550.10780.10390
    30.86550.85980.70160.22580.15130.11170.10380
    ResNet3400.88660.86740.73020.19060.14460.08490.09500
    10.88030.86060.71730.20800.14750.08910.10130
    20.87160.86100.70710.23570.15580.09790.10460
    30.88180.85740.71670.22270.15590.08330.10770
    ResNet5000.88040.86920.72210.18550.14620.09370.09670
    10.87380.86870.71310.22480.13850.09780.09460
    20.87600.86710.71520.22660.14450.09010.09900
    30.88050.87570.72110.21530.13510.09150.08790
    ResNet10100.88530.86650.72980.20140.13800.08600.09280
    10.89100.86460.73460.18730.15530.08160.10990
    20.89340.87380.74300.17610.13550.07940.10050
    30.89020.87250.73370.17750.13960.07450.08270
    ResNet15200.88520.86460.73020.20120.14120.08810.10360
    10.88450.87110.72560.20730.14020.08740.09580
    20.89740.87080.74240.18790.14810.07360.09620
    30.88490.86270.72550.21010.15450.08540.10880
    下载: 导出CSV

    表 3  不同损失函数下的实验结果

    Table 3.  Experimental results with different loss functions

    Loss functionJMHD/mmPADTER
    MedLumPlaqueMedLumMedLum-
    Smoothl10.87320.86980.70470.21310.14360.09990.09300
    Ll+Lp0.88500.87570.73190.21450.13500.09120.09290
    Lm+Lp0.89040.86360.73130.19970.15090.07940.11160
    Ll+Lm0.88080.87360.71830.21720.14470.08940.08920
    IVUS Polar IoU Loss0.89340.87380.74300.17610.13550.07940.10050
    下载: 导出CSV

    表 4  不同建模方式下的实验结果

    Table 4.  Experimental results with different modeling methods

    建模方式LossJMHD/mmPADTER
    MedLumPlaqueMedLumMedLum-
    EllipseSmoothl10.82080.81240.61000.26330.17930.13990.14020.0767
    PCM-PK0.87320.86980.70470.21310.14360.09990.09300
    下载: 导出CSV

    表 5  不同分割模型的性能比较

    Table 5.  Performance comparison of different segmentation models

    BackboneJMHD/mmPADTER
    MedLumPlaqueMedLumMedLum-
    SegNet[30]-0.88560.86180.71480.53671.57830.09480.12450.2328
    UNet[31]-0.88570.88460.73000.47000.17760.09560.09500.1319
    Deeplabv3+[29]ResNet1010.90260.88860.75670.24270.13020.06770.07870.0390
    OursResNet1010.89340.87380.74300.17610.13550.07940.10050
    下载: 导出CSV

    表 6  关键临床参数线性回归分析结果

    Table 6.  Results of linear regression analysis of key clinical parameters

    斜率截距Pearson相关系数
    LCSA0.98250.23590.9427
    VCSA1.1259−1.39110.9626
    PCSA1.2016−1.50440.9432
    下载: 导出CSV

    表 7  关键临床参数Bland-Altman分析结果

    Table 7.  Results of Bland-Altman analysis of key clinical parameters

    均值均值偏移偏移程度/%离群值比例/%
    LCSA5.9898−0.1320−2.205.25
    VCSA15.8044−0.5628−3.566.59
    PCSA9.8146−0.4308−4.407.27
    下载: 导出CSV
  • [1]

    Schoenhagen P, Crowe T, Nicholls S, et al. IVUS Made Easy: An Introduction to Coronary Intravascular Ultrasound Imaging[M]. Great Britain: Informa Healthcare, 2008.

    [2]

    Sonka M, Zhang X M, Siebes M, et al. Segmentation of intravascular ultrasound images: a knowledge-based approach[J]. IEEE Trans Med Imaging, 1995, 14(4): 719−732. doi: 10.1109/42.476113

    [3]

    Meier D S, Cothren R M, Vince D G, et al. Automated morphometry of coronary arteries with digital image analysis of intravascular ultrasound[J]. Am Heart J, 1997, 133(6): 681−690. doi: 10.1016/S0002-8703(97)70170-4

    [4]

    Essa E, Xie X H, Sazonov I, et al. Automatic IVUS media-adventitia border extraction using double interface graph cut segmentation[C]//18th IEEE International Conference on Image Processing, Brussels, 2011: 69−72. https://doi.org/10.1109/ICIP.2011.6116649.

    [5]

    Kovalski G, Beyar R, Shofti R, et al. Three-dimensional automatic quantitative analysis of intravascular ultrasound images[J]. Ultrasound Med Biol, 2000, 26(4): 527−537. doi: 10.1016/S0301-5629(99)00167-2

    [6]

    Hernandez A H, Gil D G, Radeva P R, et al. Anisotropic processing of image structures for adventitia detection in intravascular ultrasound images[C]//Computers in Cardiology, 2004, Chicago, 2004: 229−232. https://doi.org/10.1109/CIC.2004.1442914.

    [7]

    Giannoglou G D, Chatzizisis Y S, Koutkias V, et al. A novel active contour model for fully automated segmentation of intravascular ultrasound images: in vivo validation in human coronary arteries[J]. Comput Biol Med, 2007, 37(9): 1292−1302. doi: 10.1016/j.compbiomed.2006.12.003

    [8]

    Mendizabal-Ruiz E G, Kakadiaris I A. Probabilistic segmentation of the lumen from intravascular ultrasound radio frequency data[C]//Proceedings of the 15th International Conference on Medical Image Computing and Computer-Assisted Intervention, Nice, 2012: 454–461. https://doi.org/10.1007/978-3-642-33418-4_56.

    [9]

    Gil D, Radeva P, Saludes J, et al. Automatic segmentation of artery wall in coronary IVUS images: a probabilistic approach[C]//Computers in Cardiology 2000, Cambridge, 2000: 687–690. https://doi.org/10.1109/CIC.2000.898617.

    [10]

    Gil D, Radeva P, Saludes J. Segmentation of artery wall in coronary IVUS images: a probabilistic approach[C]// Proceedings of the 15th International Conference on Pattern Recognition, Barcelona, 2000: 352–355. https://doi.org/10.1109/ICPR.2000.902931.

    [11]

    Haas C, Ermert H, Holt S, et al. Segmentation of 3D intravascular ultrasonic images based on a random field model[J]. Ultrasound Med Biol, 2000, 26(2): 297−306. doi: 10.1016/S0301-5629(99)00139-8

    [12]

    Yang J, Tong L, Faraji M, et al. IVUS-Net: an intravascular ultrasound segmentation network[C]//Proceedings of the 1st International Conference on Smart Multimedia, Toulon, 2018: 367–377. https://doi.org/10.1007/978-3-030-04375-9_31.

    [13]

    Yang J, Faraji M, Basu A. Robust segmentation of arterial walls in intravascular ultrasound images using Dual Path U-Net[J]. Ultrasonics, 2019, 96: 24−33. doi: 10.1016/j.ultras.2019.03.014

    [14]

    Zhu F B, Gao Z Y, Zhao C, et al. A deep learning-based method to extract lumen and media-adventitia in intravascular ultrasound images[J]. Ultrason Imaging, 2022, 44(5−6): 191−203. doi: 10.1177/01617346221114137

    [15]

    Kim S, Jang Y, Jeon B, et al. Fully automatic segmentation of coronary arteries based on deep neural network in intravascular ultrasound images[C]//Proceedings of the 7th Joint International Workshop, CVII-STENT 2018 and Third International Workshop, LABELS 2018, Held in Conjunction with MICCAI 2018, Granada, 2018: 161−168. https://doi.org/10.1007/978-3-030-01364-6_18.

    [16]

    Xia M H, Yan W J, Huang Y, et al. Extracting membrane borders in IVUS images using a multi-scale feature aggregated U-Net[C]//2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Montreal, 2020: 1650−1653. https://doi.org/10.1109/EMBC44109.2020.9175970.

    [17]

    Bajaj R, Huang X R, Kilic Y, et al. Advanced deep learning methodology for accurate, real-time segmentation of high-resolution intravascular ultrasound images[J]. Int J Cardiol, 2021, 339: 185−191. doi: 10.1016/j.ijcard.2021.06.030

    [18]

    Blanco P J, Ziemer P G P, Bulant C A, et al. Fully automated lumen and vessel contour segmentation in intravascular ultrasound datasets[J]. Med Image Anal, 2022, 75: 102262. doi: 10.1016/j.media.2021.102262

    [19]

    Bargsten L, Raschka S, Schlaefer A. Capsule networks for segmentation of small intravascular ultrasound image datasets[J]. Int J Comput Assist Radiol Surg, 2021, 16(8): 1243−1254. doi: 10.1007/s11548-021-02417-x

    [20]

    Nishi T, Yamashita R, Imura S, et al. Deep learning-based intravascular ultrasound segmentation for the assessment of coronary artery disease[J]. Int J Cardiol, 2021, 333: 55−59. doi: 10.1016/j.ijcard.2021.03.020

    [21]

    Sinha P, Wu Y M, Psaromiligkos I, et al. Lumen & media segmentation of IVUS images via ellipse fitting using a wavelet-decomposed subband CNN[C]//2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP), Espoo, 2020: 1−6. https://doi.org/10.1109/MLSP49062.2020.9231871.

    [22]

    Xie E Z, Sun P Z, Song X G, et al. PolarMask: single shot instance segmentation with polar representation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2020: 12190−12199. https://doi.org/10.1109/CVPR42600.2020.01221.

    [23]

    Schmidt U, Weigert M, Broaddus C, et al. Cell detection with star-convex polygons[C]// Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, 2018: 265−273. https://doi.org/10.1007/978-3-030-00934-2_30.

    [24]

    Zhang Z L, Zhang X Y, Peng C, et al. ExFuse: enhancing feature fusion for semantic segmentation[C]//Proceedings of the 15th European Conference on Computer Vision, Munich, 2018: 273–288. https://doi.org/10.1007/978-3-030-01249-6_17.

    [25]

    He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016: 770–778. https://doi.org/10.1109/CVPR.2016.90.

    [26]

    Wang H H, Wu X D, Huang Z Y, et al. High-frequency component helps explain the generalization of convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, 2020: 8681–8691. https://doi.org/10.1109/CVPR42600.2020.00871.

    [27]

    Balocco S, Gatta C, Ciompi F, et al. Standardized evaluation methodology and reference database for evaluating IVUS image segmentation[J]. Comput Med Imaging Graph, 2014, 38(2): 70−90. doi: 10.1016/j.compmedimag.2013.07.001

    [28]

    Wyburd M K, Dinsdale N K, Namburete A I L, et al. TEDS-Net: enforcing diffeomorphisms in spatial transformers to guarantee topology preservation in segmentations[C]//Proceedings of the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention, Strasbourg, 2021: 250–260. https://doi.org/10.1007/978-3-030-87193-2_24.

    [29]

    Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 15th European Conference on Computer Vision, Munich, 2018: 833‒851. https://doi.org/10.1007/978-3-030-01234-2_49.

    [30]

    Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39(12): 2481−2495. doi: 10.1109/TPAMI.2016.2644615

    [31]

    Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]. Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, 2015: 234‒241. https://doi.org/10.1007/978-3-319-24574-4_28.

  • 加载中

(9)

(7)

计量
  • 文章访问数: 
  • PDF下载数: 
  • 施引文献:  0
出版历程
收稿日期:  2022-06-08
修回日期:  2022-09-07
录用日期:  2022-09-16
网络出版日期:  2022-12-22
刊出日期:  2023-01-25

目录

/

返回文章
返回