E-mail Alert
RSS
-
摘要:
针对传统云图检索方法难于获得理想的检索精度且检索效率低的问题，提出了一种基于深度度量学习的云图检索方法。首先设计了残差3D-2D卷积神经网络，以提取云图的空间及光谱特征。鉴于传统基于分类的深度网络所提取的特征可能存在类内差异大、类间差异小的问题，采用三元组训练网络，依据云图之间的相似性将云图映射到度量空间中，以使同类云图在嵌入空间中的距离小于非同类云图。在模型训练时，通过对无损三元组损失函数增加正样本对间距离的约束，改善了传统三元组损失的收敛性能，提高了云图检索的精度。在此基础上，通过哈希学习，将度量空间中的云图特征变换成哈希码，在保证检索精度的条件下提高了检索效率。实验结果表明，在东南沿海云图数据集和北半球区域云图数据集上，本文算法的平均精度均值(mean average precision, mAP)分别达到75.14%和80.14%，优于其他对比方法。
Abstract:Due to the traditional cloud image retrieval methods are difficult to obtain ideal retrieval accuracy and retrieval efficiency, a cloud image retrieval method based on deep metric learning is proposed. Firstly, a residual 3D-2D convolutional neural network is designed to extract spatial and spectral features of cloud images. Since the features extracted by the traditional classify-based deep network may have greater differences intra-classes than inter-classes, the triplet strategy is used to train the network, and the cloud images are mapped into the metric space according to the similarity between cloud images, so that the distance of similar cloud images in the embedded space is smaller than that of non-similar cloud images. In model training, the convergence performance of traditional triplet loss is improved and the precision of cloud image retrieval is increased by adding a constraint on the distance between positive sample pairs to the lossless triplet loss function. Finally, through hash learning, the cloud features in the metric space are transformed into hash codes, so as to ensure the retrieval accuracy and improve the retrieval efficiency. Experimental results show that the mean average precision (mAP) of the proposed algorithm is 75.14% and 80.14% for the southeast coastal cloud image dataset and the northern hemisphere cloud image dataset respectively, which is superior to other comparison methods.
-
Key words:
- deep learning /
- metric learning /
- triplet loss /
- satellite cloud image retrieval
-
Overview: Meteorological satellites can monitor weather phenomena of different scales from the air, and the satellite cloud images obtained by them play an important role in weather analysis and forecast. In recent years, with the development of meteorological satellite technology, the spatial and spectral resolution of satellite cloud images and the acquisition frequency of imaging spectrometer have been continuously improved. How to manage massive satellite cloud images and design an efficient cloud image retrieval system has become a difficult problem for meteorologists. However, the traditional cloud image retrieval methods are difficult to obtain ideal retrieval accuracy and retrieval efficiency. Motivated by the impressive success of the modern deep neural network (DNN) in learning the optimization features of specific tasks in an end-to-end fashion, a cloud image retrieval method based on deep metric learning is proposed in this paper. Firstly, a residual 3D-2D convolutional neural network was designed to extract spatial and spectral features of cloud images. Since the features extracted by the traditional classify-based deep network may have greater differences intra-classes than inter-classes, the triplet strategy is used to train the network, and the cloud images are mapped into the metric space according to the similarity between cloud images, so that the distance of similar cloud images in the embedded space is smaller than that of non-similar cloud images. In model training, the convergence performance of traditional triplet loss is improved and the precision of cloud image retrieval is increased by adding a constraint on the distance between positive sample pairs to the lossless triplet loss function. Finally, through hash learning, the cloud features in the metric space are transformed into hash codes, so as to ensure the retrieval accuracy and improve the retrieval efficiency. Experimental results show that the mean average precision (mAP) of the proposed algorithm is 75.14% and 80.14% for the southeast coastal cloud image dataset and the northern hemisphere cloud image dataset respectively, which is superior to other comparison methods.
-
-
图 2 不同天气系统可见光波段1的云图。
Figure 2. Visible band 1 cloud images of different weather systems.
图 5 训练后正样本对间距离缩小，负样本对间距离扩大
Figure 5. After training, the distance of the anchor-positive decreases and the distance of the anchor-negative increases
表 1 沿海云图数据集中不同损失函数的模型检索性能
Table 1. The model retrieval performance of different loss functions in coastal cloud image dataset
Loss P@5/% P@20/% mAP/% TL 83.61 72.53 64.21 LTL 85.95 76.02 71.64 C-LTL 90.96 78.14 75.14 
下载: 导出CSV
表 2 北半球云图数据集中不同损失函数的模型检索性能
Table 2. The model retrieval performance of different loss functions in the North hemisphere cloud image dataset
Loss P@5/% P@20/% mAP/% TL 80.57 79.47 71.23 LTL 87.70 82.55 73.07 C-LTL 85.20 85.63 80.14
下载: 导出CSV
表 3 各数据集在不同方法下的检索准确度
Table 3. Comparison of retrieval performance between different retrieval methods
Dataset Methods mAP/% P@5/% P@10/% P@20/% P@30/% 沿海云图数据集 KSH 60.10 73.62 71.73 67.76 66.30 DLBHS 72.44 76.19 75.28 74.35 72.15 MiLan 68.09 73.38 72.54 71.37 70.92 DSH 68.61 85.76 81.17 76.89 73.24 Proposed 75.14 90.96 82.41 78.14 76.68 北半球云图数据集 KSH 60.27 68.53 68.43 67.41 66.92 DLBHS 78.13 84.65 83.70 83.00 82.71 MiLan 74.97 81.33 80.64 80.16 79.85 DSH 70.22 69.86 72.80 73.40 73.90 Proposed 80.14 85.20 85.84 85.63 84.90
下载: 导出CSV
-
[1] 徐坚. 基于内容的卫星云图检索系统设计与实现[D]. 南京: 河海大学, 2004.
Xu J. Contented based cloudy satellite images retrieval system design and practice[D]. Nanjing: Hohai University, 2004.
[2] 上官伟. 基于内容的卫星云图处理与信息检索技术研究[D]. 哈尔滨: 哈尔滨工程大学, 2008.
Shangguan W. Research of content-based satellite cloud image processing and information retrieval technology[D]. Harbin: Harbin Engineering University, 2008.
[3] 李秀馨. 基于内容的卫星云图检索技术研究[D]. 南京: 南京航空航天大学, 2013.
Li X X. Research on the technology of content-based satellite cloud image retrieval[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2013.
[4] 徐晔烨. 基于多特征的红外云图检索技术研究[D]. 南京: 南京航空航天大学, 2014.
Xu Y Y. Infrared cloud image retrieval technology research based on multi-feature[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2014.
[5] Zhang C J, Li G D, Lei R M, et al. Deep feature aggregation network for hyperspectral remote sensing image classification[J]. IEEE J Select Top Appl Earth Observ Remote Sens, 2020, 13: 5314−5325. doi: 10.1109/JSTARS.2020.3020733
[6] Fang J S, Fu H Z, Liu J. Deep triplet hashing network for case-based medical image retrieval[J]. Med Image Anal, 2021, 69: 101981. doi: 10.1016/j.media.2021.101981
[7] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 2015 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015: 234–241.
[8] Roy S K, Krishna G, Dubey S R, et al. HybridSN: exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geosci Remote Sens Lett, 2020, 17(2): 277−281. doi: 10.1109/LGRS.2019.2918719
[9] Feng F, Wang S T, Wang C Y, et al. Learning deep hierarchical spatial–spectral features for hyperspectral image classification based on residual 3D-2D CNN[J]. Sensors, 2019, 19(23): 5276. doi: 10.3390/s19235276
[10] Hermans A, Beyer L, Leibe B. In defense of the triplet loss for person re-identification[Z]. arXiv: 1703.07737, 2017. https://doi.org/10.48550/arXiv.1703.07737
[11] Kaya M, Bilge H Ş. Deep metric learning: A survey[J]. Symmetry, 2019, 11(9): 1066. doi: 10.3390/sym11091066
[12] Schroff F, Kalenichenko D, Philbin J. FaceNet: a unified embedding for face recognition and clustering[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 815–823.
[13] Bessho K, Date K, Hayashi M, et al. An introduction to Himawari-8/9—Japan’s new-generation geostationary meteorological satellites[J]. J Meteor Soc Jpn Ser II, 2016, 94(2): 151−183. doi: 10.2151/jmsj.2016-009
[14] 郁凡, 陈渭民. 双光谱云图的云分类探讨[J]. 南京气象学院学报, 1994, 17(1): 117−124.
Yu F, Chen W M. Research on the cloud classification for the bi-spectrum cloud picture[J]. J Nanjing Inst Meteor, 1994, 17(1): 117−124.
[15] Bai C, Zhang M J, Zhang J L, et al. LSCIDMR: large-scale satellite cloud image database for meteorological research[J]. IEEE Trans Cybern, 2021. doi: 10.1109/TCYB.2021.3080121
[16] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770–778.
[17] Huang K K, Ren C X, Liu H, et al. Hyperspectral image classification via discriminative convolutional neural network with an improved triplet loss[J]. Pattern Recogn, 2021, 112: 107744. doi: 10.1016/j.patcog.2020.107744
[18] Roy S, Sangineto E, Demir B, et al. Metric-learning-based deep hashing network for content-based retrieval of remote sensing images[J]. IEEE Geosci Remote Sens Lett, 2021, 18(2): 226−230. doi: 10.1109/LGRS.2020.2974629
[19] Liu H M, Wang R P, Shan S G, et al. Deep supervised hashing for fast image retrieval[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2064–2072.
[20] Kim S, Seo M, Laptev I, et al. Deep metric learning beyond binary supervision[C]//Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 2283–2292.
[21] Arsenault Marc-Olivier. Lossless Triplet loss[EB/OL]. (2018-02-15). http://coffeeanddata.ca/lossless-triplet-loss.
[22] Cheng D, Gong Y H, Zhou S P, et al. Person Re-identification by multi-channel parts-based CNN with improved triplet loss function[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 1335–1344.
[23] Xuan H, Stylianou A, Pless R. Improved embeddings with easy positive triplet mining[C]//Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, 2020: 2463–2471.
[24] 廖列法, 李志明, 张赛赛. 基于深度残差网络的迭代量化哈希图像检索方法[J]. 计算机应用, 2021. doi: 10.11772/j.issn.1001-9081.2021071135
Liao L F, Li Z M, Zhang S S. Image retrieval method based on deep residual network and iterative quantization hashing[J]. J Comput Appl, 2021. doi: 10.11772/j.issn.1001-9081.2021071135
[25] Liu W, Wang J, Ji R R, et al. Supervised hashing with kernels[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012: 2074–2081.
[26] Lin K, Yang H F, Hsiao J H, et al. Deep learning of binary hash codes for fast image retrieval[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2015: 27–35.
[27] Torralba A, Murphy K P, Freeman W T, et al. Context-based vision system for place and object recognition[C]//Proceedings of the Ninth IEEE International Conference on Computer Vision, 2003: 273–280.
-
点击扫一扫
图(10)
表(3)
计量
- 文章访问数: 7484
- PDF下载数: 686
- 施引文献: 0
