E-mail Alert
RSS
-
摘要
针对现有彩色图像检索算法存在旋转变化鲁棒性差、特征维度高和检索时间长的问题，通过融合主曲率的改进方向梯度特征与HSV颜色特征，提出了一种创新的多尺度图像检索方法。该方法从多个尺度将图像表面的几何曲率信息融合到FHOG描述符中，得到基于主曲率的改进方向梯度算法(P-FHOG)，在此基础上进一步融合图像的颜色信息，得到基于颜色特征与改进方向梯度特征的多尺度图像检索方法(CP-FHOG)。在Corel-1000与Coil-100数据集上与先进的图像检索方法进行对比实验，分别取得了85.89%和93.38%的平均准确率，该算法相比其他算法准确率更高、旋转变化鲁棒性更强、检索时间更短，提高了检索效率。
Abstract
Aiming at the problems of poor robustness of rotation change, high feature dimension, and long retrieval time of existing color image retrieval algorithms, this paper proposed an innovative image retrieval method by fusing color features and improved directional gradient features. It proposed an improved directional gradient algorithm based on the principal curvatures (P-FHOG) by combining the geometric curvature information of the image surface into the FHOG descriptor from multiple scales. At the same time, the color information of the image was further fused to obtain the multi-scale image retrieval method based on the color features and the improved directional gradient features (CP-FHOG). The experiment was compared with the advanced image retrieval methods on the Corel-1000 and Coil-100 data sets, and the average accuracy rates of 85.89% and 93.38% were achieved, respectively. The results show that the proposed algorithm is more accurate and robust (in rotation change) than other algorithms.
-
Key words:
- image retrieval /
- color information /
- directional gradient /
- multiple scales /
- features fusion
-
Overview
Overview: With the rapid development of computer vision and digital media, image retrieval has been successfully applied to search engines, digital libraries, medical image management, and other fields. For current color image retrieval, the extraction of a single image feature is often too limited, and it is difficult to achieve the purpose of efficient and fast retrieval. Color feature and directional gradient feature are two important features of an image, which are widely used in the field of image retrieval. Color information represents the overall features of the image, and the directional gradient feature represents the partial features information of the image by extracting the texture information of the image. Aiming at the problems of poor rotation change robustness, high feature dimension, and long retrieval time in current retrieval methods, a color image retrieval method that combines color feature with improved directional gradient feature is proposed. First, the input color image is converted into a grayscale image through Gaussian space, and the surface geometric curvature information and texture information of the grayscale image are extracted and integrated into the FHOG descriptor, and the main curvature information is multi-sampled to construct a mixed sampling direction gradient feature (P-FHOG1, P-FHOG2, P-FHOG3) based on the main curvature, and the improved directional gradient feature (P-FHOGs) based on the main curvature is obtained by merging the features of three scales. At the same time, the image is converted from RGB color space to HSV color space and the color information of the image is extracted after quantization to construct the color feature histogram, and the color feature of the image is obtained. On this basis, the two features are merged to obtain an image retrieval method based on color feature and improved direction gradient feature (CP-FHOG). The experiment was compared with the advanced image retrieval methods on the Corel-1000 and Coil-100 data sets, and the average accuracy rates of 85.89% and 93.38% were achieved, respectively. On the Corel-1000 data set, the features extraction time and retrieval time of the algorithm in this paper are 0.067 s and 0.048 s, respectively, which are improved by 0.075 s and 1.06 s, respectively, compared with the second-performing algorithm. At the same time, ablation experiments were performed in the two data sets to verify the effectiveness of the fusion algorithm. The experimental results show that, compared with HSV and P-FHOGs algorithms, CP-FHOG extracts richer detailed features, has stronger rotation robustness, and significantly improves retrieval accuracy in datasets containing complex backgrounds and targets with different rotation angles. Besides, retrieval time and feature dimension have also been greatly improved. The color image retrieval method proposed in this paper introduces main curvature information and color information based on FHOG descriptors, combines the advantages of color feature and directional gradient feature, and extracts rich overall and detailed features. The experimental result proves that the retrieval accuracy of the method in this paper is higher and the method has rotation robustness.
-
-
图 5 颜色特征提取。(a) 输入图像；(b) RGB转换图像；(c) HSV转换图像
Figure 5. Extraction of the color features. (a) Input images; (b) RGB converted images; (c) HSV converted images
图 11 Corel-1000数据集的检索结果。(a) Africans；(b) Flowers
Figure 11. The retrieval results of the Corel-1000 dataset. (a) Africans; (b) Flowers
表 1 实验参数设置
Table 1. Experimental parameter setting
参数 δ m b CP-FHOG (0.2, 0.5, 1) (8, 16) 30 
下载: 导出CSV
表 2 数据集Corel-1000上的各类别检索准确率/%
Table 2. Retrieval accuracy of each category on the Corel-1000 dataset/%
Category Pavithra[9] Kundu[22] Dubey[23] Sonug[25] Xiao[26] HSV P-FHOGs CP-FHOG African 81.0 44.0 75.0 67.6 67.0 93.4 62.5 98.6 Sea 66.0 32.0 55.0 59.8 60.0 55.5 77.4 69.7 Architecture 78.8 52.0 67.0 58.0 56.0 58.7 64.8 66.7 Bus 96.3 62.0 95.0 94.0 96.0 91.5 99.0 99.6 Dinosaur 100.0 40.0 97.0 99.8 98.0 99.7 100.0 100.0 Elephant 70.8 80.0 63.0 58.0 53.0 54.6 58.2 70.4 Flower 95.8 57.0 93.0 88.6 93.0 87.5 89.3 95.8 Horse 98.8 75.0 89.0 93.8 82.0 97.6 81.7 98.7 Mountain 67.8 57.0 45.0 47.8 46. 0 57.6 54.2 73.0 Food 77. 3 56.0 70.0 49.2 58. 0 79.3 65.1 86.5
下载: 导出CSV
表 3 数据集Corel-1000上的各类别检索召回率/%
Table 3. Retrieval recall rate of each category on Corel-1000 dataset/%
Category Pavithra[9] Kundu[22] Dubey[23] Sonug[25] Xiao[26] HSV P-FHOGs CP-FHOG African 16.2 8.8 15.0 13.5 13.4 18.6 12.5 19.7 Sea 13.2 6.4 11.0 12.0 12.0 11.1 15.4 13.9 Architecture 15.8 10.4 13.4 11.6 11.2 11.7 12.9 13.3 Bus 19.3 12.4 19.0 18.8 19.2 18.3 19.0 19.9 Dinosaur 20.0 8.0 19.4 20.0 98.0 19.9 20.0 20.0 Elephant 14.2 16.0 12.6 11.6 10.6 10.9 11.6 14.1 Flower 19.2 11.4 18.6 17.7 18.6 17.5 17.8 19.2 Horse 19.8 15.0 17.8 18.8 16.4 19.5 16.3 19.7 Mountain 13.6 11.4 9.0 9.6 9.2 11.5 10.8 14.6 Food 15.5 11.2 14.0 9.8 11.6 15.8 13.0 17.3
下载: 导出CSV
表 4 数据集Corel-1000上的各参数对比
Table 4. Comparison of parameters on the Corel-1000 dataset
Category AlexNet[24] GoogleNet VGG-19 ResNet-50 CP-FHOG African 33.0 65.0 68.0 78.0 98.6 Sea 22.0 75.0 79.0 77.0 69.7 Architecture 40.0 90.0 90.0 99.0 66.7 Bus 23.3 87.0 88.0 90.0 99.6 Dinosaur 71.0 88.0 90.0 88.0 100.0 Elephant 27.5 80.0 85.0 87.0 70.4 Flower 50.0 91.0 93.0 95.0 95.8 Horse 59.2 83.0 88.0 93.0 98.7 Mountain 26.7 80.0 90.0 98.0 73.0 Food 65.0 80.0 81.0 85.0 86.5
下载: 导出CSV
表 5 数据集Corel-1000上与深度学习算法对比各类别检索准确率/%
Table 5. Retrieval accuracy of each category compared with the deep learning algorithm on the Corel-1000 dataset/%
Algorithm mAP/% Recall/% SFET/s RT/s Dimension Pavithra[9] 83.26 16.65 0.671 1.108 768 Kundu[22] 55.50 11.10 0.400 - 99 Sun[24] 83.50 16.70 9.150 1.027 900 Dubey[23] 74.90 14.98 102.400 16.490 1024 Sonug[25] 71.66 14.33 - - 4096 Xiao[26] 70.10 14.02 - - 63 HSV 77.54 14.18 0.020 0.023 72 P-FHOGs 75.22 14.02 0.053 0.021 270 CP-FHOG 85.89 17.18 0.067 0.048 342
下载: 导出CSV
表 6 数据集 Coil-100 上的各类别检索准确率/%
Table 6. Retrieval accuracy of each category in the COIL-100 dataset/%
Category CP-FHOG HSV P-FHOGs Ahmed[27] SIFT SURF MSER LBP RGBLBP Tomato 98.7 93.5 89.3 93.0 15.0 75.0 15.0 35.0 20.0 Cat 100.0 100.0 86.3 90.0 32.0 45.0 55.0 40.0 25.0 Statue 100.0 100.0 63.2 100.0 35.0 30.0 45.0 25.0 55.0 Stick 60.9 52.8 25.8 93.0 30.0 35.0 90.0 50.0 10.0 Rolaids 100.0 100.0 95.3 65.0 20.0 60.0 40.0 65.0 85.0 Mud pot 100.0 100.0 99.8 100.0 20.0 45.0 90.0 70.0 50.0 Frog 99.0 91.2 60.8 95.0 20.0 65.0 45.0 55.0 45.0 Jug 98.8 98.2 57.3 100.0 20.0 45.0 70.0 65.0 60.0 Car 93.3 98.7 16.9 98.0 22.0 65.0 22.0 60.0 55.0 Pink cup 100.0 100.0 70.1 88.0 40.0 50.0 35.0 60.0 50.0 White cup 100.0 100.0 96.8 94.0 45.0 40.0 60.0 25.0 50.0 Truck 69.9 52.1 30.8 90.0 15.0 35.0 35.0 30.0 60.0
下载: 导出CSV
-
参考文献
[1] Yan C G, Gong B, Wei Y X, et al. Deep multi-view enhancement hashing for image retrieval[J]. IEEE Trans Pattern Mach Intell, 2021, 43(4): 1445–1451. doi: 10.1109/TPAMI.2020.2975798
[2] 寇旗旗, 程德强, 于文洁, 等. 融合CLBP和局部几何特征的纹理目标分类[J]. 光电工程, 2019, 46(11): 180604. doi: 10.12086/oee.2019.180604
Kou Q Q, Cheng D Q, Yu W J, et al. Texture target classification with CLBP and local geometric features[J]. Opto-Electron Eng, 2019, 46(11): 180604. doi: 10.12086/oee.2019.180604
[3] 刘芳, 吴志威, 杨安喆, 等. 基于多尺度特征融合的自适应无人机目标检测[J]. 光学学报, 2020, 40(10): 1015002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202010016.htm
Liu F, Wu Z W, Yang A Z, et al. Multi-scale feature fusion based adaptive object detection for UAV[J]. Acta Opt Sin, 2020, 40(10): 1015002. https://www.cnki.com.cn/Article/CJFDTOTAL-GXXB202010016.htm
[4] Celik C, Bilge H S. Content based image retrieval with sparse representations and local feature descriptors: a comparative study[J]. Pattern Recognit, 2017, 68: 1–13. doi: 10.1016/j.patcog.2017.03.006
[5] Agarwal M, Maheshwari R P. HOG feature and vocabulary tree for content-based image retrieval[J]. Int J Signal Imaging Syst Eng, 2011, 3(4): 246–254.
[6] Hu R, Barnard M, Collomosse J. Gradient field descriptor for sketch based retrieval and localization[C]//Proceedings of 2010 IEEE International Conference on Image Processing, Hong Kong, China, 2010: 1025–1028.
[7] Joolee J B, Lee Y K. Video retrieval based on image queries using THOG for augmented reality environments[C]//Proceedings of 2018 IEEE International Conference on Big Data and Smart Computing, Shanghai, China, 2018: 557–560.
[8] 程德强, 张皓翔, 江曼, 等. 融合主曲率与颜色信息的彩色图像检索算法[J]. 计算机辅助设计与图形学学报, 2021, 33(2): 223–231. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJF202102008.htm
Cheng D Q, Zhang H X, Jiang M, et al. Color image retrieval method fusing principal curvature and color information[J]. J Comput Aided Des Comput Graph, 2021, 33(2): 223–231. https://www.cnki.com.cn/Article/CJFDTOTAL-JSJF202102008.htm
[9] Pavithra L K, Sharmila T S. An efficient framework for image retrieval using color, texture and edge features[J]. Comput Elect Eng, 2018, 70: 580–593. doi: 10.1016/j.compeleceng.2017.08.030
[10] Bella M I T, Vasuki A. An efficient image retrieval framework using fused information feature[J]. Comput Elect Eng, 2019, 75: 46–60. doi: 10.1016/j.compeleceng.2019.01.022
[11] Garg M, Dhiman G. A novel content-based image retrieval approach for classification using GLCM features and texture fused LBP variants[J]. Neural Comput Appl, 2020, 33(4): 1311–1328. doi: 10.1007/s00521-020-05017-z
[12] Danapur N, Dizaj S A A, Rostami V. An efficient image retrieval based on an integration of HSV, RLBP, and CENTRIST features using ensemble classifier learning[J]. Multimed Tools Appl, 2020, 79(33): 24463–24486. doi: 10.1007/s11042-020-09109-9
[13] Khwildi R, Ouled Zaid A. HDR image retrieval by using color-based descriptor and tone mapping operator[J]. Vis Comput, 2020, 36(8): 1111–1126. doi: 10.1007/s00371-019-01719-1
[14] Farid H, Simoncelli E P. Differentiation of discrete multidimensional signals[J]. IEEE Trans Image Process, 2004, 13(4): 496–508. doi: 10.1109/TIP.2004.823819
[15] Felzenszwalb P F, Girshick R B, McAllester D, et al. Object detection with discriminatively trained part-based models[J]. IEEE Trans Pattern Anal Mach Intell, 2010, 32(9): 1627–1645. doi: 10.1109/TPAMI.2009.167
[16] Kou Q Q, Cheng D Q, Zhuang H D, et al. Cross-complementary local binary pattern for robust texture classification[J]. IEEE Signal Process Lett, 2018, 26(1): 129–133. http://ieeexplore.ieee.org/document/8537935
[17] Zhang H X, Jiang M, Kou Q Q. Color image retrieval algorithm fusing color and principal curvatures information[J]. IEEE Access, 2020, 8: 184945–184954. doi: 10.1109/ACCESS.2020.3030056
[18] Wang J Z, Li J, Wiederhold G. SIMPLIcity: semantics-sensitive integrated matching for picture libraries[J]. IEEE Trans Pattern Anal Mach Intell, 2001, 23(9): 947–963. doi: 10.1109/34.955109
[19] Nene S A, Nayar S K, Murase H. Columbia object image library (COIL-100)[R]. New York: Columbia University, 1996.
[20] Kavitha H, Sudhamani M V. Object Based Image Retrieval from Database Using Combined Features[C]//Proceedings of the 2014 Fifth International Conference on Signal and Image Processing. IEEE, Bangalore, INDIA, 2014: 161–165.
[21] 吕晨, 程德强, 寇旗旗, 等. 基于YOLOv3和ASMS的目标跟踪算法[J]. 光电工程, 2021, 48(2): 200175. doi: 10.12086/oee.2021.200175
Lv C, Cheng D Q, Kou Q Q, et al. Target tracking algorithm based on YOLOv3 and ASMS[J]. Opto-Electron Eng, 2021, 48(2): 200175. doi: 10.12086/oee.2021.200175
[22] Kundu M K, Chowdhury M, Bulo S R. A graph-based relevance feedback mechanism in content-based image retrieval[J]. Knowl-Based Syst, 2015, 73: 254–264. doi: 10.1016/j.knosys.2014.10.009
[23] Dubey S R, Singh S K, Singh R K. Multichannel decoded local binary patterns for content-based image retrieval[J]. IEEE Trans Image Process, 2016, 25(9): 4018–4032. doi: 10.1109/TIP.2016.2577887
[24] 孙奇平. 基于深度学习的图像检索研究[J]. 景德镇学院学报, 2018, 33(3): 15–18. doi: 10.3969/j.issn.1008-8458.2018.03.009
Sun Q P. Research on image retrieval based on deep learning[J]. Jingdezhen Compr Coll J, 2018, 33(3): 15–18. doi: 10.3969/j.issn.1008-8458.2018.03.009
[25] Somnugpong S, Khiewwan K. Content-based image retrieval using a combination of color correlograms and edge direction histogram[C]//Proceedings of the 2016 13th International Joint Conference on Computer Science and Software Engineering, Khon Kaen, Thailand, 2016: 1–5.
[26] Xiao Y, Wu J X, Yuan J S. mCENTRIST: a multi-channel feature generation mechanism for scene categorization[J]. IEEE Trans Image Process, 2014, 23(2): 823–836. doi: 10.1109/TIP.2013.2295756
[27] Ahmed K T, Ummesafi S, Iqbal A. Content based image retrieval using image features information fusion[J]. Inf Fusion, 2019, 51: 76–99. doi: 10.1016/j.inffus.2018.11.004
-
访问统计
点击扫一扫
图(13)
表(6)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
