E-mail Alert
RSS
The optical flow detection method of moving target using deep convolution neural network
-
摘要
运动目标检测是物体检测领域的一个重要研究方向，在目标识别中有着至关重要的作用。针对传统运动检测方法精度不高、无法对运动目标进行检测，本文将深度卷积神经网络引入到运动目标光流检测中，将前后帧图像及目标光流场图像作为网络的输入，自适应地学习运动目标光流，并通过对网络放大架构的优化及网络的精简，同时采用数据增广等技术，设计出精度与实时性兼顾的目标物体光流检测网络。实验结果表明，本文方法在运动目标的光流场检测中有更好的表现，SS-sp和CS-sp网络相比原网络在检测精度上均提高了约5.0%，同时大幅减少了网络的运行时间，基本满足实时检测的要求。
Abstract
Moving target detection is an important research direction of object detection, and it plays an important role in target recognition. The accuracy of traditional motion detection methods is low, which are unable to only detect the required moving target. In this study, deep convolutional neural network is introduced into the optical flow detection of moving target. In this method, a pair of images and optical flow fields of target are used as inputs of the network to adaptively study the target optical flow. Furthermore, through optimization of the expanding part of the network and the simplification of the network, and combined with many data augmentation technologies, the optical flow detection network of target object with both accuracy and real-time is designed. Experimental results show that the proposed method has better performance in the optical flow detection of moving target. SS-sp and CS-sp network are improved by about 5.0% compared to the original network on the precision and the runtime of the network is significantly reduced, which meet the requirements of real-time detection.
-
Overview
Overview: Object detection is a hot research direction in the fields of pattern recognition and machine learning. It is used widely in areas such as security, traffic and the Internet. Computer realizes object detection automatically, which can reduce the burden of people in a certain extent. Moving target detection is an important research direction of object detection and plays an important role in target recognition. With the development of deep learning theory, the deep convolutional neural network has made remarkable achievements in the field of object detection by improving the generation strategy of candidate regions and by optimizing the network structure and training methods. Different from the usual definition, moving target detection is turned into focus on the detection of moving target that is needed rather than all of the moving targets. However, conventional convolution neural network cannot extract the dynamic characteristics of object and the accuracy of traditional motion detection methods is low and unable to only detect the required moving target. In this study, deep convolutional neural network is introduced into the optical flow detection of moving target. In this method, a pair of images and optical flow field of target are used as inputs of the network to adaptively study the target optical flow. Furthermore, through optimization of the expanding part of the network and the simplification of the network, and combined with many data augmentation technologies, the optical flow detection network of target object with both accuracy and real time is designed. Visualization technology is also applied to the network, which can directly compare the results with visual image. Experimental results show that the proposed method has better performance in the optical flow detection of moving target. The proposed network in this study retains the advantage of the deep convolutional neural network that only detect the optical flow of required target, which conforms to the practical application of object detection. Additionally, the proposed network achieves better performance in the both accuracy and the real-time of network. S-sp is improved by about 4.9% and 5.3%, respectively, on the AAE and AEE indicators compared to the original network, and SS-sp and CS-sp network are improved by about 5.0% on the AAE indicators compared to the original network. The runtime CS-sp and SS-sp network can be significantly reduced. The runtime of CS-sp network is reduced from about 179 ms to 34 ms and the runtime of the CS-sp network is reduced from 170 ms to 33 ms, which meet the requirements of real-time detection.
-
-
图 2 收缩架构。(a) FN-S收缩架构；(b) FN-C收缩架构
Figure 2. Contracting structure. (a) FN-S contracting structure; (b) FN-C contracting structure
图 5 部分实验结果1。(a)原图 1；(b)原图 2；(c)目标光流场图；(d) Horn-Schunck算法结果；(e) Lucas-Kanade算法结果；(f)本文CS-sp算法结果
Figure 5. Part of experimental results 1. (a) Original image 1; (b) Original image 2; (c) Optical flow image of target; (d) Result of Horn-Schunck algorithm; (e) Result of Lucas-Kanade algorithm; (f) Result of CS-sp algorithm
图 6 部分实验结果2。(a)原图 1；(b)原图 2；(c)目标光流场图；(d) Horn-Schunck算法结果；(e) Lucas-Kanade算法结果；(f)本文CS-sp算法结果
Figure 6. Part of experimental results 2. (a) Original image 1; (b) Original image 2; (c) Optical flow image of target; (d) Result of Horn-Schunck algorithm; (e) Result of Lucas-Kanade algorithm; (f) Result of CS-sp algorithm
表 1 优化前后放大部分网络比对
Table 1. Comparison between original expanding network and network after optimization
Network name Network configurations Deconv1 Conv10 Deconv2 Conv11 Deconv3 Conv12 Expand 512×5×5
str2- 256×5×5
str2- 128×5×5
str2- Expand-c 512×4×4
str2512×3×3
str1256×4×4
str2256×3×3
str1128×4×4
str2128×3×3
str1Network name Deconv4 Conv13 Deconv5 Conv14 Bilinear interpolation Expand 64×5×5
str2- - - √ Expand-c 64×4×4
str264×3×3
str132×4×4
str232×3×3
str1√ 
下载: 导出CSV
表 2 精简后放大部分网络
Table 2. Expanding network after simp
Network name Network configurations Deconv1 Conv10 Deconv2 Conv11 Deconv3 Conv12 Expand-sp 192×4×4
str2192×3×3
str196×4×4
str296×3×3
str148×4×4
str248×3×3
str1Network name Deconv4 Conv13 Deconv5 Conv14 Bilinear interpolation Expand-sp 24×4×4
str224×3×3
str112×4×4
str212×3×3
str1√
下载: 导出CSV
表 3 经典光流法、FlowNet以及本文网络的检测误差
Table 3. The detection errors of classical optical flow, FlowNet and our network
Network name AEE AAE Runtime/ms Horn-Schunck 9.7983 1.1958 1260 S 2.9283 0.2754 91 S-c 2.7173 0.2506 105 S-sp 2.7852 0.2609 18 C 2.8451 0.2619 95 C-c 2.7078 0.2527 108 C-sp 2.8375 0.2650 20 Lucas-Kanade 9.7813 1.1744 1220.4 SS 2.6482 0.2560 170 SS-c 2.4224 0.2343 190 SS-sp 2.5146 0.2427 33 CS 2.5596 0.2535 179 CS-c 2.3438 0.2287 195 CS-sp 2.4325 0.2376 34
下载: 导出CSV
-
参考文献
[1] 黄凯奇, 任伟强, 谭铁牛.图像物体分类与检测算法综述[J].计算机学报, 2014, 37(6): 1225-1240. http://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201406001.htm
Huang K Q, Ren W Q, Tan T N. A review on image object classification and detection[J]. Chinese Journal of Computers, 2014, 37(6): 1225-1240. http://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201406001.htm
[2] 卢宏涛, 张秦川.深度卷积神经网络在计算机视觉中的应用研究综述[J].数据采集与处理, 2016, 31(1): 1-17. http://news.hfut.edu.cn/print-28-46059-1.html
Lu H T, Zhang Q C. Applications of deep convolutional neural network in computer vision[J]. Journal of Data Acquisition and Processing, 2016, 31(1): 1-17. http://news.hfut.edu.cn/print-28-46059-1.html
[3] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1137. doi: 10.1109/TPAMI.2016.2577031
[4] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440.
[5] 王顺飞, 闫钧华, 王志刚.改进的基于局部联合特征的运动目标检测方法[J].仪器仪表学报, 2015, 36(10): 2241-2248. doi: 10.3969/j.issn.0254-3087.2015.10.011
Wang S F, Yan J H, Wang Z G. Improved moving object detection algorithm based on local united feature[J]. Chinese Journal of Scientific Instrument, 2015, 36(10): 2241-2248. doi: 10.3969/j.issn.0254-3087.2015.10.011
[6] 刘威, 赵文杰, 李成, 等.基于改进ORB特征匹配的运动小目标检测[J].光电工程, 2015, 42(10): 13-20. doi: 10.3969/j.issn.1003-501X.2015.10.003
Liu W, Zhao W J, Li C, et al. Detecting small moving target based on the improved ORB feature matching[J]. Opto-Electronic Engineering, 2015, 42(10): 13-20. doi: 10.3969/j.issn.1003-501X.2015.10.003
[7] 黄超群. 基于混合高斯模型和Kalman滤波器的运动目标检测与跟踪[D]. 昆明: 云南大学, 2010.
Huang C Q. Motion detection and tracking based on gaussian mixture model and kalman filter[D]. Kunming: Yunnan University, 2010.
http://cdmd.cnki.com.cn/Article/CDMD-10673-2010265273.htm [8] 张金敏, 王斌.光照快速变化条件下的运动目标检测[J].光电工程, 2016, 43(2): 14-21. doi: 10.3969/j.issn.1003-501X.2016.02.003
Zhang J M, Wang B. Moving object detection under condition of fast illumination change[J]. Opto-Electronic Engineering, 2016, 43(2): 14-21. doi: 10.3969/j.issn.1003-501X.2016.02.003
[9] 袁国武, 陈志强, 龚健, 等.一种结合光流法与三帧差分法的运动目标检测算法[J].小型微型计算机系统, 2013, 34(3): 668-671. http://www.cnki.com.cn/Article/CJFDTOTAL-XXWX201303050.htm
Yuan G W, Chen Z Q, Gong J, et al. A moving object detection algorithm based on a combination of optical flow and three-frame difference[J]. Journal of Chinese Computer Systems, 2013, 34(3): 668-671. http://www.cnki.com.cn/Article/CJFDTOTAL-XXWX201303050.htm
[10] 罗胜, Jiang Y Z.视频检测烟雾的研究现状[J].中国图象图形学报, 2013, 18(10): 1225-1236. doi: 10.11834/jig.20131002
Luo S, Jiang Y Z. State-of-art of video based smoke detection algorithms[J]. Journal of Image and Graphics, 2013, 18(10): 1225-1236. doi: 10.11834/jig.20131002
[11] Shi L F, Long F, Zhan Y J, et al. Video-based fire detection with spatio-temporal SURF and color features[C]//Proceedings of 2016 12th World Congress on Intelligent Control and Automation, 2016: 258-262.
[12] Dosovitskiy A, Fischery P, Ilg E, et al. Flownet: learning optical flow with convolutional networks[C]// Proceeding of 2015 IEEE International Conference on Computer Vision, 2015: 2758-2766.
[13] Ilg E, Mayer N, Saikia T, et al. Flownet 2. 0: evolution of optical flow estimation with deep networks[C]//Proceeding of 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1647-1655.
[14] 邓昊. 神经元图像识别相关方法探究[D]. 武汉: 湖北大学, 2016.
Deng H. Research on neuron image recognition[D]. Wuhan: Hubei University, 2016.
http://cdmd.cnki.com.cn/Article/CDMD-10512-1016158558.htm [15] Ouyang P, Hu H, Shi Z Z. Plankton classification with deep convolutional neural networks[C]//Proceeding of 2016 IEEE Information Technology, Networking, Electronic and Automation Control Conference, 2016: 132-136.
[16] He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//Proceeding of 2016 IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[17] He K M, Zhang X Y, Ren S Q, et al. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]//Proceeding of 2015 IEEE International Conference on Computer Vision, 2015: 1026-1034.
[18] He K M, Sun J. Convolutional neural networks at constrained time cost[C]//Proceeding of 2015 IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3992-4000.
[19] Horn B K P, Schunck B G. Determining optical flow[J]. Artificial Intelligence, 1981, 17(1-3): 185-203. doi: 10.1016/0004-3702(81)90024-2
[20] Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision (DARPA)[C]//Proceedings of the 1981 DARPA Image Understanding Workshop, 1981: 121-130.
-
访问统计
点击扫一扫
图(6)
表(3)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
