E-mail Alert
RSS
-
摘要
针对平台运动导致的视频抖动问题，提出了一种基于光流传感器的视频稳像技术。该方案首先通过对一般光流传感器的改进，使其具有旋转运动下输出准确运动矢量的能力，然后利用光流传感器获得相邻两帧图像之间的运动矢量，并通过坐标变换计算出主相机的实时平移和旋转信息；其次，对原视频图像序列进行运动补偿，以获得稳定的图像序列，最终实现了视频稳像。实验结果表明，稳像后的图像序列与未稳像之前相比峰值信噪比提高了11.86 dB。该方案在视频抖动较大的情况下，能够明显减小图像序列的抖动现象，具有稳像效果好的特点，满足视频稳像的性能要求，对提高平台抗扰能力有着较高的实用性。
Abstract
In allusion of the video jitter problem caused by platform motion, a video stabilization technique based on optical flow sensor is presented. Firstly, the scheme improves the general optical flow sensor to output accurate motion vectors under rotational motion, then motion vectors between adjacent frames are obtained by using the optical flow sensor. The real-time translation and rotation information of the main camera are calculated through coordinate transformation. Secondly, the method compensates the motion of video sequences to attain stable video sequences, and finally realizes video stabilization. Experimental results indicate that, compared with the unstable image, the peak signal-to-noise ratio (PSNR) is increased by 11.86 dB. In the case of obvious video jitter, the scheme can significantly reduce the jitter between video sequences. The method which has the characteristics of salutary video stabilization can meet the performance requirements of video stabilization and improve the capacity of disturbance resistance for platform.
-
Overview
Overview: Video is one of the most intuitive and effective information carrier in the field of machine vision. Through the analysis of video information, the machine can perceive the surrounding environment and make relevant decisions. However, in the field of camera platform, such as shipborne, on-vehicle and hand-held, the bumping and jitter of the camera system which is due to the attitude change and vibration of the platform will cause the instability of the video sequences. It will seriously affect the information collection and decision for surrounding environment situation. Therefore, the video stabilization has important significance and value in practical engineering applications. The traditional video stabilization technique has mechanical image stabilization, optical image stabilization and electronic image stabilization. Mechanical image stabilization detects motion information and compensates the offset of video sequences by motor calibration system; optical image stabilization achieves image stabilization by adjusting the structure of the optical element to change the optical path and compensate the offset of the optical axis. The first two image stabilization technique are often expensive, difficult, and complicated to operate, which limits their application in small-sized and low-cost field of camera. The electronic image stabilization technique obtains the motion vectors of the video sequences by motion analysis and compensates the motion to obtain stable video sequences. However, this method tends to have a large amount of calculation and low precision, which limits its application range. In allusion of the above video stabilization problem, a video stabilization technique based on optical flow sensor is presented. Firstly, the scheme makes the general optical flow sensor overcome the shortcomings of the SAD algorithm under rotational motion and output accurate motion vectors. In addition, since the main camera and the optical flow sensor have a fixed spatial positional relationship, the real-time translation and rotation information of the main camera is calculated through coordinate transformation with the motion vectors obtained by the optical flow sensor. Then, the original unstable video sequences are compensated by frame, and finally the video stabilization is realized. Experimental results indicate that, compared with the unstable image, the peak signal-to-noise ratio (PSNR) is increased by 11.86 dB, and compared with the electronic image stabilization algorithm based on feature point matching, the processing time is decreased by 54.65% with similar image stabilization effect. In the case of obvious video jitter, the scheme can significantly reduce the jitter between video sequences. The method which has the characteristics of salutary video stabilization and high speed can meet the performance requirements of video stabilization and improve the capacity of disturbance resistance for platform.
-
-
图 2 光流传感器改进原理示意图。(a)改进之前图像；(b)旋转校正后图像；(c)剪切后图像
Figure 2. Framework of improvement principle of optical flow sensor. (a) Before improvement; (b) After rotation correction; (c) After sheared
图 3 光流传感器改进前后的输出图像。(a)改进之前的图像；(b)改进之后的图像
Figure 3. Image before and after improvement of optical flow sensor. (a) Before improvement; (b) After improvement
图 6 两相机的运动和计算模型。(a)相机实际运动模型；(b)平移量计算模型
Figure 6. Motion and computation model of two cameras. (a) Camera real motion model; (b) Translation computation model
图 7 实验稳像的结果图。(a)第1帧图像；(b)第9帧图像；(c)第10帧图像；(d)第13帧图像；(e)第16帧图像；(f)第9帧补偿后图像；(g)第10帧补偿后图像；(h)第13帧补偿后图像；(i)第16帧补偿后图像
Figure 7. Results of experimental image stabilization. (a) First frame of image; (b) Ninth frame of image; (c) Tenth frame of image; (d) Thirteenth frame of image; (e) Sixteenth frame of image; (f) Ninth frame compensated image; (g) Tenth frame compensated image; (h) Thirteenth frame compensated image; (i) Sixteenth frame compensated image
图 8 灰度列投影曲线。(a)第1帧和第16帧截取图像的灰度投影曲线；(b)第1帧和经旋转校正后的第16帧截取图像的灰度投影曲线；(c)第1帧和经旋转平移补偿后的第16帧截取图像的灰度投影曲线
Figure 8. Gray column projection curve. (a) Gray column projection curve of intercepted image in first and sixteenth frames; (b) Gray column projection curve of intercepted image in first frame and sixteenth frame after rotation correction; (c) Gray column projection curve of intercepted image in first frame and sixteenth frame after rotation translation compensation
图 9 差值图像比较。(a)未稳像差值图；(b)稳像后差值图
Figure 9. Comparison of difference image. (a) Original difference image; (b) Stabilized difference image
表 1 视频图像序列的运动补偿量
Table 1. Motion compensation data of video image sequence
帧号 旋转角度/rad x轴平移量 y轴平移量 2 0.00043 0 0 5 0.00109 4 -2 10 0.01932 29 40 15 0.17906 -118 311 20 0.11421 -68 210 25 0.01871 22 31 30 0.05589 -2 108 35 0.09805 -32 243 40 0.00885 36 19 45 0.09153 -25 188 50 0.18182 -112 320 55 0.01479 36 32 60 0.06017 6 134 65 0.12485 -35 253 70 0.00593 35 14 75 0.13505 -49 307 80 0.05299 -8 103 85 0.00925 22 20 90 0.19287 -129 348 95 0.03074 2 7 100 0.02645 6 64 
下载: 导出CSV
表 2 算法性能比较
Table 2. Performance comparison of algorithm
特征点匹配算法 本文方法 未稳像前 时间/s 标准差 时间/s 标准差 标准差 8.569 12.44 3.886 11.73 21.18
下载: 导出CSV
-
参考文献
[1] 刘文.船载移动视频电子稳像方法研究[D].大连: 大连海事大学, 2017.
Liu W. Research on the methods of electronic image stabilization for shipborne mobile video[D]. Dalian: Dalian Maritime University, 2017.
http://cdmd.cnki.com.cn/Article/CDMD-10151-1017209269.htm [2] 张贝.基于特征提取与灰度投影联合稳像的算法研究[D].哈尔滨: 哈尔滨工程大学, 2010.
Zhang B. Research on united image stabilization based on feature tracking and gray projection[D]. Harbin: Harbin Engineering University, 2010.
[3] 冯筠, 延瑜瑜, 赵妍, 等.基于学习不变特征变换的兵马俑图像分区匹配[J].光学 精密工程, 2018, 26(7): 1774–1783. http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201807024
Feng J, Yan Y Y, Zhao Y, et al. A terracotta image partition matching method based on learned invariant feature transform[J]. Optics and Precision Engineering, 2018, 26(7): 1774–1783. http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201807024
[4] 曾峦, 翟优, 谭久彬.基于SIFT的自动匹配策略[J].光电工程, 2011, 38(2): 65–70. doi: 10.3969/j.issn.1003-501X.2011.02.011
Zeng L, Zhai Y, Tan J B. Automatic matching strategy based on SIFT[J]. Opto-Electronic Engineering, 2011, 38(2): 65–70. doi: 10.3969/j.issn.1003-501X.2011.02.011
[5] Honegger D, Meier L, Tanskanen P, et al. An open source and open hardware embedded metric optical flow CMOS camera for indoor and outdoor applications[C]//2013 IEEE International Conference on Robotics and Automation, Karlsruhe, 2013: 1736–1741.
[6] 龙达峰, 刘俊, 张晓明, 等.基于椭球拟合的三轴陀螺仪快速标定方法[J].仪器仪表学报, 2013, 34(6): 1299–1305. doi: 10.3969/j.issn.0254-3087.2013.06.015
Long D F, Liu J, Zhang X M, et al. Triaxial gyroscope fast calibration method based on ellipsoid fitting[J]. Chinese Journal of Scientific Instrument, 2013, 34(6): 1299–1305. doi: 10.3969/j.issn.0254-3087.2013.06.015
[7] 纪慧中, 贾大宇, 董恩清, 等.基于图像特征和光流场的非刚性图像配准[J].光学 精密工程, 2017, 25(9): 2469–2482. http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201709026
Ji H Z, Jia D Y, Dong E Q, et al. Non-rigid registrations based on image characteristics and optical flows[J]. Optics and Precision Engineering, 2017, 25(9): 2469–2482. http://d.old.wanfangdata.com.cn/Periodical/gxjmgc201709026
[8] Lan J H, Li J, Hu G D, et al. Vehicle speed measurement based on gray constraint optical flow algorithm[J]. Optik, 2014, 125(1): 289–295. doi: 10.1016/j.ijleo.2013.06.036
[9] Huang R M, Li J Q, Shi M Y, et al. Quadrotor hovering scheme based on improved optical flow[C]//2017 Chinese Automation Congress (CAC), Jinan, 2017: 2357–2362.
[10] Tsai S E, Zhuang S H. Optical flow sensor integrated navigation system for quadrotor in GPS-denied environment[C]//2016 International Conference on Robotics and Automation Engineering (ICRAE), Jeju, 2016: 87–91.
[11] 熊晶莹.基于特征提取与匹配的车载电子稳像方法研究[D].长春: 中国科学院长春光学精密机械与物理研究所, 2017.
Xiong J Y. Study of electronic image stabilization based on feature detection and description for vehicles[D]. Changchun: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, 2017.
http://cdmd.cnki.com.cn/Article/CDMD-80139-1017064025.htm [12] 郑晓锋, 陈跃庭, 徐之海, 等.一种补偿平移与旋转运动的快速电子稳像算法[J].光子学报, 2008, 37(9): 1890–1894. http://d.old.wanfangdata.com.cn/Periodical/gzxb200809040
Zheng X F, Chen Y T, Xu Z H, et al. A fast electronic image stabilization algorithm for translational and rotational motion compensation[J]. Acta Photonica Sinica, 2008, 37(9): 1890–1894. http://d.old.wanfangdata.com.cn/Periodical/gzxb200809040
[13] 吕高杰, 张国华, 车宏.电子稳像算法精度评价方法研究[J].电光与控制, 2011, 18(3): 76–79. doi: 10.3969/j.issn.1671-637X.2011.03.019
Lv G J, Zhang G H, Che H. A method for evaluating the accuracy of electronic image stabilization system and algorithm[J]. Electronics Optics & Control, 2011, 18(3): 76–79. doi: 10.3969/j.issn.1671-637X.2011.03.019
-
访问统计
点击扫一扫
图(9)
表(2)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
