E-mail Alert
RSS
-
摘要:
针对目前利用相机标定参数进行图像拼接的方法存在受场景限制大、标定过程复杂而耗时长的问题，提出一种多场景下基于快速相机标定的柱面图像拼接方法。首先，利用棋盘格标定板角点特征提取精度高的特点，使其分别位于两两邻接图像的重叠视场中，对该图像序列依次进行角点提取、精确化和匹配等预处理，以准确快速求解出待拼接图像间的配准参数；然后利用标定得到的配准参数快速拼接图像，通过柱面投影以保持图像的视觉一致性，并采用多频段融合以保留图像的细节信息；最后，将整个系统搭建在低功耗嵌入式平台，实现可在多场景下完成快速标定及基于标定参数的拼接过程。实验结果表明，该文方法在室内及隧道等场景下可准确快速完成相机标定，图像拼接过程耗时短，同时可保证较高的拼接精度和较好的成像效果，具有较强的鲁棒性。
Abstract:A cylindrical image mosaic method based on fast camera calibration in multi-scene is proposed to solve the problems of scene limitation and complex calibration process in image mosaic using camera calibration parameter. Firstly, the accurate corner feature of checkerboard calibration board is used to make it in the overlapping field of view of two adjacent images. Then, the image sequence is pre-processed by corner extraction, precision and matching, so that the registration parameters between the images to be stitched can be solved accurately and quickly. After that, the cylindrical projection is used to maintain the visual consistency of the images, and the multi-band fusion is used to retain the details of the images. Subsequently, the images are stitched using registration parameters obtained by calibration. Finally, the whole system is built on a low-power embedded platform to accomplish fast calibration and mosaic process based on calibration parameters in multi-scene. The experiment results show that the proposed method can accomplish camera calibration quickly and accurately in indoor and tunnel scenarios, and the image mosaic process is time-consuming. Meanwhile, it can ensure better stitching accuracy and imaging effect, and has strong robustness.
-
Key words:
- image mosaic /
- camera calibration /
- cylindrical projection /
- multi-band fusion
-
Overview: Image mosaic is the process of combining two or more images with an overlapping field of view in the same scene to produce a seamless panorama or high-resolution image. The image obtained by mosaic has a larger field of view (FOV). Most of the cameras have a FOV angle of about 35 × 50 degrees, which limits the acquisition of information. Therefore, through image mosaic, the continuous image sequence of the same scene is stitched to form a composite image with a larger FOV, which can obtain all visual information at a given point of view at one time. This technology plays an important role in many fields, such as geological survey, medical minimally invasive surgery, and virtual reality. Its technical advantages are obvious. Researchers at home and abroad have done a lot of researches on image mosaic, and image registration is the key step. At present, there are mainly three classical registration methods based on frequency domain, gray level, and feature points, respectively. Nevertheless, the above image registration algorithms generally have the problems of large computational load and low execution efficiency. In this regard, some domestic researchers have proposed a camera calibration method, which saves most of the time needed for stitching and achieves high stitching accuracy.
But at present, the mosaic algorithm based on camera calibration is limited by the scene, and the calibration process is complex. The collinear condition in imaging will be destroyed after image transformation, which is not conducive to subsequent image processing and information classification. For this reason, a cylindrical image mosaic method based on fast camera calibration in multi-scene is proposed. This method makes full use of the high accuracy of feature extraction of checkerboard calibration board, which is used to make it in the overlapping field of view of two adjacent images. In order to accurately and quickly solve the registration parameters between the images to be spliced, the image sequence is pre-processed by corner extraction, precision and matching. Then, cylindrical projection and multi-band fusion are used to maintain visual consistency and detailed information. The system is based on a Low-Power Embedded platform, which achieves fast acquisition and accurate mosaic of camera calibration parameters in multi-scene. The experiment results show that the proposed method can accomplish camera calibration quickly and accurately in indoor and tunnel scenarios, and the image mosaic process is time-consuming. Meanwhile, it can ensure better stitching accuracy and imaging effect, and has strong robustness.
The proposed method has positive significance for real-time image stitching without feature points or large environmental changes.
-
-
图 11 拼接结果。(a)室内拼接图；(b)隧道拼接图
Figure 11. The stitching results. (a) The mosaic image in indoor; (b) The mosaic image in tunnel
图 13 无柱面投影变换拼接结果
Figure 13. Stitching results without cylindrical projection transformation
图 14 Roll方向(a)和pitch方向(b)下旋转平台
Figure 14. Rotary platform in roll direction (a) and pitch direction (b)
图 15 Roll方向像素焦距(a)和平移值(b)实验结果
Figure 15. The experimental results of roll direction pixel focal length (a) and shift value (b)
图 16 Roll和pitch方向拼接结果对比
Figure 16. Comparisons of splicing results in roll and pitch direction
图 17 Pitch方向像素焦距(a)和平移值(b)实验结果
Figure 17. The experimental results of pitch direction pixel focal length (a) and shift value (b)
图 18 标定板不同位置的拼接结果
Figure 18. Stitching results of different positions of the calibration plate
表 1 单应性矩阵参数介绍
Table 1. Introduction of homography matrix parameters
参数 作用 h11、h12、h21、h22 表示图像线性变换 h13、h23 表示图像平移 h31、h32 用于产生图像透视变换 
下载: 导出CSV
表 2 图像拼接过程相关参数
Table 2. The parameters of image mosaic process
名称 参数 单幅图像分辨率 1.23千万像素 单幅图像视场角(D×H×V) 22°×36°×30° 镜头焦距 25 mm (可调) 相机固定旋转角度 18° 相机相邻位姿间图像重叠度 约22% 待拼接图像数量 8幅
下载: 导出CSV
表 3 图像拼接速度对比
Table 3. Comparisons of image mosaic speed
s 本文方法 OpenCV 基于SIFT方法 Image1r_2l 0.1857 1.3945 11.9374 Image2r_3l 0.1773 1.8812 9.1451 Image3r_4l 0.1981 1.3327 6.7098 Image4r_5l 0.1804 1.3343 7.2453 Image5r_6l 0.1851 1.8821 10.1633 Image6r_7l 0.1879 1.5326 7.7876 Image7r_8l 0.1834 1.3769 6.8534
下载: 导出CSV
表 4 标定板放置位置误差分析
Table 4. Error analysis of calibration plate placement
V/pixel Δu=640 pixels Δu=500 pixels Δu=350 pixels f/pixel Lu/pixel f/pixel Lu/pixel f/pixel Lu/pixel 1 2600 10859 3147 11809 3145 11757 3147 2 2400 10618 3135 11646 3134 10683 3143 3 2200 12331 3153 11767 3137 11882 3144 4 2000 11288 3145 11423 3145 11567 3140 5 1800 11677 3142 10895 3147 11652 3140 6 1600 11392 3136 10654 3138 11267 3136 7 1400 11523 3145 10739 3148 11516 3138 8 1200 12137 3142 11172 3148 11091 3147 9 1000 12375 3146 10539 3151 11540 3137 10 800 11709 3138 11485 3143 12335 3145 11 600 12193 3139 11592 3141 12506 3144 12 400 11639 3146 11873 3147 12184 3144
下载: 导出CSV
-
[1] Adel E, Elmogy M, Elbakry H. Image stitching based on feature extraction techniques: a survey[J]. International Journal of Computer Applications, 2014, 99(6): 1-8. doi: 10.5120/17374-7818
[2] Cheng Y F, Jin S Y, Wang M, et al. Image mosaicking approach for a double-camera system in the GaoFen2 optical remote sensing satellite based on the big virtual camera[J]. Sensors, 2017, 17(6): E1441. doi: 10.3390/s17061441
[3] Bosch J, Gracias N, Ridao P, et al. Omnidirectional underwater camera design and calibration[J]. Sensors, 2015, 15(3): 6033-6065. doi: 10.3390/s150306033
[4] 陈晨涛, 潘之玮, 沈会良, 等.一种人体热红外图像拼接及部位划分方法[J].光电工程, 2019, 46(9): 180689. doi: 10.12086/oee.2019.180689
Chen C T, Pan Z W, Shen H L, et al. Image stitching and partitioning algorithms for infrared thermal human-body images[J]. Opto-Electronic Engineering, 2019, 46(9): 180689. doi: 10.12086/oee.2019.180689
[5] Kaynig V, Fischer B, Müller E, et al. Fully automatic stitching and distortion correction of transmission electron microscope images[J]. Journal of Structural Biology, 2010, 171(2): 163-173. doi: 10.1016/j.jsb.2010.04.012
[6] 吴丽萍, 胡郁.柱面全景图图像拼接中图像平滑的虚拟现实技术[J].科学技术与工程, 2017, 17(31): 271-276. doi: 10.3969/j.issn.1671-1815.2017.31.044
Wu L P, Hu Y. Virtual reality technology of image smoothing in cylindrical panoramic image mosaic[J]. Science Technology and Engineering, 2017, 17(31): 271-276. doi: 10.3969/j.issn.1671-1815.2017.31.044
[7] Seo S, Jeonz S, Lee S. Efficient homography estimation method for panorama[C]//Proceedings of the 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, 2013: 209-212.
[8] Szeliski R. Video mosaics for virtual environments[J]. IEEE Computer Graphics and Applications, 1996, 16(2): 22-30. doi: 10.1109/38.486677
[9] Zou L H, Chen J, Zhang J, et al. An image mosaicing approach for video sequences based on space-time manifolds[C]//Proceedings of the 29th Chinese Control Conference, 2010.
[10] Li L N, Geng N. Algorithm for sequence image automatic mosaic based on sift feature[C]//Proceedings of 2010 Wase International Conference on Information Engineering, 2010.
[11] Wu Z W, Zhu L R, Sun X C. Expansion of the visual angle of a car rear-view image via an image mosaic algorithm[J]. Optical Engineering, 2015, 54(5): 053101. doi: 10.1117/1.OE.54.5.053101
[12] Zhou A W, Shao W, Guo J J. An image mosaic method for defect inspection of steel rotary parts[J]. Journal of Nondestructive Evaluation, 2016, 35(4): 60. doi: 10.1007/s10921-016-0375-3
[13] Alomran M, Chai D. Feature-based panoramic image stitching[C]//Proceedings of 2016 14th International Conference on Control, 2016.
[14] 张金玲, 孙汉旭, 贾庆轩, 等.基于相机标定的图像拼接算法[J].中北大学学报(自然科学版), 2008, 29(6): 575-579. doi: 10.3969/j.issn.1673-3193.2008.06.021
Zhang J L, Sun H X, Jia Q X, et al. Image mosaic algorithm based on camera calibration[J]. Journal of North University of China (Natural Science Edition), 2008, 29(6): 575-579. doi: 10.3969/j.issn.1673-3193.2008.06.021
[15] 王冬, 刘凤英, 陈天恩, 等.一种相机标定参数的柱面全景影像拼接方法[J].测绘科学, 2016, 41(7): 150-154, 143. http://d.old.wanfangdata.com.cn/Periodical/chkx201607027
Wang D, Liu F Y, Chen T E, et al. The method of cylinder panoramic image mosaic based on camera calibration parameters[J]. Science of Surveying and Mapping, 2016, 41(7): 150-154, 143. http://d.old.wanfangdata.com.cn/Periodical/chkx201607027
[16] 马嘉琳, 张锦明, 孙卫新.基于相机标定的全景图拼接方法研究[J].系统仿真学报, 2017, 29(5): 1112-1119. http://d.old.wanfangdata.com.cn/Periodical/xtfzxb201705024
Ma J L, Zhang J M, Sun W X. Research on panoramic image mosaic method based on camera calibration[J]. Journal of System Simulation, 2017, 29(5): 1112-1119. http://d.old.wanfangdata.com.cn/Periodical/xtfzxb201705024
[17] Zhang Z Y. Flexible camera calibration by viewing a plane from unknown orientations[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999.
[18] Hartley R, Zisserman A. Multiple view geometry in computer vision[M]. Cambridge university press, 2003
[19] Chuan Z, Long T D, Feng Z, et al. A planar homography estimation method for camera calibration[C]//Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium, 2003.
[20] Lenth R V. Least-squares means: the R package lsmeans[J]. Journal of Statistical Software, 2016, 69(1): 1-33.
[21] Janota A, Šimák V, Nemec D, et al. Improving the precision and speed of euler angles computation from low-cost rotation sensor data[J]. Sensors, 2015, 15(3): 7016-7039. doi: 10.3390/s150307016
[22] Pulli K, Baksheev A, Kornyakov K, et al. Real-time computer vision with OpenCV[J]. Communications of the ACM, 2012, 55(6): 61-69. doi: 10.1145/2184319.2184337
[23] Xu Y, Zhou Q H, Gong L W, et al. High-speed simultaneous image distortion correction transformations for a multicamera cylindrical panorama real-time video system using FPGA[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(6): 1061-1069. doi: 10.1109/TCSVT.2013.2290576
[24] 郭海霞, 郭海龙, 解凯.基于边缘信息改进的双线性插值算法[J].计算机工程与应用, 2011, 47(31): 171-174. doi: 10.3778/j.issn.1002-8331.2011.31.049
Guo H X, Guo H L, Xie K. Improved bilinear interpolation algorithm based on edge information[J]. Computer Engineering and Applications, 2011, 47(31): 171-174. doi: 10.3778/j.issn.1002-8331.2011.31.049
[25] 郭治成, 党建武, 王阳萍, 等.基于多特征融合的背景建模方法[J].光电工程, 2018, 45(12): 180206. doi: 10.12086/oee.2018.180206
Guo Z C, Dang J W, Wang Y P, et al. Background modeling method based on multi-feature fusion[J]. Opto-Electronic Engineering, 2018, 45(12): 180206. doi: 10.12086/oee.2018.180206
[26] Brown M, Lowe D G. Automatic panoramic image stitching using invariant features[J]. International Journal of Computer Vision, 2007, 74(1): 59-73. http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ026867114/
-
点击扫一扫
图(18)
表(4)
计量
- 文章访问数:
- PDF下载数:
- 施引文献: 0
