Citation: | MA Wei, LI Tong, GONG Chaofan, DING Zhiming. Stereo Matching With CNN and Constraints From Segmentation[J]. Journal of Beijing University of Technology, 2019, 45(5): 413-420. DOI: 10.11936/bjutxb2017110015 |
To solve the matching problem in textureless and occluded regions, a stereo matching algorithm was presented in this paper, which integrated segmentation clues and convolutional neural network (CNN) matching results into an Markov random field (MRF) energy function. The unary term of the energy function was formulated by the output of CNN, which was powerful in feature expression. The pairwise smooth term was defined by the over-segmentation results of the input images to constrain matching in textureless and occluded regions. Experiments on Middlebury and KITTI datasets validated the performances of the algorithm and its components. Results show that the algorithm outperforms the other methods proposed in recent years, especially in dealing with textureless and occluded regions.
[1] |
MATCHING S, OBSTACLE N, OBSTACLE P, et al. Collision-free navigation for blind persons using stereo matching[J]. International Journal of Scientific and Engineering Research, 2011, 2(12):1-8. http://cn.bing.com/academic/profile?id=d5916ed03e373c9802b6238f002d3d9a&encoded=0&v=paper_preview&mkt=zh-cn
|
[2] |
MANAP N A, CATERINA G D, SORAGHAN J, et al. Smart surveillance system based on stereo matching algorithms with IP and PTZ cameras[C]//3DTV-Conference: the True Vision-Capture, Transmission and Display of 3D Video. Piscataway: IEEE, 2010: 1-4. https://www.researchgate.net/publication/224154481_Smart_surveillance_system_based_on_stereo_matching_algorithms_with_IP_and_PTZ_cameras
|
[3] |
BLEYER M, RHEMANN C, ROTHER C. Patchmatch stereo-stereo matching with slanted support windows[C]//British Machine Vision Conference. Dundee: BMVA, 2011: 1-11.
|
[4] |
YOON K J, KWEON I S. Adaptive support-weight approach for correspondence search[J]. Pattern Analysis and Machine Intelligence, 2006, 28(4):650-656. doi: 10.1109/TPAMI.2006.70
|
[5] |
HIRSCHMULLER H. Stereo vision in structured environments by consistent semi-global matching[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2006: 2386-2393. https://www.researchgate.net/publication/4246293_Stereo_Vision_in_Structured_Environments_by_Consistent_Semi-Global_Matching
|
[6] |
MEI X, SUN X, DONG W, et al. Segment-tree based cost aggregation for stereo matching[C]//Computer Vision and Pattern Recognition. Piscataway: IEEE, 2013: 313-320. https://www.researchgate.net/publication/256473396_Segment-Tree_Based_Cost_Aggregation_for_Stereo_Matching
|
[7] |
YANG Q. A non-local cost aggregation method for stereo matching[C]//IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2012: 1402-1409. https://www.researchgate.net/publication/261115904_A_non-local_cost_aggregation_method_for_stereo_matching
|
[8] |
WEI D, LIU C, FREEMAN W T. A data-driven regularization model for stereo and flow[C]//International Conference on 3D Vision. Piscataway: IEEE, 2015: 277-284. https://www.researchgate.net/publication/282922132_A_Data-Driven_Regularization_Model_for_Stereo_and_Flow
|
[9] |
GUNEY F, GEIGER A. Displets: resolving stereo ambiguities using object knowledge[C]//Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 4165-4175. https://www.researchgate.net/publication/308842942_Displets_Resolving_stereo_ambiguities_using_object_knowledge
|
[10] |
HAMZAH R A, IBRAHIM H, HASSAN A H A. Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation[J]. Journal of Visual Communication & Image Representation, 2016, 42:145-160. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=99f88733d43487b5afcd493feff69580
|
[11] |
MA N, MEN Y, MEN C, et al. Accurate dense stereo matching based on image segmentation using an adaptive multi-cost approach[J]. Symmetry, 2016, 8(12):159-180. doi: 10.3390/sym8120159
|
[12] |
BRICOLA J C, BILODEAU M, BEUCHER S. Morphological processing of stereoscopic image superimpositions for disparity map estimation[C]//European Conference on Computer Vision. Berlin: Springer-Verlag, 2016: 1308-1383.
|
[13] |
ZAGORUYKO S, KOMODAKIS N. Learning to compare image patches via convolutional neural networks[C]//Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 4353-4361. https://www.researchgate.net/publication/275055327_Learning_to_Compare_Image_Patches_via_Convolutional_Neural_Networks?ev=auth_pub
|
[14] |
KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-end learning of geometry and context for deep stereo regression[C]//IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 66-75. https://www.researchgate.net/publication/314943348_End-to-End_Learning_of_Geometry_and_Context_for_Deep_Stereo_Regression
|
[15] |
XU S, ZHANG F, HE X, et al. PM-PM:patchmatch with potts model for object segmentation and stereo matching[J]. Transactions on Image Processing, 2015, 24(7):2182-2196. doi: 10.1109/TIP.2015.2416654
|
[16] |
DUPLAN J F, MONNOT P. Joint optimisation for object class segmentation and dense stereo reconstruction[J]. International Journal of Computer Vision, 2012, 100(2):122-133. http://cn.bing.com/academic/profile?id=c01a50b84e6a53cf35cf59f9d3b17851&encoded=0&v=paper_preview&mkt=zh-cn
|
[17] |
MOZEROV M G, WEIJER J V D. Accurate stereo matching by two-step energy minimization[J]. IEEE Transactions on Image Processing, 2015, 24(3):1153-1163. doi: 10.1109/TIP.2015.2395820
|
[18] |
ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Aralysis and Machine Intelligence, 2012, 34(1):2274-2281. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=218e2dd693ea07f7be3f296ed4e6aaba
|
[19] |
BOYKOV Y, KOLMOGOROV V. An experimental comparison of min-cut/max-flow algorithms for energy minimi-zation in vision[J]. Transactions on Pattern Analysis & Machine Intelligence, 2004, 26(9):1124-1137. http://cn.bing.com/academic/profile?id=829adb32d472e680537bef613c3895dd&encoded=0&v=paper_preview&mkt=zh-cn
|
[20] |
ŽBONTAR J, LECUN Y. Stereo matching by training a convolutional neural network to compare image patches[J]. Journal of Machine Learning Research, 2016, 17(1):2287-2318. http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=Arxiv000001194580
|
[21] |
SCHARSTEIN D, HIRSCHMÜLLER H, KITAJIMA Y, et al. High-resolution stereo datasets with subpixel-accurate ground truth[C]//German Conference on Pattern Recognition. Berlin: Springer-Verlag, 2014: 31-42. doi: 10.1007%2F978-3-319-11752-2_3
|
[22] |
HIRSCHMÜLLER H, SCHARSTEIN D. Evaluation of cost functions for stereo matching[C]//Computer Vision & Pattern Recognition. Piscataway: IEEE, 2007: 1-8. https://www.researchgate.net/publication/224989203_Evaluation_of_Cost_Functions_for_Stereo_Matching
|
[23] |
MENZE M, GEIGER A. Object scene flow for autonomous vehicles[C]//Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 3061-3070. https://www.researchgate.net/publication/304604590_Object_scene_flow_for_autonomous_vehicles
|
[24] |
RICHRD S, RAMIN Z, DANIEL S, et al. A comparative study of energy minimization methods for Markov Random Fields[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2007, 30(6):1068-1080. http://cn.bing.com/academic/profile?id=e5979d2408d2877f216e6a5efd23556d&encoded=0&v=paper_preview&mkt=zh-cn
|