Stereo Matching With CNN and Constraints From Segmentation

To solve the matching problem in textureless and occluded regions, a stereo matching algorithm was presented in this paper, which integrated segmentation clues and convolutional neural network (CNN) matching results into an Markov random field (MRF) energy function. The unary term of the energy function was formulated by the output of CNN, which was powerful in feature expression. The pairwise smooth term was defined by the over-segmentation results of the input images to constrain matching in textureless and occluded regions. Experiments on Middlebury and KITTI datasets validated the performances of the algorithm and its components. Results show that the algorithm outperforms the other methods proposed in recent years, especially in dealing with textureless and occluded regions.