Abstract: A method based on neighborhood information constraint and self-adaptive window was proposed to solve the problem of mismatching in disparity in discontinuous areas due to the sensitivity of traditional Census stereo matching algorithm to noise. First, in view of the high dependence of traditional Census algorithm of the central pixels, the weighted average sum of neighborhood cross windows was used to assign the central pixel. Second, by setting an adaptive threshold, the secondary cost of similarity between neighborhood pixels supporting windows and central pixels was calculated and fused with the initial cost, and the matching results were further constrained. In the cost aggregation stage, a three-constraint method with changing color thresholds was used to construct windows, and noise elimination strategy was introduced in the aggregation process. Finally, in the disparity refinement stage, the disparity map was further optimized by combining left-right consistency detection and regional voting. The standard stereo image of the Middlebury test platform was used to carry out experiments. Results show that this method can effectively reduce the sensitivity of image to Gauss noise, and the mismatch rate is lower than that of many stereo matching algorithms.