Abstract: This paper adopts the moving object tracking strategy based on object detection. Moving object detection is carried out using time difference method to obtain the initial position of the moving object, the position of the moving object is predicted at the next moment with Kalman filter; the prediction with the inspection result of the moving object is evaluated and corrected to achieve its precise position. In accordance with the tracking results, vehicle behavior can be analyzed. The experiments demonstrate that this method can provide an effective solution to reliable tracking of moving object partially shielded or fully shielded for a moment.