Abstract: To make the faster region-based convolutional neural network (Faster R-CNN) to fit the detection of crack objects with small-scale structures, a faster R-CNN-based crack detection and extraction method was proposed, which is able to preserve details of small objects and thus improve the performance of the detection. In the crack detection step, the faster R-CNN was considered as the framework of object detection, and the pre-trained visual geometry group model on ImageNet dataset was used as the feature extraction network. Then, the network model was adjusted to fit the small-scale crack object, and the optimal hyper-parameters was set by training to get object bounding-boxes of cracks. In the crack extraction step, according to the analysis of cracks, a crack extraction algorithm was proposed based on mathematical morphology to segment the object from the background, and one-pixel-wide cracks were finally extraced after denoising, linking and thinning. In this case, the length of the crack was be obtained by counting the number of pixels. Experimental results show that the proposed method can extract accurate and complete cracks, the average precision on the railroad crack dataset can achieve 63.87%, and the F₁-score on the road crack dataset can achieve 65.6%.