Handwritten Formula Recognition Based on Segmentation of Adhesive Symbols and Multi-feature Fusion

To solve the problem that character adhesion affects the automatic recognition of offline handwritten mathematical formulas, a segmentation method of single-point adhesive characters was proposed in this paper based on character contour features. The segmentation point and directions were obtained by using the information of contour direction codes on the upper and lower sides of characters. Then, the multi-feature fusion was realized by combining the geometric characteristics such as width, height, number of corners and projection contour. Finally, combined with the spatial position relationship characteristics of the special symbols and the surrounding characters, such as up, down, left, right, overlap and half encirclement, the structure analysis was conducted, and the common characters recognized by convolution neural network were substituted by the structure analysis sequence to elicit the overall recognition of the formula. The experimental results show that this method can effectively deal with the adhesion and special symbol recognition in mathematical formulas. The segmentation accuracy of adhesive characters has reached 87.25%. At the same time, the overall recognition rate of handwritten mathematical formula is further improved.