Abstract: To improve the accuracy and efficiency, an image retrieval method based on the fusion of global features and local features was proposed. The deep feature was selected as the global feature, and the speeded up robust features (SURF)and local binary pattern (LBP) were used as the local feature. The feature fusion based on canonical correlation analysis (CCA) has two shortcomings, i.e., information loss and information redundancy. To solve these problems, the criterion function was improved to obtain the basis vector that can minimize the correlation between features. Through the projection transformation on the basis vector, the independent information of the two feature vectors was obtained. On the basis of this, the final fusion was the independent information joined with the correlated information which was contained in one of the two features. The improved fusion method represented the original data more comprehensively and eliminated redundant information at the same time. In the experiments, it proves firstly that the fusion of depth feature and LBP feature has better discriminant ability in the application of image classification, the average accuracy reaches 99.1% with high time performance. A group of experiments were carried out to discuss the influence of different dimensions on feature fusion performance. Results show that increasing the dimension of feature selection can improve the classification accuracy to a certain extent. Finally, the image retrieval based on the fusion of deep feature and local feature was validated. In the experiment, the Manhattan distance was used to measure the similarity of the fused features, and the retrieval ranking was obtained based on the similarity measure. An accurate retrieval results was achieved on the experimental data set, the precision reached 98.0% while the recall reached 46.0%. The comparison shows that this method not only achieves reliable accuracy, but also has high time performance.