Principal Component Initialization Method With Unlabeled Data

To solve the problem of uncertainty of initialization methods and insufficient amount of labeled data, an initialization method, which used unlabeled data for principal component initialization of network parameters and included four steps, i.e., sampling, principal component computation, initializing and rearranging the convolutional kernels, was proposed in this paper. First, the feature map was sampled with moving the reception fields, then the image blocks corresponding to all the receptive fields of the feature map were obtained and a sampling set was formed. Second, the principal components of the sampling set were computed. Finally, the principal components were used to initialize the network parameters and the convolutional kernels were rearranged to improve recognition performance. In cases with the same network architecture and datasets, i.e., STL-10 and CIFAR-10 datasets, an increase of recognition accuracy of 4%-20% was obtained by this method in comparison with the traditional initialization methods. The results indicate that the proposed method can make full use of unlabeled data for initializing network parameters to obtain remarkable recognition effectiveness. Moreover, through the performance evaluation of the algorithm, it proves that the method is obviously better than the traditional initialization methods.