Deep Perception Structure Design Via Layer-wise Principal Component Analysis

To design a deep perception structure, an effective method was presented in this paper. By appropriately controlling information loss of training data, the number of neurons in each layer of a deep perception was adaptively determined by layer-wise principal component analysis (LPCA). At first, the number of input neurons and output neurons were taken as the training data dimension and the number of class labels respectively. Then, the number of neurons in the second layer was computed as a principal component analysis (PCA) dimension from the training data. Finally, the number of neurons in a layer between the second and the output layer were repeatedly computed from the activations of neurons in its previous layer followed by a PCA. Experimental results show that this LPCA method has superior performances in deep perception structure designing, such as simplifying the structure of deep perception, decreasing number of parameters, accelerating process of training, saving time for convergence. The idea of LPCA provides a new reference for designing deep perceptions and for applications.