- 综合性科技类中文核心期刊
- 中国科技论文统计源期刊
- 中国科学引文数据库来源期刊
- 中国学术期刊文摘数据库（核心版）来源期刊
- 中国学术期刊综合评价数据库来源期刊
| Citation: |
BAO Zhenshan, MI Bowen, ZHANG Wenbo. NAS Algorithm Based on Manual Experience Network Architecture Initialization[J]. Journal of Beijing University of Technology, 2021, 47(8): 854-862. DOI: 10.11936/bjutxb2021010004
|
To solve the shortcomings of neural architecture search (NAS), such as high computing power requirements and long search time, combined with the manual design experience of deep neural networks, an NAS algorithm based on manual experience network architecture initialization was proposed. The algorithm redesigned the search space and selected VGG-11 as the initial architecture, which effectively reduced the invalid search caused by the random initialization of the parameters. Based on the above design scheme, experimental verification was carried out on the classic image classification dataset Cifar-10. The VGG-Lite structure was obtained by searching for 12 hours, and the error rate of this model was 2.63%. The model VGG-Lite was 0.83% more accurate than DenseNet-BC, the best-performing artificial design structure at this stage. The number of parameters of this architecture was 1.48 M, which was about 1/17 of the DenseNet-BC number of parameters. Results show that this method can search for excellent network architectures and significantly improve the search efficiency, which is of great significance to the popularization of NAS algorithms.
| [1] |
ZOPH B, LE Q V. Neural architecture search with reinforcement learning[J]. ArXiv Preprint ArXiv, 2016: 1611.01578.
|
| [2] |
HOWARD A, SANDLER M, CHU G, et al. Searching for mobilenetv3[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 1314-1324.
|
| [3] |
CAI H, GAN C, WANG T, et al. Once-for-all: train one network and specialize it for efficient deployment[J]. ArXiv Preprint ArXiv, 2019: 1908.09791.
|
| [4] |
TAN M, CHEN B, PANG R, et al. Mnasnet: platform-aware neural architecture search for mobile[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 2820-2828.
|
| [5] |
PENG H, DU H, YU H, et al. Cream of the crop: distilling prioritized paths for one-shot neural architecture search[J]. ArXiv Preprint ArXiv, 2020: 2010.15821.
|
| [6] |
REN P, XIAO Y, CHANG X, et al. A comprehensive survey of neural architecture search: challenges and solutions[J]. ArXiv Preprint ArXiv, 2020: 2006.02903.
|
| [7] |
ZHOU D, ZHOU X, ZHANG W, et al. EcoNAS: finding proxies for economical neural architecture search[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11396-11404.
|
| [8] |
尹宝才, 王文通, 王立春. 深度学习研究综述[J]. 北京工业大学学报, 2015, 41(1): 48-59. doi: 10.11936/bjutxb2014100026
YIN B C, WANG W T WANG L C. Review of deep learning[J]. Journal of Beijing University of Technology, 2015, 41(1): 48-59. (in Chinese) doi: 10.11936/bjutxb2014100026
|
| [9] |
成科扬, 王宁, 师文喜, 等. 深度学习可解释性研究进展[J]. 计算机研究与发展, 2020, 57(6): 1208-1217. https://www.cnki.com.cn/Article/CJFDTOTAL-JFYZ202006008.htm
CHENG K Y, WANG N, SHI W X, et al. Research advances in the interpretability of deep learning[J]. Journal of Computer Research and Development, 2020, 57(6): 1208-1217. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-JFYZ202006008.htm
|
| [10] |
SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. ArXiv Preprint ArXiv, 2014: 1409.1556.
|
| [11] |
CIFAR-10 and CIFAR-100 datasets (toronto. edu)[DB/OL]. [2020-12-01]. http://www.cs.toronto.edu/~kriz/cifar.html.
|
| [12] |
MNIH V, KAVUKCUOGLU K, SILVER D, et al. Playing atari with deep reinforcement learning[J]. ArXiv Preprint ArXiv, 2013: 1312.5602.
|
| [13] |
HE X, ZHAO K, CHU X. AutoML: a survey of the state-of-the-art[J]. Knowledge-Based Systems, 2019, 212: 106622. http://www.sciencedirect.com/science/article/pii/S0950705120307516
|
| [14] |
ELSKEN T, METZEN J H, HUTTER F. Neural architecture search: a survey[J]. J Mach Learn Res, 2019, 20(55): 1-21.
|
| [15] |
SAMPSON J R. Adaptation in natural and artificial systems[J]. SIAM Review, 1976, 18(3): 529-530. doi: 10.1137/1018105
|
| [16] |
孟子尧, 谷雪, 梁艳春, 等. 深度神经架构搜索综述[J]. 计算机研究与发展, 2021, 58(1): 22-33.
MENG Z Y, GU X, LIANG Y C, et al. Deep neural architecture search: a survey[J]. Journal of Computer Research and Development, 2021, 58(1): 22-33. (in Chinese)
|
| [17] |
KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. doi: 10.1145/3065386
|
| [18] |
ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//European Conference on Computer Vision. Berlin: Springer, 2014: 818-833.
|
| [19] |
IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//International Conference on Machine Learning. New York: ACM, 2015: 448-456.
|
| [20] |
HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
|
| [21] |
XIE S, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1492-1500.
|
| [22] |
HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
|
| [23] |
HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780. doi: 10.1162/neco.1997.9.8.1735
|
| [24] |
WILLIAMS R J. Simple statistical gradient-following algorithms for connectionist reinforcement learning[J]. Machine Learning, 1992, 8(3/4): 229-256. doi: 10.1023/A:1022672621406
|
| [25] |
HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 4700-4708.
|
| [26] |
ZOPH B, VASUDEVAN V, SHLENS J, et al. Learning transferable architectures for scalable image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 8697-8710.
|
| [27] |
REAL E, AGGARWAL A, HUANG Y, et al. Regularized evolution for image classifier architecture search[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2019: 4780-4789.
|
| [28] |
PHAM H, GUAN M, ZOPH B, et al. Efficient neural architecture search via parameters sharing[C]//International Conference on Machine Learning. New York: ACM, 2018: 4095-4104.
|
| [29] |
LIU H, SIMONYAN K, YANG Y. DARTS: differentiable architecture search[C]//International Conference on Learning Representations. New Orleans: ICLR, 2019.
|
| [30] |
SANDLER M, HOWARD A, ZHU M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4510-4520.
|
| [1] | WANG Lichun, FU Fangyu, XU Kai, XU Hongbo, YIN Baocai. Region-sensitive Scene Graph Generation Method[J]. Journal of Beijing University of Technology, 2025, 51(1): 51-58. DOI: 10.11936/bjutxb2023020036 |
| [2] | SUN Yanfeng, DU Pengfei. Deep Clustering Network Based on Graph Relationship Selection[J]. Journal of Beijing University of Technology, 2024, 50(12): 1428-1436. DOI: 10.11936/bjutxb2023060023 |
| [3] | SUN Yanfeng, DU Pengfei. Node Clustering Based on Multi-channel Graph Convolutional Network[J]. Journal of Beijing University of Technology, 2023, 49(3): 355-362. DOI: 10.11936/bjutxb2021120027 |
| [4] | JI Chang-wei, GAO Bin-bin, LIU Xiao-long, WANG Shuo-feng, YANG Jin-xin. Development of a Quasi-dimensional Model for Predicting the Combustion Process of Hydrogen-enriched Gasoline Engines[J]. Journal of Beijing University of Technology, 2014, 40(12): 1863-1868. |
| [5] | JI Chang-wei, WANG Shuo-feng, ZHANG Bo, ZHENG Chao, ZHANG Xu, FAN Bo-yuan. Improving the Economic Performance of a Hydrogen-blended Gasoline Engine at Idle[J]. Journal of Beijing University of Technology, 2012, 38(10): 1548-1553. |
| [6] | JI Chang-wei, YAN Hao, WANG Shuo-feng, ZHANG Bo, ZHANG Min-yue, ZHOU Xiao-long. Simulation of Lean Combustion Characteristics of a Spark Ignition Engine Fueled With Gasoline-Hydrogen Fuel Mixture[J]. Journal of Beijing University of Technology, 2012, 38(3): 417-422. |
| [7] | JI Chang-wei, ZHANG Bo, WANG Shuo-feng, ZHANG Min-yue, ZHANG Jian, FAN Bo-yuan. Emission Performance of a Hydrogen-enriched Gasoline Engine[J]. Journal of Beijing University of Technology, 2012, 38(1): 151-155. |
| [8] | HAN Ai-min, LIN Xin-feng, SUN Bai-gang, LIU Ping, WANG Jia-xing. Optimization Performance of Model BN6V87QE Gasoline Engine by GT-Power[J]. Journal of Beijing University of Technology, 2007, 33(6): 617-621,626. |
| [9] | LI Xing-hu, CUI Hai-long, ZHANG Bo-yan, HAN Ai-min, JIN Ke-cheng, ZHANG Xi-zhen. Continual Measurement and Analysis of Exhaust Composition of a Gasoline Engine[J]. Journal of Beijing University of Technology, 2002, 28(2): 173-177,193. |
| [10] | Joseph I. Tao. Graph and Matrix Operation[J]. Journal of Beijing University of Technology, 1995, 21(1): 1-4. |
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: