New Frontiers in Communication and Intelligent Systems
Comparative Study of Surrogate Techniques for CNN Hyperparameter Optimization
Authors: Nurshazlyn Mohd Aszemi, Nordin Zakaria and P. D. D. Dominic
Publishing Date: 06-05-2022
ISBN: 978-81-95502-00-4
Abstract
Optimizing hyper parameters in Convolutional Neural networks is a tedious process for many researchers and practitioners. It requires a high degree of expertise or experience to optimize the hyper parameters, and manual optimization is likely to be biased. To date, methods or approaches to automate hyper parameter optimization include grid search, random search, and Genetic Algorithms (GAs). However, evaluating large number of sample points in the hyperparameter configuration space, as is typically required by these methods, is a computationally expensive process. Hence, the objective of this paper is to explore regression as a surrogate technique in CNN hyperparameter optimization. Performance in terms of accuracy, error rate, training time and coefficient of determination (R2) are evaluated and recorded. Although there is no significant performance difference between the resulting optimized Deep Learning and state-of-the-art on CIFAR-10 datasets, using regression as a surrogate technique for CNN hyperparameter optimization contributes to minimizing the time taken for the optimization process, a benefit which has not been fully explored in the literature to the best of the author's knowledge.
Keywords
Convolutional Neural Network, Regression, Hyperparameter, Optimization, Deep Learning, Machine Learning
Cite as
Nurshazlyn Mohd Aszemi, Nordin Zakaria and P. D. D. Dominic, "Comparative Study of Surrogate Techniques for CNN Hyperparameter Optimization", In: Rahul Srivastava and Aditya Kr. Singh Pundir (eds), New Frontiers in Communication and Intelligent Systems, SCRS, India, 2022, pp. 463-473. https://doi.org/10.52458/978-81-95502-00-4-48