International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

A comparison of methods to reduce overfitting in neural networks

  • Kim, Ho-Chan (Department of Electrical Engineering, Jeju National University) ;
  • Kang, Min-Jae (Department of Electronic Engineering, Jeju National University)
  • Received : 2020.05.12
  • Accepted : 2020.05.21
  • Published : 2020.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

A common problem with neural network learning is that it is too suitable for the specificity of learning. In this paper, various methods were compared to avoid overfitting: regularization, drop-out, different numbers of data and different types of neural networks. Comparative studies of the above-mentioned methods have been provided to evaluate the test accuracy. I found that the more data using method is better than the regularization and dropout methods. Moreover, we know that deep convolutional neural networks outperform multi-layer neural networks and simple convolution neural networks.

Keywords

References

  1. Smith, Craig S, "The Man Who Helped Turn Toronto into a High-Tech Hotbed," The New York Times. Retrieved 27 June 2017.
  2. Wu H, Shapiro J L. (2006) Does overfitting affect performance in estimation of distribution algorithms. Conference on Genetic and Evolutionary Computation. ACM, pp.433-434.
  3. Karystinos G N, Pados D A. (2000) On overfitting, generalization, and randomly expanded training sets. IEEE Transactions on Neural Networks, 11(5):1050. https://doi.org/10.1109/72.870038
  4. Yann LeCun, Yoshua Bengio & Geoffrey Hinton, "Deep learning," Nature volume521, pages436-444 (28 May 2015). https://doi.org/10.1038/nature14539
  5. Srivastava N, Hinton G, Krizhevsky A, et al. (2014) Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1):1929-1958.
  6. Yip K Y, Gerstein M. (2009) Training set expansion: an approach to improving the reconstruction of biological networks from limited and uneven reliable interactions. State of the art of air pollution control techniques for industrial processes and power generation. Dept. of Civil Engineering, College of Engineering, University of Tennessee, pp.243-250.
  7. T. J. O'shea, J. Corgan, and T. C. Clancy, "Convolutional radio modulation recognition networks," in Proc. International conference on engineering applications of neural networks, 2016, pp. 213-226.