• Title, Summary, Keyword: Hyperparameter

Search Result 28, Processing Time 0.031 seconds

Method that determining the Hyperparameter of CNN using HS algorithm (HS 알고리즘을 이용한 CNN의 Hyperparameter 결정 기법)

  • Lee, Woo-Young;Ko, Kwang-Eun;Geem, Zong-Woo;Sim, Kwee-Bo
    • Journal of Korean Institute of Intelligent Systems
    • /
    • v.27 no.1
    • /
    • pp.22-28
    • /
    • 2017
  • The Convolutional Neural Network(CNN) can be divided into two stages: feature extraction and classification. The hyperparameters such as kernel size, number of channels, and stride in the feature extraction step affect the overall performance of CNN as well as determining the structure of CNN. In this paper, we propose a method to optimize the hyperparameter in CNN feature extraction stage using Parameter-Setting-Free Harmony Search (PSF-HS) algorithm. After setting the overall structure of CNN, hyperparameter was set as a variable and the hyperparameter was optimized by applying PSF-HS algorithm. The simulation was conducted using MATLAB, and CNN learned and tested using mnist data. We update the parameters for a total of 500 times, and it is confirmed that the structure with the highest accuracy among the CNN structures obtained by the proposed method classifies the mnist data with an accuracy of 99.28%.

Analysis of Open-Source Hyperparameter Optimization Software Trends

  • Lee, Yo-Seob;Moon, Phil-Joo
    • International Journal of Advanced Culture Technology
    • /
    • v.7 no.4
    • /
    • pp.56-62
    • /
    • 2019
  • Recently, research using artificial neural networks has further expanded the field of neural network optimization and automatic structuring from improving inference accuracy. The performance of the machine learning algorithm depends on how the hyperparameters are configured. Open-source hyperparameter optimization software can be an important step forward in improving the performance of machine learning algorithms. In this paper, we review open-source hyperparameter optimization softwares.

The Effect of Hyperparameter Choice on ReLU and SELU Activation Function

  • Kevin, Pratama;Kang, Dae-Ki
    • International journal of advanced smart convergence
    • /
    • v.6 no.4
    • /
    • pp.73-79
    • /
    • 2017
  • The Convolutional Neural Network (CNN) has shown an excellent performance in computer vision task. Applications of CNN include image classification, object detection in images, autonomous driving, etc. This paper will evaluate the performance of CNN model with ReLU and SELU as activation function. The evaluation will be performed on four different choices of hyperparameter which are initialization method, network configuration, optimization technique, and regularization. We did experiment on each choice of hyperparameter and show how it influences the network convergence and test accuracy. In this experiment, we also discover performance improvement when using SELU as activation function over ReLU.

Outlier Robust Learning Algorithm for Gaussian Process Classification (가우시안 과정 분류를 위한 극단치에 강인한 학습 알고리즘)

  • Kim, Hyun-Chul;Ghahramani, Zoubin
    • Proceedings of the Korean Information Science Society Conference
    • /
    • /
    • pp.485-489
    • /
    • 2007
  • Gaussian process classifiers (GPCs) are fully statistical kernel classification models which have a latent function with Gaussian process prior Recently, EP approximation method has been proposed to infer the posterior over the latent function. It can have a special hyperparameter which can treat outliers potentially. In this paper, we propose the outlier robust algorithm which alternates EP and the hyperparameter updating until convergence. We also show its usefulness with the simulation results.

  • PDF

A Study on Prediction of Attendance in Korean Baseball League Using Artificial Neural Network (인경신경망을 이용한 한국프로야구 관중 수요 예측에 관한 연구)

  • Park, Jinuk;Park, Sanghyun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.12
    • /
    • pp.565-572
    • /
    • 2017
  • Traditional method for time series analysis, autoregressive integrated moving average (ARIMA) allows to mine significant patterns from the past observations using autocorrelation and to forecast future sequences. However, Korean baseball games do not have regular intervals to analyze relationship among the past attendance observations. To address this issue, we propose artificial neural network (ANN) based attendance prediction model using various measures including performance, team characteristics and social influences. We optimized ANNs using grid search to construct optimal model for regression problem. The evaluation shows that the optimal and ensemble model outperform the baseline model, linear regression model.

LSTM Hyperparameter Optimization for an EEG-Based Efficient Emotion Classification in BCI (BCI에서 EEG 기반 효율적인 감정 분류를 위한 LSTM 하이퍼파라미터 최적화)

  • Aliyu, Ibrahim;Mahmood, Raja Majid;Lim, Chang-Gyoon
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.14 no.6
    • /
    • pp.1171-1180
    • /
    • 2019
  • Emotion is a psycho-physiological process that plays an important role in human interactions. Affective computing is centered on the development of human-aware artificial intelligence that can understand and regulate emotions. This field of study is also critical as mental diseases such as depression, autism, attention deficit hyperactivity disorder, and game addiction are associated with emotion. Despite the efforts in emotions recognition and emotion detection from nonstationary, detecting emotions from abnormal EEG signals requires sophisticated learning algorithms because they require a high level of abstraction. In this paper, we investigated LSTM hyperparameters for an optimal emotion EEG classification. Results of several experiments are hereby presented. From the results, optimal LSTM hyperparameter configuration was achieved.

Semiparametric Kernel Poisson Regression for Longitudinal Count Data

  • Hwang, Chang-Ha;Shim, Joo-Yong
    • Communications for Statistical Applications and Methods
    • /
    • v.15 no.6
    • /
    • pp.1003-1011
    • /
    • 2008
  • Mixed-effect Poisson regression models are widely used for analysis of correlated count data such as those found in longitudinal studies. In this paper, we consider kernel extensions with semiparametric fixed effects and parametric random effects. The estimation is through the penalized likelihood method based on kernel trick and our focus is on the efficient computation and the effective hyperparameter selection. For the selection of hyperparameters, cross-validation techniques are employed. Examples illustrating usage and features of the proposed method are provided.

Hyperparameter Selection for APC-ECOC

  • Seok, Kyung-Ha
    • Journal of the Korean Data and Information Science Society
    • /
    • v.19 no.4
    • /
    • pp.1219-1231
    • /
    • 2008
  • The main object of this paper is to develop a leave-one-out(LOO) bound of all pairwise comparison error correcting output codes (APC-ECOC). To avoid using classifiers whose corresponding target values are 0 in APC-ECOC and requiring pilot estimates we developed a bound based on mean misclassification probability(MMP). It can be used to tune kernel hyperparameters. Our empirical experiment using kernel mean squared estimate(KMSE) as the binary classifier indicates that the bound leads to good estimates of kernel hyperparameters.

  • PDF

Mixed Effects Kernel Binomial Regression

  • Hwang, Chang-Ha
    • Journal of the Korean Data and Information Science Society
    • /
    • v.19 no.4
    • /
    • pp.1327-1334
    • /
    • 2008
  • Mixed effect binomial regression models are widely used for analysis of correlated count data in which the response is the result of a series of one of two possible disjoint outcomes. In this paper, we consider kernel extensions with nonparametric fixed effects and parametric random effects. The estimation is through the penalized likelihood method based on kernel trick, and our focus is on the efficient computation and the effective hyperparameter selection. For the selection of hyperparameters, cross-validation techniques are employed. Examples illustrating usage and features of the proposed method are provided.

  • PDF

COMPARATIVE ANALYSIS ON MACHINE LEARNING MODELS FOR PREDICTING KOSPI200 INDEX RETURNS

  • Gu, Bonsang;Song, Joonhyuk
    • The Pure and Applied Mathematics
    • /
    • v.24 no.4
    • /
    • pp.211-226
    • /
    • 2017
  • In this paper, machine learning models employed in various fields are discussed and applied to KOSPI200 stock index return forecasting. The results of hyperparameter analysis of the machine learning models are also reported and practical methods for each model are presented. As a result of the analysis, Support Vector Machine and Artificial Neural Network showed a better performance than k-Nearest Neighbor and Random Forest.