• Proceedings of the IEEK Conference
• /
• 1999.06a
• /
• pp.482-485
• /
• 1999

In this paper we proposed a heterogeneous hidden layer consisting of both sigmoid functions and RBFs(Radial Basis Function) in multi-layered neural networks. Focusing on the orthogonal relationship between the sigmoid function and its derivative, a derived RBF that is a derivative of the sigmoid function is used as the RBF in the neural network. so the proposed neural network is called ONN(Orthogonal Neural Network). Identification results using a nonlinear function confirm both the ONN＇s feasibility and characteristics by comparing with those obtained using a conventional neural network which has sigmoid function or RBF in hidden layer

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2023.01a
• /
• pp.25-26
• /
• 2023

본 논문에서는 OpenAI Gym 환경에서 제공하는 CartPole-v1에 대해 강화학습을 통해 에이전트를 학습시키고, 학습에 적용되는 활성화함수의 성능을 비교분석하였다. 본 논문에서 적용한 활성화함수는 Sigmoid, ReLU, ReakyReLU 그리고 softplus 함수이며, 각 활성화함수를 DQN(Deep Q-Networks) 강화학습에 적용했을 때 보상 값을 비교하였다. 실험결과 ReLU 활성화함수를 적용하였을 때의 보상이 가장 높은 것을 알 수 있었다.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.7 no.4
• /
• pp.74-81
• /
• 1997

This paper proposes an evolutionary learning algorithm to discipline the projection neural nctworks (PNNs) with special type of hidden nodes which can activate radial basis functions as well as sigmoid functions. The proposed algorithm not only trains the parameters and the connection weights hut also c~ptimizes the network structure. Through the structure optimization, the number of hidden node:; necessary to represent a given target function is determined and the role of each hidden node is decided whether it activates a radial basis function or a sigmoid function. To apply the algorithm, PNN is realized by a self-organizing genotype representation with a linked list data structure. Simulations show that the algorithm can build the PNN with less hidden nodes than thc existing learning algorithm using error hack propagation(EE3P) and network growing strategy.

• Journal of KIISE:Software and Applications
• /
• v.31 no.11
• /
• pp.1474-1482
• /
• 2004

In this paper, we propose an approach for extracting and filtering block motion vectors using an adaptive weight function. We first extract motion vectors from a sequence of images by using size-varibale block matching and then process them by adaptive robust estimation to filter out outliers (motion vectors out of concern). The proposed adaptive robust estimation defines a continuous sigmoid weight function. It then adaptively tunes the sigmoid function to its hard-limit as the residual errors between the model and input data are decreased, so that we can effectively separate non-outliers (motion vectors of concern) from outliers with the finally tuned hard-limit of the weight function. The experimental results show that the suggested approach is very effective in filtering block motion vectors.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2023.05a
• /
• pp.520-520
• /
• 2023

제주도 동부 중산간 지역은 화산암으로 구성된 지하지질로 인해 지하수위의 변동폭이 크고 변동양상이 복잡하여 인공신경망(Artificial Neural Network, ANN) 모델 등을 활용한 지하수위의 예측이 어렵다. ANN에 적용되는 활성화함수에 따라 지하수의 예측성능은 달라질 수 있으므로 활성화함수의 비교분석 후 적절한 활성화함수의 사용이 반드시 필요하다. 본 연구에서는 5개 활성화함수(sigmoid, hyperbolic tangent(tanh), Rectified Linear Unit(ReLU), Leaky Rectified Linear Unit(Leaky ReLU), Exponential Linear Unit(ELU))를 제주도 동부 중산간지역에 위치한 2개 지하수 관정에 대해 비교분석하여 최적 활성화함수 도출을 목표로 한다. 또한 최적 활성화함수를 활용한 ANN의 적용성을 평가하기 위해 최근 널리 사용되고 있는 순환신경망 모델인 Long Short-Term Memory(LSTM) 모델과 비교분석 하였다. 그 결과, 2개 관정 중 지하수위 변동폭이 상대적으로 큰 관정은 ELU 함수, 상대적으로 작은 관정은 Leaky ReLU 함수가 지하수위 예측에 적절하였다. 예측성능이 가장 낮은 활성화함수는 sigmoid 함수로 나타나 첨두 및 최저 지하수위 예측 시 사용을 지양해야 할 것으로 판단된다. 도출된 최적 활성화함수를 사용한 ANN-ELU 모델 및 ANN-Leaky ReLU 모델을 LSTM 모델과 비교분석한 결과 대등한 지하수위 예측성능을 나타내었다. 이것은 feed-forward 방식인 ANN 모델을 사용하더라도 적절한 활성화함수를 사용하면 최신 순환신경망과 대등한 결과를 도출하여 활용 가능성이 충분히 있다는 것을 의미한다. 마지막으로 LSTM 모델은 가장 적절한 예측성능을 나타내어 다양한 인공지능 모델의 예측성능 비교를 위한 기준이 되는 참고모델로 활용 가능하다. 본 연구에서 제시한 방법은 지하수위 예측과 더불어 하천수위 예측 등 다양한 시계열예측 및 분석연구에 유용하게 사용될 수 있다.

• Journal of Intelligence and Information Systems
• /
• v.10 no.2
• /
• pp.53-63
• /
• 2004

Recurrent-Cascade-Correlation(RCC) is a supervised teaming algorithm that automatically determines the size and topology of the network. RCC adds new hidden neurons one by one and creates a multi-layer structure in which each hidden layer has only one neuron. By second order RCC, new hidden neurons are added to only one hidden layer. These created neurons are not connected to each other. We present a generalization of the RCC Architecture by combining the standard RCC Architecture and the second order RCC Architecture. Whenever a hidden neuron has to be added, the new RCC teaming algorithm automatically determines whether the network topology grows vertically or horizontally. This new algorithm using sigmoid, tanh and new activation functions was tested with the morse-benchmark-problem. Therefore we recognized that the number of hidden neurons was decreased by the experiments of the RCC network generalization which used the activation functions.

• The Journal of Engineering Geology
• /
• v.33 no.1
• /
• pp.137-149
• /
• 2023

In order to intuitively grasp the earth pressure direction acting on the structure and displacement state, displacement data in the horizontal and vertical directions were processed using the sigmoid function. A displacement coordinate system was set up for each axis. The system can intuitively check the current displacement and assess the management stage of each point. A displacement path can be compiled from continuously recorded points, allowing trends in the displacement's history and stress direction to be known. Analysis of data measured for excavated ground, found that displacement occurred in the direction of destressing at all points, and that the points' management state steady. Similar behavior trends were found among measurement points with high spatial correlation, whereas differing behavior trends occurred among measurement points with low spatial correlation. If the correlation analysis of the precursor and behavior area is performed using the continuously distributed surface settlement data and displacement coordinate system, it will be possible to predict the failure time and area.

• Journal of Korea Society of Digital Industry and Information Management
• /
• v.19 no.4
• /
• pp.137-146
• /
• 2023

The dropout of university freshmen is a very important issue in the financial problems of universities. Moreover, the dropout rate is one of the important indicators among the external evaluation items of universities. Therefore, universities need to predict dropout students in advance and apply various dropout prevention programs targeting them. This paper proposes a method to predict such dropout students in advance. This paper is about a method for predicting dropout students. It proposes a method to select dropouts by applying logistic regression using a shift sigmoid classification function using only quantitative data from the first semester of the first year, which most universities have. It is based on logistic regression and can select the number of prediction subjects and prediction accuracy by using the shift sigmoid function as an classification function. As a result of the experiment, when the proposed algorithm was applied, the number of predicted dropout subjects varied from 100% to 20% compared to the actual number of dropout subjects, and it was found to have a prediction accuracy of 75% to 98%.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.16 no.2
• /
• pp.287-292
• /
• 2021

In this paper, we propose an efficient CNN application method by analyzing the effect of activation function on the failure diagnosis of the inductive motor stator. Generally, the main purpose of the inductive motor stator failure diagnosis is to prevent the failure by rapidly diagnosing the minute turn short. In the application of activation function, experiments show that the Sigmoid function is 23.23% more useful in accuracy of diagnosis than the ReLu function, although it is shown that ReLu has superiority in overall fixer failure in utilizing the activation function.

• Journal of Korea Water Resources Association
• /
• v.54 no.spc1
• /
• pp.1143-1154
• /
• 2021

The selection of activation function has a great influence on the groundwater level prediction performance of artificial neural network (ANN) model. In this study, five activation functions were applied to ANN model for two groundwater level observation wells in the middle mountainous area of the Pyoseon watershed in Jeju Island. The results of the prediction of the groundwater level were compared and analyzed, and the optimal activation function was derived. In addition, the results of LSTM model, which is a widely used recurrent neural network model, were compared and analyzed with the results of the ANN models with each activation function. As a result, ELU and Leaky ReLU functions were derived as the optimal activation functions for the prediction of the groundwater level for observation well with relatively large fluctuations in groundwater level and for observation well with relatively small fluctuations, respectively. On the other hand, sigmoid function had the lowest predictive performance among the five activation functions for training period, and produced inappropriate results in peak and lowest groundwater level prediction. The ANN-ELU and ANN-Leaky ReLU models showed groundwater level prediction performance comparable to that of the LSTM model, and thus had sufficient potential for application. The methods and results of this study can be usefully used in other studies.