• 제어로봇시스템학회:학술대회논문집
• /
• 2005.06a
• /
• pp.941-946
• /
• 2005

In this paper an active random noise control using adaptive learning rate neural networks is presented. The adaptive learning rate strategy increases the learning rate by a small constant if the current partial derivative of the objective function with respect to the weight and the exponential average of the previous derivatives have the same sign, otherwise the learning rate is decreased by a proportion of its value. The use of an adaptive learning rate attempts to keep the learning step size as large as possible without leading to oscillation. It is expected that a cost function minimize rapidly and training time is decreased. Numerical simulations and experiments of active random noise control with the transfer function of the error path will be performed, to validate the convergence properties of the adaptive learning rate Neural Networks. Control results show that adaptive learning rate Neural Networks control structure can outperform linear controllers and conventional neural network controller for the active random noise control.

• Phonetics and Speech Sciences
• /
• v.3 no.3
• /
• pp.91-98
• /
• 2011

In this paper, a multi-channel speech enhancement method using the linearly constrained minimum variance (LCMV) algorithm and a variable learning rate control is proposed. To control the learning rate for adaptive filters of the LCMV algorithm, the direction of arrival (DOA) is measured for each short-time input signal and the likelihood function of the target speech presence is estimated to control the filter learning rate. Using the likelihood measure, the learning rate is increased during the pure noise interval and decreased during the target speech interval. To optimize the parameter of the mapping function between the likelihood value and the corresponding learning rate, an exhaustive search is performed using the Bark's scale distortion (BSD) as the performance index. Experimental results show that the proposed algorithm outperforms the conventional LCMV with fixed learning rate in the BSD by around 1.5 dB.

• International Journal of Computer Science & Network Security
• /
• v.21 no.1
• /
• pp.220-225
• /
• 2021

Human population growth rate is an important parameter for real-world planning. Common approaches rely upon fixed parameters like human population, mortality rate, fertility rate, which is collected historically to determine the region's population growth rate. Literature does not provide a solution for areas with no historical knowledge. In such areas, machine learning can solve the problem, but a multitude of machine learning algorithm makes it difficult to determine the best approach. Further, the missing feature is a common real-world problem. Thus, it is essential to compare and select the machine learning techniques which provide the best and most robust in the presence of missing features. This study compares 17 machine learning techniques (base learners and ensemble learners) performance in predicting the human population growth rate of the country. Among the 17 machine learning techniques, random forest outperformed all the other techniques both in predictive performance and robustness towards missing features. Thus, the study successfully demonstrates and compares machine learning techniques to predict the human population growth rate in settings where historical data and feature information is not available. Further, the study provides the best machine learning algorithm for performing population growth rate prediction.

• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.20 no.43
• /
• pp.175-195
• /
• 1997

The learning curve has an important effect the growth of corporation. But, in Korea, the study and inference on the learning rate of each industry are unprepared, and so, Korean industires have difficult in productivity and cost. At this point, this study infers the learning rate of the oil industries and investigates the productivity and growth of them. In conclusion, this study presents the direction of the oil industries' development. With the intention of this objects, this study seizes the status which is concerned the total quantity, the operating rate, the plant capacity, the indicators concerning productivity, the investment of R & D and the scales, and then, infers and verifies the relevancy in connection with the learning rate. In the oil industry, the average rate of learning is 65.96% from 1982 to 1994 which the total quantity and the average operation time are used to infer the rate. To observe the low rate within a same period of time, this study takes the consequences that the learning rate is almost indentical with them each year. This steady state is caused by a difference between the employee and the decision maker about the acquirement and assimiliated of technology. When the high-quality technologies posses the environment to applicate in the scene of labor with them, this technology applies to the productivities. As the learning rate increases, the productivity has more effectiveness. The result of analysis about the effectiveness of the learning rate follows that the R & D unfoldes to exist and does not contribute to the growth of the oil industry. To analyze the variables of the growth, such as the learning rate, the investement of R & D, the operating rate and the gross value added to property, plant and equipment, the model is established and examined. The business strategy in the oil industry must be developed to achive the internal growth as well as the external.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.6 no.2
• /
• pp.52-56
• /
• 1996

In trainir~ga back-propagation neural network, the learning speed of the network is greatly affected by its learning rate. Most of off-line fashioned learning-rate selection methods, however, are empirical except for some deterministic methods. It is very tedious and difficult to find a good learning rate using the empirical methods. The deterministic methods cannot guarantee the quality of the quality of the learning rate. This paper proposes a new learning-rate selection method. Our off-line fashioned method selects a good learning rate through stochastically searching process using evolutionary programming. The simulation results show that the learning speed achieved by our method is superior to that of deterministic and empirical methods.

• Journal of information and communication convergence engineering
• /
• v.1 no.3
• /
• pp.157-162
• /
• 2003

We propose a learning method for improving the performance of the backpropagation algorithm. The proposed method is using a fuzzy logic system for automatic tuning of the learning rate of each weight. Instead of choosing a fixed learning rate, the fuzzy logic system is used to dynamically adjust the learning rate. The inputs of fuzzy logic system are delta and delta bar, and the output of fuzzy logic system is the learning rate. In order to verify the effectiveness of the proposed method, we performed simulations on the XOR problem, character classification, and function approximation. The results show that the proposed method considerably improves the performance compared to the general backpropagation, the backpropagation with momentum, and the Jacobs'delta-bar-delta algorithm.

• Journal of Intelligence and Information Systems
• /
• v.25 no.4
• /
• pp.53-65
• /
• 2019

Object tracking is one of important steps to achieve video-based surveillance systems. Object tracking is considered as an essential task similar to object detection and recognition. In order to perform object tracking, various machine learning methods (e.g., least-squares, perceptron and support vector machine) can be applied for different designs of tracking systems. In general, generative methods (e.g., principal component analysis) were utilized due to its simplicity and effectiveness. However, the generative methods were only focused on modeling the target object. Due to this limitation, discriminative methods (e.g., binary classification) were adopted to distinguish the target object and the background. Among the machine learning methods for binary classification, total error rate minimization can be used as one of successful machine learning methods for binary classification. The total error rate minimization can achieve a global minimum due to a quadratic approximation to a step function while other methods (e.g., support vector machine) seek local minima using nonlinear functions (e.g., hinge loss function). Due to this quadratic approximation, the total error rate minimization could obtain appropriate properties in solving optimization problems for binary classification. However, this total error rate minimization was based on a batch mode setting. The batch mode setting can be limited to several applications under offline learning. Due to limited computing resources, offline learning could not handle large scale data sets. Compared to offline learning, online learning can update its solution without storing all training samples in learning process. Due to increment of large scale data sets, online learning becomes one of essential properties for various applications. Since object tracking needs to handle data samples in real time, online learning based total error rate minimization methods are necessary to efficiently address object tracking problems. Due to the need of the online learning, an online learning based total error rate minimization method was developed. However, an approximately reweighted technique was developed. Although the approximation technique is utilized, this online version of the total error rate minimization could achieve good performances in biometric applications. However, this method is assumed that the total error rate minimization can be asymptotically achieved when only the number of training samples is infinite. Although there is the assumption to achieve the total error rate minimization, the approximation issue can continuously accumulate learning errors according to increment of training samples. Due to this reason, the approximated online learning solution can then lead a wrong solution. The wrong solution can make significant errors when it is applied to surveillance systems. In this paper, we propose an exactly reweighted technique to recursively update the solution of the total error rate minimization in online learning manner. Compared to the approximately reweighted online total error rate minimization, an exactly reweighted online total error rate minimization is achieved. The proposed exact online learning method based on the total error rate minimization is then applied to object tracking problems. In our object tracking system, particle filtering is adopted. In particle filtering, our observation model is consisted of both generative and discriminative methods to leverage the advantages between generative and discriminative properties. In our experiments, our proposed object tracking system achieves promising performances on 8 public video sequences over competing object tracking systems. The paired t-test is also reported to evaluate its quality of the results. Our proposed online learning method can be extended under the deep learning architecture which can cover the shallow and deep networks. Moreover, online learning methods, that need the exact reweighting process, can use our proposed reweighting technique. In addition to object tracking, the proposed online learning method can be easily applied to object detection and recognition. Therefore, our proposed methods can contribute to online learning community and object tracking, detection and recognition communities.

• Korea Trade Review
• /
• v.46 no.6
• /
• pp.191-209
• /
• 2021

In this paper, various time series analysis models and machine learning models are presented for long-term prediction of export growth rate, and the prediction performance is compared and reviewed by RMSE and MAE. Export growth rate is one of the major economic indicators to evaluate the economic status. And It is also used to predict economic forecast. The export growth rate may have a negative (-) value as well as a positive (+) value. Therefore, Instead of using the ReLU function, which is often used for time series prediction of deep learning models, the PReLU function, which can have a negative (-) value as an output value, was used as the activation function of deep learning models. The time series prediction performance of each model for three types of data was compared and reviewed. The forecast data of long-term prediction of export growth rate was deduced by three forecast methods such as a fixed forecast method, a recursive forecast method and a rolling forecast method. As a result of the forecast, the traditional time series analysis model, ARDL, showed excellent performance, but as the time period of learning data increases, the performance of machine learning models including LSTM was relatively improved.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.39 no.4
• /
• pp.19-27
• /
• 2002

We proposed an auto-tuning method of learning rate for performance improvement of backpropagation algorithm. Proposed method is used a fuzzy logic system for automatic tuning of learning rate. Instead of choosing a fixed learning rate, the fuzzy logic system is used to dynamically adjust learning rate. The inputs of fuzzy logic system are ${\Delta}$ and $\bar{{\Delta}}$, and the output is the learning rate. In order to verify the effectiveness of the proposed method, we performed simulations on a N-parity problem, function approximation, and Arabic numerals classification. The results show that the proposed method has considerably improved the performance compared to the backpropagation, the backpropagation with momentum, and the Jacobs' delta-bar-delta.

• Proceedings of the IEEK Conference
• /
• 2003.07d
• /
• pp.1239-1242
• /
• 2003

The neural networks are a famous model to learn the nonlinear function or nonlinear system. The main point of neural network is that the difference actual output from desired output is used to update weights. Usually, the gradient descent method is used for the learning process. On training process, if learning rate is too large, neural networks hardly guarantee convergence of neural networks. On the other hand, if learning rate is too small, the training spends much time. Therefore, one major problem in use of neural networks are to decrease the teaming time while neural networks are guaranteed convergence. In this paper, we suggest the model of fuzzy logic to neural networks to calibrate learning rate. This method is to tune learning rate dynamically according to error and demonstrates the optimization of training.