• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• 제24권4호
• /
• pp.331-350
• /
• 2020

Exploding gradient is a widely known problem in training recurrent neural networks. The explosion problem has often been coped with cutting off the gradient norm by some fixed value. However, this strategy, commonly referred to norm clipping, is an ad hoc approach to attenuate the explosion. In this research, we opt to view the problem from a different perspective, the discrete-time optimal control with infinite horizon for a better understanding of the problem. Through this perspective, we fathom the region at which gradient explosion occurs. Based on the analysis, we introduce a gradient-explosion-free algorithm that keeps the training process away from the region. Numerical tests show that this algorithm is at least three times faster than the clipping strategy.

• ETRI Journal
• /
• 제42권6호
• /
• pp.922-931
• /
• 2020

A hierarchical iterative algorithm for the canonical polyadic decomposition (CPD) of tensors is proposed by improving the traditional conjugate gradient least squares (CGLS) method. Methods based on algebraic operations are investigated with the objective of estimating the direction of arrival (DoA) and polarization parameters of signals impinging on an array with electromagnetic (EM) vector-sensors. The proposed algorithm adopts a hierarchical iterative strategy, which enables the algorithm to obtain a fast recovery for the highly collinear factor matrix. Moreover, considering the same accuracy threshold, the proposed algorithm can achieve faster convergence compared with the alternating least squares (ALS) algorithm wherein the highly collinear factor matrix is absent. The results reveal that the proposed algorithm can achieve better performance under the condition of fewer snapshots, compared with the ALS-based algorithm and the algorithm based on generalized eigenvalue decomposition (GEVD). Furthermore, with regard to an array with a small number of sensors, the observed advantage in estimating the DoA and polarization parameters of the signal is notable.

• ETRI Journal
• /
• 제28권1호
• /
• pp.59-66
• /
• 2006

Learning behaviors of a radial basis function network (RBFN) using a singular value decomposition (SVD) and stochastic gradient (SG) algorithm, together named RBF-SVD-SG, for odor sensing systems are analyzed, and a fast training method is proposed. RBF input data is from a conducting polymer sensor array. It is revealed in this paper that the SG algorithm for the fine-tuning of centers and widths still shows ill-behaving learning results when a sufficiently small convergence coefficient is not used. Since the tuning of centers in RBFN plays a dominant role in the performance of RBFN odor sensing systems, our analysis is focused on the center-gradient variance of the RBFN-SVD-SG algorithm. We found analytically that the steadystate weight fluctuation and large values of a convergence coefficient can lead to an increase in variance of the center-gradient estimate. Based on this analysis, we propose to use the least mean square algorithm instead of SVD in adjusting the weight for stable steady-state weight behavior. Experimental results of the proposed algorithm have shown faster learning speed and better classification performance.

• 한국지능시스템학회논문지
• /
• 제11권6호
• /
• pp.454-458
• /
• 2001

이 논문에서는 디지털 영상의 퍼지 시스템 표현으로부터 유도된 Edge 검출 알고리듬에 대하여 기술한다. 이 알고리듬은 Gradient을 기반으로 한 것으로 Convolution Kernel이 기존의 Roberts, Prewitt 또는 Sobel등이 제안한 Gradient Kernel과 다른 새로운 것이다. 사용한 퍼지시스템은 디지털 영상을 근사적으로 표현한 Bicubic Spline 함수를 퍼지시스템 화한것으로서 2차 도함수가 연속이기 때문에 Gradient나 Laplacian 연산이 가능하다. Grid 점들에서 이 함수의 Gradient는 두 개의 축 방향으로 각각 한개의 3$\times$3행렬과 영상과의 Covolution에 의하여 산출됨을 보였으며 이를 이용하여 검출된 Edge들은 기존의 다른 방법을 사용하여 검출된 Edge 영상보다 훨씬 선명함을 확인하였다. 이 알고리듬 적용사례 2개에 대한 기술에 포함되어 있다.

• 대한교통학회:학술대회논문집
• /
• 대한교통학회 1999년도 제36회 학술발표회논문집
• /
• pp.65-70
• /
• 1999

• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• 제8권2호
• /
• pp.7-13
• /
• 2004

A Conjugate Gradient (CG) algorithm for unconstrained minimization is proposed which is invariant to a nonlinear scaling of a strictly convex quadratic function and which generates mutually conjugate directions for extended quadratic functions. It is derived for inexact line searches and is designed for the minimization of general nonlinear functions. It compares favorably in numerical tests with the original Dixon algorithm on which the new algorithm is based.

• 대한전기학회:학술대회논문집
• /
• 대한전기학회 1998년도 하계학술대회 논문집 B
• /
• pp.515-518
• /
• 1998

This paper proposes a new hybrid algorithm for auto-tuning fuzzy controllers improving the performance. The presented algorithm estimates automatically the optimal values of membership functions, fuzzy rules, and scaling factors for fuzzy controllers, using a genetic-MGM algorithm. The object of the proposed algorithm is to promote search efficiency by a genetic and modified gradient optimization techniques. The proposed genetic and MGM algorithm is based on both the standard genetic algorithm and a gradient method. If a maximum point don't be changed around an optimal value at the end of performance during given generation, the genetic-MGM algorithm searches for an optimal value using the initial value which has maximum point by converting the genetic algorithms into the MGM(Modified Gradient Method) algorithms that reduced the number of variables. Using this algorithm is not only that the computing time is faster than genetic algorithm as reducing the number of variables, but also that can overcome the disadvantage of genetic algorithms. Simulation results verify the validity of the presented method.

• 디지털산업정보학회논문지
• /
• 제5권4호
• /
• pp.117-126
• /
• 2009

The recent interest in sensor networks has led to a number of routing schemes that use the limited resources available at sensor nodes more efficiently. These schemes typically try to find the minimum energy path to optimize energy usage at a node. Some schemes, however, are prone to unbalance of the traffic and energy. To solve this problem, we propose a novel solution: a gradient-field approach which takes account of the minimum cost data delivery, energy consumption balancing, and traffic equalization. We also modify the backoff-based cost field setup algorithm to establish our gradient-field based sensor network and give the algorithm. Simulation results show that the overhead of routing establishment obtained by our algorithm is much less than the one obtained by Flooding. What's more, our approach guarantees the basic Quality of Service (QoS) without extra spending.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제7권11호
• /
• pp.2754-2768
• /
• 2013

Video fingerprints generated from global features are usually vulnerable against general geometric transformations. In this paper, a novel video fingerprinting algorithm is proposed, in which a new spatio-temporal gradient is designed to represent the spatial and temporal information for each frame, and a new partition scheme, based on concentric circle and rings, is developed to resist the attacks efficiently. The centroids of spatio-temporal gradient orientations (CSTGO) within the circle and rings are then calculated to generate a robust fingerprint. Our experiments with different attacks have demonstrated that the proposed approach outperforms the state-of-the-art methods in terms of robustness and discrimination.

• 제어로봇시스템학회:학술대회논문집
• /
• 제어로봇시스템학회 2000년도 제15차 학술회의논문집
• /
• pp.373-373
• /
• 2000

In this paper, we propose a fuzzy modeling algorithm which divides the input space more efficiently than convention methods by taking into consideration correlations between components of sample data. The proposed fuzzy modeling algorithm consists of two steps: coarse tuning, which determines consequent parameters approximately using FCRM clustering method, and fine tuning, which adjusts the premise and consequent parameters more precisely by gradient descent algorithm. To evaluate the performance of the proposed fuzzy mode, we use the numerical data of nonlinear function.