• Bulletin of the Korean Mathematical Society
• /
• v.55 no.4
• /
• pp.1303-1315
• /
• 2018

For principal component analysis (PCA) to efficiently analyze large scale matrices, it is crucial to find a few singular vectors in cheaper computational cost and under lower memory requirement. To compute those in a fast and robust way, we propose a new stochastic method. Especially, we adopt the stochastic variance reduced gradient (SVRG) method [11] to avoid asymptotically slow convergence in stochastic gradient descent methods. For that purpose, we reformulate the PCA problem as a unconstrained optimization problem using a quadratic penalty. In general, increasing the penalty parameter to infinity is needed for the equivalence of the two problems. However, in this case, exact penalization is guaranteed by applying the analysis in [24]. We establish the convergence rate of the proposed method to a stationary point and numerical experiments illustrate the validity and efficiency of the proposed method.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.18 no.5
• /
• pp.1141-1162
• /
• 2024

Humidity is an important parameter in meteorology and is closely related to weather, human health, and the environment. Due to the limitations of the number of observation stations and other factors, humidity data are often not as good as expected, so high-resolution humidity fields are of great interest and have been the object of desire in the research field and industry. This study presents a novel super-resolution algorithm for humidity fields based on the Wasserstein generative adversarial network(WGAN) framework, with the objective of enhancing the resolution of low-resolution humidity field information. WGAN is a more stable generative adversarial networks(GANs) with Wasserstein metric, and to make the training more stable and simple, the gradient cropping is replaced with gradient penalty, and the network feature representation is improved by sub-pixel convolution, residual block combined with convolutional block attention module(CBAM) and other techniques. We evaluate the proposed algorithm using ERA5 relative humidity data with an hourly resolution of 0.25°×0.25°. Experimental results demonstrate that our approach outperforms not only conventional interpolation techniques, but also the super-resolution generative adversarial network(SRGAN) algorithm.

• Journal of the Architectural Institute of Korea Structure & Construction
• /
• v.33 no.12
• /
• pp.53-62
• /
• 2017

Many gradient-based mathematical methods have been developed and are in use for structural size optimization problems, in which the cross-sectional areas or sizing variables are usually assumed to be continuous. In most practical structural engineering design problems, however, the design variables are discrete. The main objective of this paper is to propose an efficient optimization method for structures with discrete-sized variables based on the harmony search (HS) meta-heuristic algorithm that is derived using penalty function. The recently developed HS algorithm was conceptualized using the musical process of searching for a perfect state of harmony. It uses a stochastic random search instead of a gradient search so that derivative information is unnecessary. In this paper, a discrete search strategy using the HS algorithm with a static penalty function is presented in detail and its applicability using several standard truss examples is discussed. The numerical results reveal that the HS algorithm with the static penalty function proposed in this study is a powerful search and design optimization technique for structures with discrete-sized members.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.31 no.8C
• /
• pp.786-794
• /
• 2006

We proposed a novel preconditioner based PCG(Preconditioned Conjugate Gradient) method for super resolution image reconstruction. Compared with the conventional block circulant type preconditioner, the proposed preconditioner can be more effectively applied for objective functions that include roughness penalty functions. The effectiveness of the proposed method was shown by simulations and experiments.

• Journal of Korea Multimedia Society
• /
• v.24 no.1
• /
• pp.13-20
• /
• 2021

A Wasserstein GAN(WGAN), optimum in terms of minimizing Wasserstein distance, still suffers from inconsistent convergence or unexpected output due to inherent learning instability. It is widely known some kinds of restriction on the discriminative function should be considered to solve such problems, which implies the importance of Lipschitz continuity. Unfortunately, there are few known methods to satisfactorily maintain the Lipschitz continuity of the discriminative function. In this paper we propose techniques to stably maintain the Lipschitz continuity of the discriminative function by adding effective regularization terms to the objective function, which limit the magnitude of the gradient vectors of the discriminator to one or less. Extensive experiments are conducted to evaluate the performance of the proposed techniques, which shows the single-sided penalty improves convergence compared with the gradient penalty at the early learning process, while the proposed additional penalty increases inception scores by 0.18 after 100,000 number of learning.

• Communications for Statistical Applications and Methods
• /
• v.31 no.2
• /
• pp.203-212
• /
• 2024

The area under the ROC curve (AUC) is one of the most common criteria used to measure the overall performance of binary classifiers for a wide range of machine learning problems. In this article, we propose a L₁-penalized AUC-optimization classifier that directly maximizes the AUC for high-dimensional data. Toward this, we employ the AUC-consistent surrogate loss function and combine the L₁-norm penalty which enables us to estimate coefficients and select informative variables simultaneously. In addition, we develop an efficient optimization algorithm by adopting k-means clustering and proximal gradient descent which enjoys computational advantages to obtain solutions for the proposed method. Numerical simulation studies demonstrate that the proposed method shows promising performance in terms of prediction accuracy, variable selectivity, and computational costs.

• The Journal of the Acoustical Society of Korea
• /
• v.38 no.6
• /
• pp.670-677
• /
• 2019

As the presence of background noise in acoustic signal degrades the performance of speech or acoustic event recognition, it is still challenging to extract noise-robust acoustic features from noisy signal. In this paper, we propose a combined structure of Wasserstein Generative Adversarial Network (WGAN) and MultiTask AutoEncoder (MTAE) as deep learning architecture that integrates the strength of MTAE and WGAN respectively such that it estimates not only noise but also speech features from noisy acoustic source. The proposed MTAE-WGAN structure is used to estimate speech signal and the residual noise by employing a gradient penalty and a weight initialization method for Leaky Rectified Linear Unit (LReLU) and Parametric ReLU (PReLU). The proposed MTAE-WGAN structure with the adopted gradient penalty loss function enhances the speech features and subsequently achieve substantial Phoneme Error Rate (PER) improvements over the stand-alone Deep Denoising Autoencoder (DDAE), MTAE, Redundant Convolutional Encoder-Decoder (R-CED) and Recurrent MTAE (RMTAE) models for robust speech recognition.

• Transactions of Materials Processing
• /
• v.12 no.1
• /
• pp.26-33
• /
• 2003

This paper is concerned with the numerical analysis of frictional contact problems in axisymmetric bodies using the rigid-plastic finite element method. A contact finite element method, based on a penalty function, are derived from variational formulations. The contact boundary condition between two deformable bodies is prescribed by the proposed algorithm. The program which can handle frictional contact problem is developed by using pre-existing rigid-plastic finite element code. Some examples used in this paper illustrate the effectiveness of the proposed formulations and algorithms. Efforts focus on the deformation patterns, contact force, and velocity gradient through the various simulations.

• Proceedings of the KIEE Conference
• /
• 2004.07d
• /
• pp.2242-2244
• /
• 2004

This paper deals with the design of a reduced-order stabilizing controller for the linear system. The coupled lineal matrix inequality (LMI) problem subject to a rank condition is solved by a sequential semidefinite programming (SDP) approach. The nonconvex rank constraint is incorporated into a strictly linear penalty function, and the computation of the gradient and Hessian function for the Newton method is not required. The penalty factor and related term are updated iteratively. Therefore the overall procedure leads to a successive LMI relaxation method. Extensive numerical experiments illustrate the proposed algorithm.

• Journal of the Korean Society for Precision Engineering
• /
• v.17 no.5
• /
• pp.93-99
• /
• 2000

In many dynamic structural optimization problems, the goal is to reduce the total weight of the structure without causing the resonance. Up to now, gradient informations(i.e., design sensitivity) have been used to achieve the goal. For some class of dynamic problems, especially coalescent eigenvalue Problems with multiobjective optimization, the design sensitivity analysis is too much complicated mathematically and numerically. Therefore, this article proposes a new technique fur structural dynamic modification using a mode modification method with Genetic Algorithm(GA). In GA formulation, fitness is defined based on penalty function approach. Design variables are iteratively improved by using genetic algorithm. Two numerical examples are shown, (ⅰ) a cantilevered plate, and (ⅱ) H-shaped structure. The results demonstrate that the proposed method is highly efficient.