• Communications for Statistical Applications and Methods
• /
• v.28 no.1
• /
• pp.21-37
• /
• 2021

There is a direct connection between linear discriminant analysis (LDA) and linear regression since the direction vector of the LDA can be obtained by the least square estimation. The connection motivates the penalized LDA when the model is high-dimensional where the number of predictive variables is larger than the sample size. In this paper, we study the penalized LDA for a class of penalties, called the moderately clipped LASSO (MCL), which interpolates between the least absolute shrinkage and selection operator (LASSO) and minimax concave penalty. We prove that the MCL penalized LDA correctly identifies the sparsity of the Bayes direction vector with probability tending to one, which is supported by better finite sample performance than LASSO based on concrete numerical studies.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.20 no.4
• /
• pp.534-540
• /
• 2010

High dimensionality and highly correlated features are the major characteristics of hyperspectral data. Linear projections such as LDA and its variants have been used in extracting low-dimensional features from high-dimensional spectral data. Regularization of LDA has been introduced to alleviate the overfitting that often occurs in a small-sized training data set and leads to poor generalization performance. Among them, a smoothness regularized LDA seems to be effective in the feature extraction for hyperspectral data due to its capability of utilizing the high correlatedness. This paper studies the performance of the regularized LDA in hyperspectral data classification experimentally with varying conditions of the training data. In addition, a new dual smoothness regularized LDA is proposed and evaluated that makes use of both the spectral-domain and spatial-domain correlations between neighboring pixels.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.30 no.12C
• /
• pp.1162-1166
• /
• 2005

A new low dimensional feature representation technique is presented in this paper. Linear discriminant analysis is a popular feature extraction method. However, in the case of high dimensional data, the computational difficulty and the small sample size problem are often encountered. In order to solve these problems, we propose two dimensional direct LDA algorithm, which directly extracts the image scatter matrix from 2D image and uses Direct LDA algorithm for face recognition. The ORL face database is used to evaluate the performance of the proposed method. The experimental results indicate that the performance of the proposed method is superior to DLDA.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.9
• /
• pp.2991-3007
• /
• 2022

Two dimensional locality preserving projections (2D-LPP) is an improved algorithm of 2D image to solve the small sample size (SSS) problems which locality preserving projections (LPP) meets. It's able to find the low dimension manifold mapping that not only preserves local information but also detects manifold embedded in original data spaces. However, 2D-LPP is simple and elegant. So, inspired by the comparison experiments between two dimensional linear discriminant analysis (2D-LDA) and linear discriminant analysis (LDA) which indicated that matrix based methods don't always perform better even when training samples are limited, we surmise 2D-LPP may meet the same limitation as 2D-LDA and propose a novel matrix exponential method to enhance the performance of 2D-LPP. 2D-MELPP is equivalent to employing distance diffusion mapping to transform original images into a new space, and margins between labels are broadened, which is beneficial for solving classification problems. Nonetheless, the computational time complexity of 2D-MELPP is extremely high. In this paper, we replace some of matrix multiplications with multiple multiplications to save the memory cost and provide an efficient way for solving 2D-MELPP. We test it on public databases: random 3D data set, ORL, AR face database and Polyu Palmprint database and compare it with other 2D methods like 2D-LDA, 2D-LPP and 1D methods like LPP and exponential locality preserving projections (ELPP), finding it outperforms than others in recognition accuracy. We also compare different dimensions of projection vector and record the cost time on the ORL, AR face database and Polyu Palmprint database. The experiment results above proves that our advanced algorithm has a better performance on 3 independent public databases.

• Smart Media Journal
• /
• v.7 no.4
• /
• pp.90-98
• /
• 2018

A Multilinear LDA Method of Tensor Representation for ECG Signal Based Individual Identification Electrocardiogram signals, included in the cardiac electrical activity, are often analyzed and used for various purposes such as heart rate measurement, heartbeat rhythm test, heart abnormality diagnosis, emotion recognition and biometrics. The objective of this paper is to perform individual identification operation based on Multilinear Linear Discriminant Analysis (MLDA) with the tensor feature. The MLDA can solve dimensional aspects of classification problems in high-dimensional tensor, and correlated subspaces can be used to distinguish between different classes. In order to evaluate the performance, we used MPhysionet's MIT-BIH database. The experimental results on this database showed that the individual identification by MLDA outperformed that by PCA and LDA.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.15 no.9
• /
• pp.5446-5451
• /
• 2014

Because there are some mistakes by hand in processing electronic maps using a navigation terminal, this paper proposes an automatic offline recognition for traffic signs, which are considered ingredient navigation information. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA), which have been used widely in the field of 2D face recognition as computer vision and pattern recognition applications, was used to recognize traffic signs. First, using PCA, a high-dimensional 2D image data was projected to a low-dimensional feature vector. The LDA maximized the between scatter matrix and minimized the within scatter matrix using the low-dimensional feature vector obtained from PCA. The extracted traffic signs under a real-world road environment were recognized successfully with a 92.3% recognition rate using the 40 feature vectors created by the proposed algorithm.

• Proceedings of the IEEK Conference
• /
• 2002.07c
• /
• pp.1670-1673
• /
• 2002

For pattern recognition on high-dimensional data, such as images, the dimensionality reduction as a preprocessing is effective. By dimensionality reduction, we can (1) reduce storage capacity or amount of calculation, and (2) avoid "the curse of dimensionality" and improve classification performance. Popular tools for dimensionality reduction are Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and Independent Component Analysis (ICA) recently. Among them, only LDA takes the class labels into consideration. Nevertheless, it, has been reported that, the classification performance with ICA is better than that with LDA because LDA has restriction on the number of dimensions after reduction. To overcome this dilemma, we propose a new dimensionality reduction technique based on an information theoretic measure for difference of distribution. It takes the class labels into consideration and still it does not, have restriction on number of dimensions after reduction. Improvement of classification performance has been confirmed experimentally.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.44 no.3
• /
• pp.98-107
• /
• 2007

For high-dimensional pattern recognition, such as face classification, the small number of training samples leads to the Small Sample Size problem when the number of pattern samples is smaller than the number of dimensionality. Recently, various LDA-extensions have been developed, including LDA, PCA+LDA, and Direct-LDA, to address the problem. This paper proposes a method of improving the classification efficiency by increasing the number of (sub)-classes through pre-clustering a training set prior to the execution of Direct-LDA. In LDA (or Direct-LDA), since the number of classes of the training set puts a limit to the dimensionality to be reduced, it is increased to the number of sub-classes that is obtained through clustering so that the classification performance of LDA-extensions can be improved. In other words, the eigen space of the training set consists of the range space and the null space, and the dimensionality of the range space increases as the number of classes increases. Therefore, when constructing the transformation matrix, through minimizing the null space, the loss of discriminatve information resulted from this space can be minimized. Experimental results for the artificial data of X-OR samples as well as the bench mark face databases of AT&T and Yale demonstrate that the classification efficiency of the proposed method could be improved.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.16 no.3
• /
• pp.367-371
• /
• 2006

Multilayer percepteon has the advantage of learning their optimal parameters and efficiency. However, MLP shows some drawbacks when dealing with high dimensional data within the input space. Also, it Is very difficult to find the optimal parameters when the input data are highly correlated such as large scale face dataset. In this paper, we propose a novel technique for face recognition based on LDA and local MLP. To resolve the main drawback of MLP, we calculate the reduced features by LDA in advance. And then, we construct a local MLP per group consisting of subset of facedatabase to find its optimal learning parameters rather than using whole faces. Finally, we designed the face recognition system combined with the local MLPs. From various experiments, we obtained better classification performance in comparison with the results produced by conventional methods such as PCA and LDA.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.45 no.6
• /
• pp.64-69
• /
• 2008

This paper discusses low resolution face recognition using the photon-counting linear discriminant analysis (LDA). The photon-counting LDA asymptotically realizes the Fisher criterion without dimensionality reduction since it does not suffer from the singularity problem of the fisher LDA. The linear discriminant function for optimal projection is determined in high dimensional space to classify unknown objects, thus, it is more efficient in dealing with low resolution facial images as well as conventional face distortions. The simulation results show that the proposed method is superior to Eigen face and Fisher face in terms of the accuracy and false alarm rates.