• ETRI Journal
• /
• v.22 no.3
• /
• pp.10-18
• /
• 2000

Although an n-th order cross-entropy (nCE) error function resolves the incorrect saturation problem of conventional error backpropagation (EBP) algorithm, performance of multilayer perceptrons (MLPs) trained using the nCE function depends heavily on the order of nCE. In this paper, we propose an adaptive learning rate to markedly reduce the sensitivity of MLP performance to the order of nCE. Additionally, we propose to limit error signal values at out-put nodes for stable learning with the adaptive learning rate. Through simulations of handwritten digit recognition and isolated-word recognition tasks, it was verified that the proposed method successfully reduced the performance dependency of MLPs on the nCE order while maintaining advantages of the nCE function.

• Proceedings of the KIEE Conference
• /
• 2015.07a
• /
• pp.1365-1366
• /
• 2015

본 논문에서는 HOG 특징을 이용한 다항식 방사형 기저함수 신경회로망 기반 숫자 인식 시스템의 설계를 제안한다. 제안한 숫자 인식 시스템은 HOG 특징을 이용하여 숫자를 입력 데이터로 사용하기 위해 특징을 계산한다. 다항식 방사형 기저 함수 신경회로망은 고차원 데이터의 입-출력 형태를 갖는 클래스를 분류하는데 용이하며, 활성함수의 중심점 및 분포상수는 Fuzzy C-Means(FCM) 알고리즘에 의해 초기 값을 설정한다. 또한 제안한 분류기의 최적화를 위해 Particle Swarm Optimization(PSO)를 사용하여 최적화된 분류기의 성능을 비교한다. 숫자 인식을 위하여 공인 데이터베이스인 MNIST handwritten digit database를 사용하여 분류기의 성능을 평가하고 분석한다.

• Proceedings of the Korea Contents Association Conference
• /
• 2005.11a
• /
• pp.364-370
• /
• 2005

LBL(Layer-By-Layer) algorithms have been proposed to accelerate the training speed of MLPs(Multilayer Perceptrons). In this LBL algorithms, each layer needs a error function for optimization. Especially, error function for hidden layer has a great effect to achieve good performance. In this sense, this paper proposes a new hidden layer error function for improving the performance of LBL algorithm for MLPs. The hidden layer error function is derived from the mean squared error of output layer. Effectiveness of the proposed error function was demonstrated for a handwritten digit recognition and an isolated-word recognition tasks and very fast learning convergence was obtained.

• The Journal of the Korea Contents Association
• /
• v.4 no.3
• /
• pp.99-105
• /
• 2004

This paper proposes optimal learning rates in the gradient descent training of multilayer perceptrons, which are a separate learning rate for weights associated with each neuron and a separate one for assigning virtual hidden targets associated with each training pattern Effectiveness of the proposed error function was demonstrated for a handwritten digit recognition and an isolated-word recognition tasks and very fast learning convergence was obtained.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.24 no.9B
• /
• pp.1775-1784
• /
• 1999

This paper proposes a method of removing unnecessary hidden nodes by a new cost function that evaluates the variance and the mean of hidden node outputs during training. The proposed cost function makes necessary hidden nodes be activated and unnecessary hidden nodes be constants. We can remove the constant hidden nodes without performance degradation. Using the CEDAR handwritten digit recognition, we have shown that the proposed method can remove the number of hidden nodes up to 37.2%, with higher recognition rate and shorter learning time.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2004.11a
• /
• pp.505-508
• /
• 2004

본 논문은 동적 3-D 뉴로 시스템(A Dynamic 3-D Neuro System)모델을 이용하여 오프라인 필기체 숫자 인식 실험을 하였다. 3-D 뉴로 시스템 모델을 사용함으로써 기존에 교육된 정보를 유지하면서 새로운 정보를 추가할 수 있는 추가학습이 가능했고, 동일한 범주의 정보에 대해서는 반복교육 횟수에 따라 교육정도가 점점 누적되는 반복교육이 가능했다. 교육과정을 통해 누적된 정보로부터 일반화된 패턴(Generalized Pattern)을 도출해 인식시 사용할 수 있었다. 패턴 인식기는 피드백루틴을 통해 미지의 입력이미지를 원형이미지로 복원한 후, 그 결과 데이터를 사용하여 문자를 인식하도록 동작한다. NIST의 MNIST 데이터베이스를 사용해 실험을 하였고, 결과로 $99.0\%$의 정인식률을 얻었다.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.05a
• /
• pp.561-562
• /
• 2021

CNN (Convolutional Neural Network) is widely used to recognize various images. In this presentation, a single digit handwritten by humans was recognized by applying the CNN technique of deep learning. The deep learning network consists of a convolutional layer, a pooling layer, and a platen layer, and finally, we set an optimization method, learning rate and loss functions.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.10a
• /
• pp.397-399
• /
• 2022

Digit recognition is one of the applications of deep learning, which appears in many fields. CNN network enables us to recognize handwritten digits. Also, It can process various types of data. As we stack more layers in CNN network, we expect more performance improvements. In this paper, we added a convolution layer. as a result, we achieved an accuracy improvement from 76.96% to 98.87%, which is a nearly 21.81% increase.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.25 no.4
• /
• pp.355-360
• /
• 2015

In this paper, we propose the design of Radial Basis Function Neural Network based on PCA in order to recognize handwritten digits. The proposed pattern classifier consists of the preprocessing step of PCA and the pattern classification step of pRBFNNs. In the preprocessing step, Feature data is obtained through preprocessing step of PCA for minimizing the information loss of given data and then this data is used as input data to pRBFNNs. The hidden layer of the proposed classifier is built up by Fuzzy C-Means(FCM) clustering algorithm and the connection weights are defined as linear polynomial function. In the output layer, polynomial parameters are obtained by using Least Square Estimation (LSE). MNIST database known as one of the benchmark handwritten dataset is applied for the performance evaluation of the proposed classifier. The experimental results of the proposed system are compared with other existing classifiers.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.12 no.3
• /
• pp.201-207
• /
• 2012

A RAM-based neural network is a weightless neural network based on binary neural network(BNN) which is efficient neural network with a one-shot learning. RAM-based neural network has multiful information bits and store counts of training in BNN. Supervised learning based on the RAM-based neural network has the excellent performance in pattern recognition but in pattern categorization with unsupervised learning as unsuitable. In this paper, we propose a unsupervised learning algorithm in the RAM-based neural network to perform pattern categorization. By the proposed unsupervised learning algorithm, RAM-based neural network create categories depending on the input pattern by itself. Therefore, RAM-based neural network for supervised learning and unsupervised learning should proof of all possible complex models. The training data for experiments provided by the MNIST offline handwritten digits which is consist of 0 to 9 multi-pattern.