• Korean Journal of Mathematics
• /
• v.30 no.1
• /
• pp.131-146
• /
• 2022

This is an expository article about the gradients in deep neural network. It is hard to find a place where gradients in a deep neural network are dealt in details in a systematic and mathematical way. We review and compute the gradients and Jacobians to derive formulas for gradients which appear in the backpropagation and implement them in vectorized forms in Python.

• Journal of the Korea Society of Computer and Information
• /
• v.16 no.1
• /
• pp.31-38
• /
• 2011

A RAM-based Neural Net is a weightless neural network based on binary neural network. 3-D neural network using this paper is binary neural network with multiful information bits and store counts of training. Recognition method by MRD technique is based on the supervised learning. Therefore neural network by itself can not distinguish between the categories and well-separated categories of training data can achieve only through the performance. In this paper, unsupervised learning algorithm is proposed which is trained existing 3-D neural network without distinction of data, to distinguish between categories depending on the only input training patterns. The training data for proposed unsupervised learning provided by the NIST handwritten digits of MNIST which is consist of 0 to 9 multi-pattern, a randomly materials are used as training patterns. Through experiments, neural network is to determine the number of discriminator which each have an idea of the handwritten digits that can be interpreted.

• Journal of the Society of Naval Architects of Korea
• /
• v.34 no.3
• /
• pp.19-26
• /
• 1997

This paper is concerned with the design of neural-network based autopilot control system. The back-propagation neural network introduced in the previous paper by authors is applied to the autopilot control system. As a result, two neural-network controllers are developed, which are the model reference adaptive neural controller and the instantaneous optimal neural controller. The model reference adaptive neural controller is the control technique that the heading angle and angular velocity are controlled by the rudder angle to follow the output of the reference model. The instantaneous optimal neural controller optimizes the transition from one state to the next state. These control techniques are applied to a simple ship maneuvering model and their effectiveness is proved by numerical examples.

• Journal of the Korean Society of Industry Convergence
• /
• v.27 no.1
• /
• pp.151-160
• /
• 2024

Artificial Neural Networks that enabled Artificial Intelligence are being used in many fields. However, the application to mechanical structures has several problems and research is incomplete. One of the problems is that it is difficult to secure a large amount of data necessary for learning Artificial Neural Networks. In particular, it is important to detect and recognize external forces and forces for safety working and accident prevention of mechanical structures. This study examined the possibility by applying the Current Neural Network of Artificial Neural Networks to detect and recognize the load on the machine. Tens of thousands of data are required for general learning of Recurrent Neural Networks, and to secure large amounts of data, this paper derives load data from ANSYS structural analysis results and applies a stacked auto-encoder technique to secure the amount of data that can be learned. The usefulness of Stacked Auto-Encoder data was examined by comparing Stacked Auto-Encoder data and ANSYS data. In addition, in order to improve the accuracy of detection and recognition of load data with a Recurrent Neural Network, the optimal conditions are proposed by investigating the effects of related functions.

• The Journal of the Acoustical Society of Korea
• /
• v.18 no.3
• /
• pp.62-67
• /
• 1999

In this paper, we investigates a reliable model of the Predictive Recurrent Neural Network for the speech recognition. Predictive Neural Networks are modeled by syllable units. For the given input syllable, then a model which gives the minimum prediction error is taken as the recognition result. The Predictive Neural Network which has the structure of recurrent network was composed to give the dynamic feature of the speech pattern into the network. We have compared with the recognition ability of the Recurrent Network proposed by Elman and Jordan. ETRI's SAMDORI has been used for the speech DB. In order to find a reliable model of neural networks, the changes of two recognition rates were compared one another in conditions of: (1) changing prediction order and the number of hidden units: and (2) accumulating previous values with self-loop coefficient in its context. The result shows that the optimum prediction order, the number of hidden units, and self-loop coefficient have differently responded according to the structure of neural network used. However, in general, the Jordan's recurrent network shows relatively higher recognition rate than Elman's. The effects of recognition rate on the self-loop coefficient were variable according to the structures of neural network and their values.

• Proceedings of the KSPS conference
• /
• 1996.02a
• /
• pp.137-155
• /
• 1996

The purpose of this paper is to generate a pitch contour which is affected by tile phonetic environment and the number of syllables in each Korean isolated word using a neural network. To do this, we analyzed a set of 513 Korean isolated words, consisting of 1-4 syllables and extracted the pitch contour and the duration of each phoneme in all the words. The total number of phonemes we analyzed is about 3800. After that we approximated the pitch contour with a 1st order polynominal by a regression analysis. We could get the slope, the initial pitch and the duration of each phoneme. We used these 3 parameters as the target pattern of the neural network and let the neural network learn the rule of the variation of the pitch and duration, which was affected by the phonetic environment of each phoneme. We used 7 consecutive phoneme strings as an input pattern for a neural network to make the network learn the effect of phonetic environment around the center phoneme. In the learning phase, we used 3545 items(463 words) as target patterns which contained the phonetic environment of front and rear 3 phonemes and the neural network showed the correctness rate of 98.43%, 98.59%, 97.7% in the estimation of the duration, the slope, the initial pitch. In the recall phase, we tested the performance of tile neural network with 251 items(50 words) which weren't need as learning data and we could get the good correctness rate of 97.34%, 95.45%, 96.3% in the generation of the duration, the slope, and the initial pitch of each phoneme.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2002.05a
• /
• pp.227-230
• /
• 2002

The Loose Part Monitoring System(LPMS) has been designed to detect, locate and evaluate detached or loosened parts and foreign objects in the reactor coolant system. In this paper, at first, we presents an application of the back propagation neural network. At the preprocessing step, the moving window average filter is adopted to reject the low frequency background noise components. And then, extracting the acoustic signature such as Starting point of impact signal, Rising time, Half period, and Global time, they are used as the inputs to neural network. Secondly, we applied the neural network algorithm to LPMS in order to estimate the mass of loose parts. We trained the impact test data of YGN3 using the backpropagation method. The input parameter for training is Rising Time, Half Period, Maximum amplitude. The result showed that the neural network would be applied to LPMS. Also, applying the neural network to the Practical false alarm data during startup and impact test signal at nuclear power Plant, the false alarms are reduced effectively. 1.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.11 no.4
• /
• pp.354-358
• /
• 2001

This paper presents a fuzzy neural network model which solves the underutilization problem. This fuzzy neural network has both stability and flexibility because it uses the control structure similar to AHT(Adaptive Resonance Theory)-l neural network. And this fuzzy nenral network does not need to initialize weights and is less sensitive to noise than ART-l neural network is. The learning rule of this fuzzy neural network is the modified and fuzzified version of Kohonen learning rule and is based on the fuzzification of leaky competitive leaming and the fuzzification of conditional probability. The similarity measure of vigilance test, which is performed after selecting a winner among output neurons, is the relative distance. This relative distance considers Euclidean distance and the relative location between a datum and the prototypes of clusters. To compare the performance of the proposed fuzzy neural network with that of Kohonen Self-Organizing Feature Map the IRIS data and Gaussian-distributed data are used.

• 제어로봇시스템학회:학술대회논문집
• /
• 1999.10a
• /
• pp.214-217
• /
• 1999

In this paper, genetic algorithm (GA) is implemented to search for the optimal structures (i.e. the kind of neural networks, the number of inputs and hidden neurons) of neural networks which are used approximating a given nonlinear function. Two kinds of neural networks, i.e. the multilayer feedforward [1] and time delay neural networks (TDNN) [2] are involved in this paper. The synapse weights of each neural network in each generation are obtained by associated training algorithms. The simulation results of nonlinear function approximation are given out and some improvements in the future are outlined.

• Journal of Korean Academy of Oral and Maxillofacial Radiology
• /
• v.29 no.1
• /
• pp.149-159
• /
• 1999

Purpose: The purpose of this study was to evaluate cervical lymph node metastasis of oral squamous cell carcinoma patients by MRI film and neural network system. Materials and Methods: The oral squamous cell carcinoma patients(21 patients. 59 lymph nodes) who have visited SNU hospital and been taken by MRI. were included in this study. Neck dissection operations were done and all of the cervical lymph nodes were confirmed with biopsy. In MR images. each lymph node were evaluated by using 6 MR imaging criteria(size. roundness. heterogeneity. rim enhancement. central necrosis, grouping) respectively. Positive predictive value. negative predictive value. and accuracy of each MR imaging criteria were calculated. At neural network system. the layers of neural network system consisted of 10 input layer units. 10 hidden layer units and 1 output layer unit. 6 MR imaging criteria previously described and 4 MR imaging criteria (site I-node level II and submandibular area. site II-other node level. shape I-oval. shape II-bean) were included for input layer units. The training files were made of 39 lymph nodes(24 metastatic lymph nodes. 10 non-metastatic lymph nodes) and the testing files were made of other 20 lymph nodes(10 metastatic lymph nodes. 10 non-metastatic lymph nodes). The neural network system was trained with training files and the output level (metastatic index) of testing files were acquired. Diagnosis was decided according to 4 different standard metastatic index-68. 78. 88. 98 respectively and positive predictive values. negative predictive values and accuracy of each standard metastatic index were calculated. Results: In the diagnosis of using single MR imaging criteria. the rim enhancement criteria had highest positive predictive value (0.95) and the size criteria had highest negative predictive value (0.77). In the diagnosis of using single MR imaging criteria. the highest accurate criteria was heterogeneity (accuracy: 0.81) and the lowest one was central necrosis (accuracy: 0.59). In the diagnosis of using neural network systems. the highest accurate standard metastatic index was 78. and that time. the accuracy was 0.90. Neural network system was more accurate than any other single MR imaging criteria in evaluating cervical lymph node metastasis. Conclusion: Neural network system has been shown to be more useful than any other single MR imaging criteria. In future. Neural network system will be powerful aiding tool in evaluating cervical node metastasis.