• KIPS Transactions on Software and Data Engineering
• /
• v.7 no.8
• /
• pp.307-324
• /
• 2018

Recently, recurrent neural networks have been attracting attention in solving prediction problem of sequential data through structure considering time dependency. However, as the time step of sequential data increases, the problem of the gradient vanishing is occurred. Long short-term memory models have been proposed to solve this problem, but there is a limit to storing a lot of data and preserving it for a long time. Therefore, research on memory-augmented neural network (MANN), which is a learning model using recurrent neural networks and memory elements, has been actively conducted. In this paper, we describe the structure and characteristics of MANN models that emerged as a hot topic in deep learning field and present the latest techniques and future research that utilize MANN.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.16 no.3
• /
• pp.448-459
• /
• 2012

It is important to understand how we can deal with elements for the modeling of neural networks when we are investigating the dynamical performance and the information processing capabilities. The information processing capabilities of model neural networks will change for different response, synaptic weights or learning rules. Using the statistical neurodynamics method, we evaluate the capabilities of neural networks in order to understand the basic concept of parallel distributed processing. In this paper, we explain the results of theoretical analysis of the effects of the hysteretic property on the performance of sequential associative neural networks.

• Journal of Platform Technology
• /
• v.6 no.4
• /
• pp.34-40
• /
• 2018

Currently, artificial neural networks perform well for a single task, but NN have the problem of forgetting previous learning by learning other kinds of tasks. This is called catastrophic forgetting. To use of artificial neural networks in general purpose this should be solved. There are many efforts to overcome catastrophic forgetting. However, even though there was a lot of effort, it did not completely overcome the catastrophic forgetting. In this paper, we propose sequential iterative learning using core concepts used in elastic weight consolidation (EWC). The experiment was performed to reproduce catastrophic forgetting phenomenon using EMNIST data set which extended MNIST, which is widely used for artificial neural network learning, and overcome it through sequential iterative learning.

• Structural Engineering and Mechanics
• /
• v.51 no.6
• /
• pp.989-1003
• /
• 2014

In the preliminary design stage of an RC 3D-frame, repeated sequential analyses to determine optimal members' sizes and the investigation of the parameters required to minimize the differential column shortening are computational effort consuming, especially when considering various types of loads such as dead load, temperature action, time dependent effects, construction and live loads. Because the desired accuracy at this stage does not justify such luxury, two backpropagation feedforward artificial neural networks have been proposed in order to approximate this information. Instead of using a commercial software package, many references providing advanced principles have been considered to code a program and generate these neural networks. The first one predicts the typical amount of time between two phases, needed to achieve the minimum maximorum differential column shortening. The other network aims to prognosticate sequential analysis results from those of the simultaneous analysis. After the training stages, testing procedures have been carried out in order to ensure the generalization ability of these respective systems. Numerical cases are studied in order to find out how good these ANN match with the sequential finite element analysis. Comparison reveals an acceptable fit, enabling these systems to be safely used in the preliminary design stage.

• Communications for Statistical Applications and Methods
• /
• v.9 no.1
• /
• pp.155-166
• /
• 2002

Neural networks have been studied as a popular tool for classification and they are very flexible. Also, they are used for many applications of pattern classification and pattern recognition. This paper focuses on Bayesian approach to feed-forward neural networks with single hidden layer of units with logistic activation. In this model, we are interested in deciding the number of nodes of neural network model with p input units, one hidden layer with m hidden nodes and one output unit in Bayesian setup for fixed m. Here, we use the latent variable into the prior of the coefficient regression, and we introduce the 'sequential step' which is based on the idea of the data augmentation by Tanner and Wong(1787). The MCMC method(Gibbs sampler and Metropolish algorithm) can be used to overcome the complicated Bayesian computation. Finally, a proposed method is applied to a simulated data.

• Journal of the Semiconductor & Display Technology
• /
• v.22 no.2
• /
• pp.128-132
• /
• 2023

Recently, with the development of deep learning technology, research on pedestrian attribute recognition technology using deep neural networks has been actively conducted. Existing pedestrian attribute recognition techniques can be obtained in such a way as global-based, regional-area-based, visual attention-based, sequential prediction-based, and newly designed loss function-based, depending on how pedestrian attributes are detected. It is known that the performance of these pedestrian attribute recognition technologies varies greatly depending on the type of backbone network that constitutes the deep neural networks model. Therefore, in this paper, several backbone networks are applied to the baseline pedestrian attribute recognition model and the performance changes of the model are analyzed. In this paper, the analysis is conducted using Resnet34, Resnet50, Resnet101, Swin-tiny, and Swinv2-tiny, which are representative backbone networks used in the fields of image classification, object detection, etc. Furthermore, this paper analyzes the change in time complexity when inferencing each backbone network using a CPU and a GPU.

• Proceedings of the Korean Operations and Management Science Society Conference
• /
• 2003.05a
• /
• pp.884-888
• /
• 2003

In this paper, we introduce the hybrid data mining mechanism based in association rule and fuzzy neural networks (FNN). Most of data mining mechanisms are depended in the association rule extraction algorithm. However, the basic association rule-based data mining has not the learning ability. In addition, sequential patterns of association rules could not represent the complicate fuzzy logic. To resolve these problems, we suggest the hybrid mechanism using association rule-based data mining, and fuzzy neural networks. Our hybrid data mining mechanism was consisted of four phases. First, we used general association rule mining mechanism to develop the initial rule-base. Then, in the second phase, we used the fuzzy neural networks to learn the past historical patterns embedded in the database. Third, fuzzy rule extraction algorithm was used to extract the implicit knowledge from the FNN. Fourth, we combine the association knowledge base and fuzzy rules. Our proposed hybrid data mining mechanism can reflect both association rule-based logical inference and complicate fuzzy logic.

• Proceedings of the KIEE Conference
• /
• 1993.07b
• /
• pp.1004-1006
• /
• 1993

A new method for the optimal design of a single-sided linear induction motor(SLIM) is presented. The method utilizes the neural networks and finite element method for optimizing the design parameters of SLIM. The finite element analysis is used to produce a variety of neural networks training data and the neural networks is used for optimizing the design parameters by sequential unconstrained minimization technique(SUMT). As a result, it is known that the novel method is very efficient and accurate as an optimization technique.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.18 no.3
• /
• pp.510-518
• /
• 2014

In this study, in order to improve and optimize the performance of the turning mechanism for a fish robot in the fluid, we propose the tail joint trajectories using neural networks to mimic the CST(C-shape Sharp Turn) patterns of a real fish which is optimized in the natural environment. In order to mimic the CST patterns of a fish, we convert the sequential recording CST patterns into the coordinate data, and change the numerical coordinate data into a functions. We change the motion functions to the relative joint angles which is adapted to suit robot's shape and data. However, these relative joint trajectories obtained by the sequential recording of the carp have low-precision. It is difficult to apply to the control of a fish robot. Therefore, the relative joint trajectories are interpolated using neural networks with superior generalization ability and applied to the fish robot. we have found that the proposed method using neural networks is superior to ones using high-order polynomial equation through the computer simulations.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.15 no.2
• /
• pp.13-22
• /
• 2005

To improve the anomaly IDS using system calls, this study focuses on Neural Networks Learning using the Soundex algorithm which is designed to change feature selection and variable length data into a fixed length learning pattern. That is, by changing variable length sequential system call data into a fixed length behavior pattern using the Soundex algorithm, this study conducted neural networks learning by using a backpropagation algorithm with fuzzy membership function. The back-propagation neural networks and Neuro-Fuzzy technique are applied for anomaly intrusion detection of system calls using Sendmail Data of UNM to demonstrate its aspect of he complexity of time, space and MDL performance.