• Journal of Intelligence and Information Systems
• /
• v.22 no.2
• /
• pp.127-142
• /
• 2016

Deep learning model is a kind of neural networks that allows multiple hidden layers. There are various deep learning architectures such as convolutional neural networks, deep belief networks and recurrent neural networks. Those have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. Among those architectures, convolutional neural networks and recurrent neural networks are classified as the supervised learning model. And in recent years, those supervised learning models have gained more popularity than unsupervised learning models such as deep belief networks, because supervised learning models have shown fashionable applications in such fields mentioned above. Deep learning models can be trained with backpropagation algorithm. Backpropagation is an abbreviation for "backward propagation of errors" and a common method of training artificial neural networks used in conjunction with an optimization method such as gradient descent. The method calculates the gradient of an error function with respect to all the weights in the network. The gradient is fed to the optimization method which in turn uses it to update the weights, in an attempt to minimize the error function. Convolutional neural networks use a special architecture which is particularly well-adapted to classify images. Using this architecture makes convolutional networks fast to train. This, in turn, helps us train deep, muti-layer networks, which are very good at classifying images. These days, deep convolutional networks are used in most neural networks for image recognition. Convolutional neural networks use three basic ideas: local receptive fields, shared weights, and pooling. By local receptive fields, we mean that each neuron in the first(or any) hidden layer will be connected to a small region of the input(or previous layer's) neurons. Shared weights mean that we're going to use the same weights and bias for each of the local receptive field. This means that all the neurons in the hidden layer detect exactly the same feature, just at different locations in the input image. In addition to the convolutional layers just described, convolutional neural networks also contain pooling layers. Pooling layers are usually used immediately after convolutional layers. What the pooling layers do is to simplify the information in the output from the convolutional layer. Recent convolutional network architectures have 10 to 20 hidden layers and billions of connections between units. Training deep learning networks has taken weeks several years ago, but thanks to progress in GPU and algorithm enhancement, training time has reduced to several hours. Neural networks with time-varying behavior are known as recurrent neural networks or RNNs. A recurrent neural network is a class of artificial neural network where connections between units form a directed cycle. This creates an internal state of the network which allows it to exhibit dynamic temporal behavior. Unlike feedforward neural networks, RNNs can use their internal memory to process arbitrary sequences of inputs. Early RNN models turned out to be very difficult to train, harder even than deep feedforward networks. The reason is the unstable gradient problem such as vanishing gradient and exploding gradient. The gradient can get smaller and smaller as it is propagated back through layers. This makes learning in early layers extremely slow. The problem actually gets worse in RNNs, since gradients aren't just propagated backward through layers, they're propagated backward through time. If the network runs for a long time, that can make the gradient extremely unstable and hard to learn from. It has been possible to incorporate an idea known as long short-term memory units (LSTMs) into RNNs. LSTMs make it much easier to get good results when training RNNs, and many recent papers make use of LSTMs or related ideas.

• Journal of Korea Multimedia Society
• /
• v.21 no.3
• /
• pp.391-399
• /
• 2018

In this paper, we construct and compare various guitar chord classification systems using perceptron neural network and convolutional neural network without pre-processing other than Fourier transform to identify the optimal chord classification system. Conventional guitar chord classification schemes use, for better feature extraction, computationally demanding pre-processing techniques such as stochastic analysis employing a hidden markov model or an acoustic data filtering and hence are burdensome for real-time chord classifications. For this reason, we construct various perceptron neural networks and convolutional neural networks that use only Fourier tranform for data pre-processing and compare them with dataset obtained by playing an electric guitar. According to our comparison, convolutional neural networks provide optimal performance considering both chord classification acurracy and fast processing time. In particular, convolutional neural networks exhibit robust performance even when only small fraction of low frequency components of the data are used.

• Journal of IKEEE
• /
• v.22 no.4
• /
• pp.1131-1139
• /
• 2018

This paper proposes a method of recognition abandoned objects using convolutional neural networks. The method first detects an area for an abandoned object in image and, if there is a detected area, applies convolutional neural networks to that area to recognize which object is represented. Experiments were conducted through an application system that detects illegal trash dumping. The experiments result showed the area of abandoned object was detected efficiently. The detected areas enter the input of convolutional neural networks and are classified into whether it is a trash or not. To do this, I trained convolutional neural networks with my own trash dataset and open database. As a training result, I achieved high accuracy for the test set not included in the training set.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.11
• /
• pp.5427-5445
• /
• 2019

While deep neural networks have achieved remarkable performance in representation learning, a huge amount of labeled training data are usually required by supervised deep models such as convolutional neural networks. In this paper, we propose a new representation learning method, namely generative adversarial networks (GAN) based bagging deep convolutional autoencoders (GAN-BDCAE), which can map data to diverse hierarchical representations in an unsupervised fashion. To boost the size of training data, to train deep model and to aggregate diverse learning machines are the three principal avenues towards increasing the capabilities of representation learning of neural networks. We focus on combining those three techniques. To this aim, we adopt GAN for realistic unlabeled sample generation and bagging deep convolutional autoencoders (BDCAE) for robust feature learning. The proposed method improves the discriminative ability of learned feature embedding for solving subsequent pattern recognition problems. We evaluate our approach on three standard benchmarks and demonstrate the superiority of the proposed method compared to traditional unsupervised learning methods.

• Journal of Information Technology Services
• /
• v.15 no.4
• /
• pp.85-95
• /
• 2016

Recently, deep learning and machine learning have attracted considerable attention and many supporting frameworks appeared. In artificial intelligence field, a large body of research is underway to apply the relevant knowledge for complex problem-solving, necessitating the application of various learning algorithms and training methods to artificial intelligence systems. In addition, there is a dearth of performance evaluation of decision making agents. The decision making agent that can find optimal solutions by using reinforcement learning methods designed through this research can collect raw pixel data observed from dynamic environments and make decisions by itself based on the data. The decision making agent uses convolutional neural networks to classify situations it confronts, and the data observed from the environment undergoes preprocessing before being used. This research represents how the convolutional neural networks and the decision making agent are configured, analyzes learning performance through a value-based algorithm and a policy-based algorithm : a Deep Q-Networks and a Policy Gradient, sets forth their differences and demonstrates how the convolutional neural networks affect entire learning performance when using pixel data. This research is expected to contribute to the improvement of artificial intelligence systems which can efficiently find optimal solutions by using features extracted from raw pixel data.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.21 no.6
• /
• pp.1143-1148
• /
• 2017

Currently, Artificial Intelligence and Deep Learning are rising as hot social issues, and these technologies are applied to various fields. A good method among the various algorithms in Artificial Intelligence is Convolutional Neural Networks. Convolutional Neural Network is a form that adds Convolution Layers to Multi Layer Neural Network. If you use Convolutional Neural Networks for small amount of data, or if the structure of layers is not complicated, you don't have to pay attention to speed. But the learning should take long time when the size of the learning data is large and the structure of layers is complicated. In these cases, GPU-based parallel processing is frequently needed. In this paper, we developed Convolutional Neural Networks using CUDA, and show that its learning is faster and more efficient than learning using some other frameworks or programs.

• Journal of Korea Society of Digital Industry and Information Management
• /
• v.19 no.4
• /
• pp.127-136
• /
• 2023

Many research results on the traffic enforcement of illegal driving of twowheeled motor vehicles using license plate recognition are introduced. Deep learning convolutional neural networks can be used for character and word recognition of license plates because of better generalization capability compared to traditional Backpropagation neural networks. In the plates of twowheeled motor vehicles, the interdependent government and city words are included. If we implement the mutually independent word recognizers using error correction rules for two word recognition results, efficient license plate recognition results can be derived. The CPU based convolutional neural network without library under real time processing has an advantage of low cost real application compared to GPU based convolutional neural network with library. In this paper twowheeled motor vehicle license plate recognition algorithm is introduced using CPU based deep-learning convolutional neural network. The experimental results show that the proposed plate recognizer has 96.2% success rate for outdoor twowheeled motor vehicle images in real time.

• The Journal of Information Systems
• /
• v.30 no.4
• /
• pp.203-226
• /
• 2021

Purpose The purpose of this study is to enhance the accuracy score of personal credit scoring using the convolutional neural networks and secure the transparency of the deep learning model using eXplainalbe Artifical Inteligence(XAI) technique. Design/methodology/approach This study built a classification model by using the convolutional neural networks(CNN) and applied a methodology that is transformation of numerical data to imaged data to apply CNN on personal credit data. Then layer-wise relevance propagation(LRP) was applied to model we constructed to find what variables are more influenced to the output value. Findings According to the empirical analysis result, this study confirmed that accuracy score by model using CNN is highest among other models using logistic regression, neural networks, and support vector machines. In addition, With the LRP that is one of the technique of XAI, variables that have a great influence on calculating the output value for each observation could be found.

• The Journal of the Convergence on Culture Technology
• /
• v.9 no.1
• /
• pp.649-654
• /
• 2023

This paper proposes the design of a neural network structure search model using graph convolutional neural networks. Deep learning has a problem of not being able to verify whether the designed model has a structure with optimized performance due to the nature of learning as a black box. The neural network structure search model is composed of a recurrent neural network that creates a model and a convolutional neural network that is the generated network. Conventional neural network structure search models use recurrent neural networks, but in this paper, we propose GC-NAS, which uses graph convolutional neural networks instead of recurrent neural networks to create convolutional neural network models. The proposed GC-NAS uses the Layer Extraction Block to explore depth, and the Hyper Parameter Prediction Block to explore spatial and temporal information (hyper parameters) based on depth information in parallel. Therefore, since the depth information is reflected, the search area is wider, and the purpose of the search area of the model is clear by conducting a parallel search with depth information, so it is judged to be superior in theoretical structure compared to GC-NAS. GC-NAS is expected to solve the problem of the high-dimensional time axis and the range of spatial search of recurrent neural networks in the existing neural network structure search model through the graph convolutional neural network block and graph generation algorithm. In addition, we hope that the GC-NAS proposed in this paper will serve as an opportunity for active research on the application of graph convolutional neural networks to neural network structure search.

• Journal of the Korean Society of Industry Convergence
• /
• v.23 no.3
• /
• pp.515-521
• /
• 2020

Deep learning methods have been effectively used to provide great improvement in various research fields such as machine learning, image processing and computer vision. One of the most frequently used deep learning methods in image processing is the convolutional neural networks. Compared to the traditional artificial neural networks, convolutional neural networks do not use the predefined kernels, but instead they learn data specific kernels. This property makes them to be used as feature extractors as well. In this study, we compared the quality of CNN features for traditional texture feature extraction methods. Experimental results demonstrate the superiority of the CNN features. Additionally, the recognition process and result of a pioneering CNN on MNIST database are presented.