• Title/Summary/Keyword: convolution layer

Search Result 138, Processing Time 0.034 seconds

Comparison of Number Recognition Rates According to Changes in Convolutional Neural Structure (합성곱 신경망 네트워크 구조 변화에 따른 숫자 인식률 비교)

  • Lee, Jong-Chan;Kim, Young-Hyun;Song, Teuk-Seob
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.10a
    • /
    • pp.397-399
    • /
    • 2022
  • Digit recognition is one of the applications of deep learning, which appears in many fields. CNN network enables us to recognize handwritten digits. Also, It can process various types of data. As we stack more layers in CNN network, we expect more performance improvements. In this paper, we added a convolution layer. as a result, we achieved an accuracy improvement from 76.96% to 98.87%, which is a nearly 21.81% increase.

  • PDF

A simple approach to refraction statics with the Generalized Reciprocal Method and the Refraction Convolution Section (GRM과 RCS 방법을 이용한 굴절파 정적 시간차를 구하는 간단한 방법)

  • Palmer Derecke;Jones Leonie
    • Geophysics and Geophysical Exploration
    • /
    • v.8 no.1
    • /
    • pp.18-25
    • /
    • 2005
  • We derive refraction statics for seismic data recorded in a hard rock terrain, in which there are large and rapid variations in the depth of weathering. The statics corrections range from less than 10 ms to more than 70 ms, often over distances as short as 12 receiver intervals. This study is another demonstration of the importance in obtaining accurate initial refraction models of the weathering in hard rock terrains in which automatic residual statics may fail. We show that the statics values computed with a simple model of the weathering using the Generalized Reciprocal Method (GRM) and the Refraction Convolution Section (RCS) are comparable in accuracy to those computed with a more complex model of the weathering, using least-mean-squares inversion with the conjugate gradient algorithm (Taner et al., 1998). The differences in statics values between the GRM model and that of Taner et al. (1998) systematically vary from an average of 2ms to 4ms over a distance of 8.8 km. The differences between these two refraction models and the final statics model, which includes the automatic residual values, are generally less than 5 ms. The residuals for the GRM model are frequently less than those for the model of Taner et al. (1998). The RCS statics are picked approximately 10 ms later, but their relative accuracy is comparable to that of the GRM statics. The residual statics values show a general correlation with the refraction statics values, and they can be reduced in magnitude by using a lower average seismic velocity in the weathering. These results suggest that inaccurate average seismic velocities in the weathered layer may often be a source of short-wavelength statics, rather than any shortcomings with the inversion algorithms in determining averaged delay times from the traveltimes.

Efficient Implementation of Convolutional Neural Network Using CUDA (CUDA를 이용한 Convolutional Neural Network의 효율적인 구현)

  • Ki, Cheol-Min;Cho, Tai-Hoon
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.6
    • /
    • pp.1143-1148
    • /
    • 2017
  • Currently, Artificial Intelligence and Deep Learning are rising as hot social issues, and these technologies are applied to various fields. A good method among the various algorithms in Artificial Intelligence is Convolutional Neural Networks. Convolutional Neural Network is a form that adds Convolution Layers to Multi Layer Neural Network. If you use Convolutional Neural Networks for small amount of data, or if the structure of layers is not complicated, you don't have to pay attention to speed. But the learning should take long time when the size of the learning data is large and the structure of layers is complicated. In these cases, GPU-based parallel processing is frequently needed. In this paper, we developed Convolutional Neural Networks using CUDA, and show that its learning is faster and more efficient than learning using some other frameworks or programs.

Classification Algorithms for Human and Dog Movement Based on Micro-Doppler Signals

  • Lee, Jeehyun;Kwon, Jihoon;Bae, Jin-Ho;Lee, Chong Hyun
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.6 no.1
    • /
    • pp.10-17
    • /
    • 2017
  • We propose classification algorithms for human and dog movement. The proposed algorithms use micro-Doppler signals obtained from humans and dogs moving in four different directions. A two-stage classifier based on a support vector machine (SVM) is proposed, which uses a radial-based function (RBF) kernel and $16^{th}$-order linear predictive code (LPC) coefficients as feature vectors. With the proposed algorithms, we obtain the best classification results when a first-level SVM classifies the type of movement, and then, a second-level SVM classifies the moving object. We obtain the correct classification probability 95.54% of the time, on average. Next, to deal with the difficult classification problem of human and dog running, we propose a two-layer convolutional neural network (CNN). The proposed CNN is composed of six ($6{\times}6$) convolution filters at the first and second layers, with ($5{\times}5$) max pooling for the first layer and ($2{\times}2$) max pooling for the second layer. The proposed CNN-based classifier adopts an auto regressive spectrogram as the feature image obtained from the $16^{th}$-order LPC vectors for a specific time duration. The proposed CNN exhibits 100% classification accuracy and outperforms the SVM-based classifier. These results show that the proposed classifiers can be used for human and dog classification systems and also for classification problems using data obtained from an ultra-wideband (UWB) sensor.

A Time Series Graph based Convolutional Neural Network Model for Effective Input Variable Pattern Learning : Application to the Prediction of Stock Market (효과적인 입력변수 패턴 학습을 위한 시계열 그래프 기반 합성곱 신경망 모형: 주식시장 예측에의 응용)

  • Lee, Mo-Se;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.1
    • /
    • pp.167-181
    • /
    • 2018
  • Over the past decade, deep learning has been in spotlight among various machine learning algorithms. In particular, CNN(Convolutional Neural Network), which is known as the effective solution for recognizing and classifying images or voices, has been popularly applied to classification and prediction problems. In this study, we investigate the way to apply CNN in business problem solving. Specifically, this study propose to apply CNN to stock market prediction, one of the most challenging tasks in the machine learning research. As mentioned, CNN has strength in interpreting images. Thus, the model proposed in this study adopts CNN as the binary classifier that predicts stock market direction (upward or downward) by using time series graphs as its inputs. That is, our proposal is to build a machine learning algorithm that mimics an experts called 'technical analysts' who examine the graph of past price movement, and predict future financial price movements. Our proposed model named 'CNN-FG(Convolutional Neural Network using Fluctuation Graph)' consists of five steps. In the first step, it divides the dataset into the intervals of 5 days. And then, it creates time series graphs for the divided dataset in step 2. The size of the image in which the graph is drawn is $40(pixels){\times}40(pixels)$, and the graph of each independent variable was drawn using different colors. In step 3, the model converts the images into the matrices. Each image is converted into the combination of three matrices in order to express the value of the color using R(red), G(green), and B(blue) scale. In the next step, it splits the dataset of the graph images into training and validation datasets. We used 80% of the total dataset as the training dataset, and the remaining 20% as the validation dataset. And then, CNN classifiers are trained using the images of training dataset in the final step. Regarding the parameters of CNN-FG, we adopted two convolution filters ($5{\times}5{\times}6$ and $5{\times}5{\times}9$) in the convolution layer. In the pooling layer, $2{\times}2$ max pooling filter was used. The numbers of the nodes in two hidden layers were set to, respectively, 900 and 32, and the number of the nodes in the output layer was set to 2(one is for the prediction of upward trend, and the other one is for downward trend). Activation functions for the convolution layer and the hidden layer were set to ReLU(Rectified Linear Unit), and one for the output layer set to Softmax function. To validate our model - CNN-FG, we applied it to the prediction of KOSPI200 for 2,026 days in eight years (from 2009 to 2016). To match the proportions of the two groups in the independent variable (i.e. tomorrow's stock market movement), we selected 1,950 samples by applying random sampling. Finally, we built the training dataset using 80% of the total dataset (1,560 samples), and the validation dataset using 20% (390 samples). The dependent variables of the experimental dataset included twelve technical indicators popularly been used in the previous studies. They include Stochastic %K, Stochastic %D, Momentum, ROC(rate of change), LW %R(Larry William's %R), A/D oscillator(accumulation/distribution oscillator), OSCP(price oscillator), CCI(commodity channel index), and so on. To confirm the superiority of CNN-FG, we compared its prediction accuracy with the ones of other classification models. Experimental results showed that CNN-FG outperforms LOGIT(logistic regression), ANN(artificial neural network), and SVM(support vector machine) with the statistical significance. These empirical results imply that converting time series business data into graphs and building CNN-based classification models using these graphs can be effective from the perspective of prediction accuracy. Thus, this paper sheds a light on how to apply deep learning techniques to the domain of business problem solving.

A Study on Random Selection of Pooling Operations for Regularization and Reduction of Cross Validation (정규화 및 교차검증 횟수 감소를 위한 무작위 풀링 연산 선택에 관한 연구)

  • Ryu, Seo-Hyeon
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.4
    • /
    • pp.161-166
    • /
    • 2018
  • In this paper, we propose a method for the random selection of pooling operations for the regularization and reduction of cross validation in convolutional neural networks. The pooling operation in convolutional neural networks is used to reduce the size of the feature map and for its shift invariant properties. In the existing pooling method, one pooling operation is applied in each pooling layer. Because this method fixes the convolution network, the network suffers from overfitting, which means that it excessively fits the models to the training samples. In addition, to find the best combination of pooling operations to maximize the performance, cross validation must be performed. To solve these problems, we introduce the probability concept into the pooling layers. The proposed method does not select one pooling operation in each pooling layer. Instead, we randomly select one pooling operation among multiple pooling operations in each pooling region during training, and for testing purposes, we use probabilistic weighting to produce the expected output. The proposed method can be seen as a technique in which many networks are approximately averaged using a different pooling operation in each pooling region. Therefore, this method avoids the overfitting problem, as well as reducing the amount of cross validation. The experimental results show that the proposed method can achieve better generalization performance and reduce the need for cross validation.

A Study On Memory Optimization for Applying Deep Learning to PC (딥러닝을 PC에 적용하기 위한 메모리 최적화에 관한 연구)

  • Lee, Hee-Yeol;Lee, Seung-Ho
    • Journal of IKEEE
    • /
    • v.21 no.2
    • /
    • pp.136-141
    • /
    • 2017
  • In this paper, we propose an algorithm for memory optimization to apply deep learning to PC. The proposed algorithm minimizes the memory and computation processing time by reducing the amount of computation processing and data required in the conventional deep learning structure in a general PC. The algorithm proposed in this paper consists of three steps: a convolution layer configuration process using a random filter with discriminating power, a data reduction process using PCA, and a CNN structure creation using SVM. The learning process is not necessary in the convolution layer construction process using the discriminating random filter, thereby shortening the learning time of the overall deep learning. PCA reduces the amount of memory and computation throughput. The creation of the CNN structure using SVM maximizes the effect of reducing the amount of memory and computational throughput required. In order to evaluate the performance of the proposed algorithm, we experimented with Yale University's Extended Yale B face database. The results show that the algorithm proposed in this paper has a similar performance recognition rate compared with the existing CNN algorithm. And it was confirmed to be excellent. Based on the algorithm proposed in this paper, it is expected that a deep learning algorithm with many data and computation processes can be implemented in a general PC.

Investigation of the Super-resolution Algorithm for the Prediction of Periodontal Disease in Dental X-ray Radiography (치주질환 예측을 위한 치과 X-선 영상에서의 초해상화 알고리즘 적용 가능성 연구)

  • Kim, Han-Na
    • Journal of the Korean Society of Radiology
    • /
    • v.15 no.2
    • /
    • pp.153-158
    • /
    • 2021
  • X-ray image analysis is a very important field to improve the early diagnosis rate and prediction accuracy of periodontal disease. Research on the development and application of artificial intelligence-based algorithms to improve the quality of such dental X-ray images is being widely conducted worldwide. Thus, the aim of this study was to design a super-resolution algorithm for predicting periodontal disease and to evaluate its applicability in dental X-ray images. The super-resolution algorithm was constructed based on the convolution layer and ReLU, and an image obtained by up-sampling a low-resolution image by 2 times was used as an input data. Also, 1,500 dental X-ray data used for deep learning training were used. Quantitative evaluation of images used root mean square error and structural similarity, which are factors that can measure similarity through comparison of two images. In addition, the recently developed no-reference based natural image quality evaluator and blind/referenceless image spatial quality evaluator were additionally analyzed. According to the results, we confirmed that the average similarity and no-reference-based evaluation values were improved by 1.86 and 2.14 times, respectively, compared to the existing bicubic-based upsampling method when the proposed method was used. In conclusion, the super-resolution algorithm for predicting periodontal disease proved useful in dental X-ray images, and it is expected to be highly applicable in various fields in the future.

Verification of Radiation Therapy Planning Dose Based on Electron Density Correction of CT Number: XiO Experiments (컴퓨터영상의 전자밀도보정에 근거한 치료선량확인: XiO 실험)

  • Choi Tae-Jin;Kim Jin-Hee;Kim Ok-Bae
    • Progress in Medical Physics
    • /
    • v.17 no.2
    • /
    • pp.105-113
    • /
    • 2006
  • This study peformed to confirm the corrected dose In different electron density materials using the superposition/FFT convolution method in radiotherapy Planning system. The experiments of the $K_2HPO_4$ diluted solution for bone substitute, Cork for lung and n-Glucose for soft tissue are very close to effective atomic number of tissue materials. The image data acquisited from the 110 KVp and 130 KVp CT scanner (Siemes, Singo emotions). The electron density was derived from the CT number (H) and adapted to planning system (Xio, CMS) for heterogeneity correction. The heterogeneity tissue phantom used for measurement dose comparison to that of delivered computer planning system. In the results, this investigations showed the CT number is highly affected in photoelectric effect in high Z materials. The electron density in a given energy spectrum showed the relation of first order as a function of H in soft tissue and bone materials, respectively. In our experiments, the ratio of electron density as a function of H was obtained the 0.001026H+1.00 in soft tissue and 0.000304H+1.07 for bone at 130 KVp spectrum and showed 0.000274H+1.10 for bone tissue in low 110 KVp. This experiments of electron density calibrations from CT number used to decide depth and length of photon transportation. The Computed superposition and FFT convolution dose showed very close to measurements within 1.0% discrepancy in homogeneous phantom for 6 and 15 MV X rays, but it showed -5.0% large discrepancy in FFT convolution for bone tissue correction of 6 MV X rays. In this experiments, the evaluated doses showed acceptable discrepancy within -1.2% of average for lung and -2.9% for bone equivalent materials with superposition method in 6 MV X rays. However the FFT convolution method showed more a large discrepancy than superposition in the low electron density medium in 6 and 15 MV X rays. As the CT number depends on energy spectrum of X rays, it should be confirm gradient of function of CT number-electron density regularly.

  • PDF

Customer Behavior Prediction of Binary Classification Model Using Unstructured Information and Convolution Neural Network: The Case of Online Storefront (비정형 정보와 CNN 기법을 활용한 이진 분류 모델의 고객 행태 예측: 전자상거래 사례를 중심으로)

  • Kim, Seungsoo;Kim, Jongwoo
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.2
    • /
    • pp.221-241
    • /
    • 2018
  • Deep learning is getting attention recently. The deep learning technique which had been applied in competitions of the International Conference on Image Recognition Technology(ILSVR) and AlphaGo is Convolution Neural Network(CNN). CNN is characterized in that the input image is divided into small sections to recognize the partial features and combine them to recognize as a whole. Deep learning technologies are expected to bring a lot of changes in our lives, but until now, its applications have been limited to image recognition and natural language processing. The use of deep learning techniques for business problems is still an early research stage. If their performance is proved, they can be applied to traditional business problems such as future marketing response prediction, fraud transaction detection, bankruptcy prediction, and so on. So, it is a very meaningful experiment to diagnose the possibility of solving business problems using deep learning technologies based on the case of online shopping companies which have big data, are relatively easy to identify customer behavior and has high utilization values. Especially, in online shopping companies, the competition environment is rapidly changing and becoming more intense. Therefore, analysis of customer behavior for maximizing profit is becoming more and more important for online shopping companies. In this study, we propose 'CNN model of Heterogeneous Information Integration' using CNN as a way to improve the predictive power of customer behavior in online shopping enterprises. In order to propose a model that optimizes the performance, which is a model that learns from the convolution neural network of the multi-layer perceptron structure by combining structured and unstructured information, this model uses 'heterogeneous information integration', 'unstructured information vector conversion', 'multi-layer perceptron design', and evaluate the performance of each architecture, and confirm the proposed model based on the results. In addition, the target variables for predicting customer behavior are defined as six binary classification problems: re-purchaser, churn, frequent shopper, frequent refund shopper, high amount shopper, high discount shopper. In order to verify the usefulness of the proposed model, we conducted experiments using actual data of domestic specific online shopping company. This experiment uses actual transactions, customers, and VOC data of specific online shopping company in Korea. Data extraction criteria are defined for 47,947 customers who registered at least one VOC in January 2011 (1 month). The customer profiles of these customers, as well as a total of 19 months of trading data from September 2010 to March 2012, and VOCs posted for a month are used. The experiment of this study is divided into two stages. In the first step, we evaluate three architectures that affect the performance of the proposed model and select optimal parameters. We evaluate the performance with the proposed model. Experimental results show that the proposed model, which combines both structured and unstructured information, is superior compared to NBC(Naïve Bayes classification), SVM(Support vector machine), and ANN(Artificial neural network). Therefore, it is significant that the use of unstructured information contributes to predict customer behavior, and that CNN can be applied to solve business problems as well as image recognition and natural language processing problems. It can be confirmed through experiments that CNN is more effective in understanding and interpreting the meaning of context in text VOC data. And it is significant that the empirical research based on the actual data of the e-commerce company can extract very meaningful information from the VOC data written in the text format directly by the customer in the prediction of the customer behavior. Finally, through various experiments, it is possible to say that the proposed model provides useful information for the future research related to the parameter selection and its performance.