• Title/Summary/Keyword: 랜덤추출

Search Result 191, Processing Time 0.032 seconds

A Case Study on Text Analysis Using Meal Kit Product Review Data (밀키트 제품 리뷰 데이터를 이용한 텍스트 분석 사례 연구)

  • Choi, Hyeseon;Yeon, Kyupil
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.5
    • /
    • pp.1-15
    • /
    • 2022
  • In this study, text analysis was performed on the mealkit product review data to identify factors affecting the evaluation of the mealkit product. The data used for the analysis were collected by scraping 334,498 reviews of mealkit products in Naver shopping site. After preprocessing the text data, wordclouds and sentiment analyses based on word frequency and normalized TF-IDF were performed. Logistic regression model was applied to predict the polarity of reviews on mealkit products. From the logistic regression models derived for each product category, the main factors that caused positive and negative emotions were identified. As a result, it was verified that text analysis can be a useful tool that provides a basis for maximizing positive factors for a specific category, menu, and material and removing negative risk factors when developing a mealkit product.

Development of machine learning framework to inverse-track a contaminant source of hazardous chemicals in rivers (하천에 유입된 유해화학물질의 역추적을 위한 기계학습 프레임워크 개발)

  • Kwon, Siyoon;Seo, Il Won
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2020.06a
    • /
    • pp.112-112
    • /
    • 2020
  • 하천에서 유해화학물질 유입 사고 발생 시 수환경 피해를 최소화하기 위해 신속한 초기 대응이 필요하다. 따라서, 본 연구에서는 수환경 화학사고 대응 시스템 구축을 위해 하천 실시간 모니터링 지점에서 관측된 유해화학물질의 농도 자료를 이용하여 발생원의 유입 지점과 유입량을 역추적하는 프레임워크를 개발하였다. 본 연구에서 제시하는 프레임워크는 첫 번째로 하천 저장대 모형(Transient Storage Zone Model; TSM)과 HEC-RAS 모형을 이용하여 다양한 유량의 수리 조건에서 화학사고 시나리오를 생성하는 단계, 두번째로 생성된 시나리오의 유입 지점과 유입량에 대한 시간-농도 곡선 (BreakThrough Curve; BTC)을 21개의 곡선특징 (BTC feature)으로 추출하는 단계, 최종적으로 재귀적 특징 선택법(Recursive Feature Elimination; RFE)을 이용하여 의사결정나무 모형, 랜덤포레스트 모형, Xgboost 모형, 선형 서포트 벡터 머신, 커널 서포트 벡터 머신 그리고 Ridge 모형에 대한 모형별 주요 특징을 학습하고 성능을 비교하여 각각 유입 위치와 유입 질량 예측에 대한 최적 모형 및 특징 조합을 제시하는 단계로 구축하였다. 또한, 현장 적용성 제고를 위해 시간-농도 곡선을 2가지 경우 (Whole BTC와 Fractured BTC)로 가정하여 기계학습 모형을 학습시켜 모의결과를 비교하였다. 제시된 프레임워크의 검증을 위해서 낙동강 지류인 감천에 적용하여 모형을 구축하고 시나리오 자료 기반 검증과 Rhodamine WT를 이용한 추적자 실험자료를 이용한 검증을 수행하였다. 기계학습 모형들의 비교 검증 결과, 각 모형은 가중항 기반과 불순도 감소량 기반 특징 중요도 산출 방식에 따라 주요 특징이 상이하게 산출되었으며, 전체 시간-농도 곡선 (WBTC)과 부분 시간-농도 곡선 (FBTC)별 최적 모형도 다르게 산출되었다. 유입 위치 정확도 및 유입 질량 예측에 대한 R2는 대부분의 모형이 90% 이상의 우수한 결과를 나타냈다.

  • PDF

Landslide Susceptibility Mapping Using Ensemble FR and LR models at the Inje Area, Korea (FR과 LR 앙상블 모형을 이용한 산사태 취약성 지도 제작 및 검증)

  • Kim, Jin Soo;Park, So Young
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.25 no.1
    • /
    • pp.19-27
    • /
    • 2017
  • This research was aimed to analyze landslide susceptibility and compare the prediction accuracy using ensemble frequency ratio (FR) and logistic regression at the Inje area, Korea. The landslide locations were identified with the before and after aerial photographs of landslide occurrence that were randomly selected for training (70%) and validation (30%). The total twelve landslide-related factors were elevation, slope, aspect, distance to drainage, topographic wetness index, stream power index, soil texture, soil sickness, timber age, timber diameter, timber density, and timber type. The spatial relationship between landslide occurrence and landslide-related factors was analyzed using FR and ensemble model. The produced LSI maps were validated and compared using relative operating characteristics (ROC) curve. The prediction accuracy of produced ensemble LSI map was about 2% higher than FR LSI map. The LSI map produced in this research could be used to establish land use planning and mitigate the damages caused by disaster.

A Study on Pre-evaluation of Tree Species Classification Possibility of CAS500-4 Using RapidEye Satellite Imageries (농림위성 활용 수종분류 가능성 평가를 위한 래피드아이 영상 기반 시험 분석)

  • Kwon, Soo-Kyung;Kim, Kyoung-Min;Lim, Joongbin
    • Korean Journal of Remote Sensing
    • /
    • v.37 no.2
    • /
    • pp.291-304
    • /
    • 2021
  • Updating a forest type map is essential for sustainable forest resource management and monitoring to cope with climate change and various environmental problems. According to the necessity of efficient and wide-area forestry remote sensing, CAS500-4 (Compact Advanced Satellite 500-4; The agriculture and forestry satellite) project has been confirmed and scheduled for launch in 2023. Before launching and utilizing CAS500-4, this study aimed to pre-evaluation the possibility of satellite-based tree species classification using RapidEye, which has similar specifications to the CAS500-4. In this study, the study area was the Chuncheon forest management complex, Gangwon-do. The spectral information was extracted from the growing season image. And the GLCM texture information was derived from the growing and non-growing seasons NIR bands. Both information were used to classification with random forest machine learning method. In this study, tree species were classified into nine classes to the coniferous tree (Korean red pine, Korean pine, Japanese larch), broad-leaved trees (Mongolian oak, Oriental cork oak, East Asian white birch, Korean Castanea, and other broad-leaved trees), and mixed forest. Finally, the classification accuracy was calculated by comparing the forest type map and classification results. As a result, the accuracy was 39.41% when only spectral information was used and 69.29% when both spectral information and texture information was used. For future study, the applicability of the CAS500-4 will be improved by substituting additional variables that more effectively reflect vegetation's ecological characteristics.

Increasing Accuracy of Stock Price Pattern Prediction through Data Augmentation for Deep Learning (데이터 증강을 통한 딥러닝 기반 주가 패턴 예측 정확도 향상 방안)

  • Kim, Youngjun;Kim, Yeojeong;Lee, Insun;Lee, Hong Joo
    • The Journal of Bigdata
    • /
    • v.4 no.2
    • /
    • pp.1-12
    • /
    • 2019
  • As Artificial Intelligence (AI) technology develops, it is applied to various fields such as image, voice, and text. AI has shown fine results in certain areas. Researchers have tried to predict the stock market by utilizing artificial intelligence as well. Predicting the stock market is known as one of the difficult problems since the stock market is affected by various factors such as economy and politics. In the field of AI, there are attempts to predict the ups and downs of stock price by studying stock price patterns using various machine learning techniques. This study suggest a way of predicting stock price patterns based on the Convolutional Neural Network(CNN) among machine learning techniques. CNN uses neural networks to classify images by extracting features from images through convolutional layers. Therefore, this study tries to classify candlestick images made by stock data in order to predict patterns. This study has two objectives. The first one referred as Case 1 is to predict the patterns with the images made by the same-day stock price data. The second one referred as Case 2 is to predict the next day stock price patterns with the images produced by the daily stock price data. In Case 1, data augmentation methods - random modification and Gaussian noise - are applied to generate more training data, and the generated images are put into the model to fit. Given that deep learning requires a large amount of data, this study suggests a method of data augmentation for candlestick images. Also, this study compares the accuracies of the images with Gaussian noise and different classification problems. All data in this study is collected through OpenAPI provided by DaiShin Securities. Case 1 has five different labels depending on patterns. The patterns are up with up closing, up with down closing, down with up closing, down with down closing, and staying. The images in Case 1 are created by removing the last candle(-1candle), the last two candles(-2candles), and the last three candles(-3candles) from 60 minutes, 30 minutes, 10 minutes, and 5 minutes candle charts. 60 minutes candle chart means one candle in the image has 60 minutes of information containing an open price, high price, low price, close price. Case 2 has two labels that are up and down. This study for Case 2 has generated for 60 minutes, 30 minutes, 10 minutes, and 5minutes candle charts without removing any candle. Considering the stock data, moving the candles in the images is suggested, instead of existing data augmentation techniques. How much the candles are moved is defined as the modified value. The average difference of closing prices between candles was 0.0029. Therefore, in this study, 0.003, 0.002, 0.001, 0.00025 are used for the modified value. The number of images was doubled after data augmentation. When it comes to Gaussian Noise, the mean value was 0, and the value of variance was 0.01. For both Case 1 and Case 2, the model is based on VGG-Net16 that has 16 layers. As a result, 10 minutes -1candle showed the best accuracy among 60 minutes, 30 minutes, 10 minutes, 5minutes candle charts. Thus, 10 minutes images were utilized for the rest of the experiment in Case 1. The three candles removed from the images were selected for data augmentation and application of Gaussian noise. 10 minutes -3candle resulted in 79.72% accuracy. The accuracy of the images with 0.00025 modified value and 100% changed candles was 79.92%. Applying Gaussian noise helped the accuracy to be 80.98%. According to the outcomes of Case 2, 60minutes candle charts could predict patterns of tomorrow by 82.60%. To sum up, this study is expected to contribute to further studies on the prediction of stock price patterns using images. This research provides a possible method for data augmentation of stock data.

  • PDF

NEAR-INFRARED VARIABILITY OF OPTICALLY BRIGHT TYPE 1 AGN (가시광에서 밝은 1형 활동은하핵의 근적외선 변광)

  • JEON, WOOYEOL;SHIM, HYUNJIN;KIM, MINJIN
    • Publications of The Korean Astronomical Society
    • /
    • v.36 no.3
    • /
    • pp.47-63
    • /
    • 2021
  • Variability is one of the major characteristics of Active Galactic Nuclei (AGN), and it is used for understanding the energy generation mechanism in the center of AGN and/or related physical phenomena. It it known that there exists a time lag between AGN light curves simultaneously observed at different wavelengths, which can be used as a tool to estimate the size of the area that produce the radiation. In this paper, We present long term near-infrared variability of optically bright type 1 AGN using the Wide-field Infrared Survey Explorer data. From the Milliquas catalogue v6.4, 73 type 1 QSOs/AGN and 140 quasar candidates are selected that are brighter than 18 mag in optical and located within 5 degree around the ecliptic poles. Light curves in the W1 band (3.4 ㎛) and W2 band (4.6 ㎛) during the period of 2010-2019 were constructed for these objects by extracting multi-epoch photometry data from WISE and NEOWISE all sky survey database. Variability was analyzed based on the excess variance and the probability Pvar. Applying both criteria, the numbers of variable objects are 19 (i.e., 26%) for confirmed AGN and 12 (i.e., 9%) for AGN candidates. The characteristic time scale of the variability (τ) and the variability amplitude (σ) were derived by fitting the DRW model to W1 and W2 light curves. No significant correlation is found between the W1/W2 magnitude and the derived variability parameters. Based on the subsample that are identified in the X-ray source catalog, there exists little correlation between the X-ray luminosity and the variability parameters. We also found four AGN with changing W1-W2 color.

Vegetation classification based on remote sensing data for river management (하천 관리를 위한 원격탐사 자료 기반 식생 분류 기법)

  • Lee, Chanjoo;Rogers, Christine;Geerling, Gertjan;Pennin, Ellis
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2021.06a
    • /
    • pp.6-7
    • /
    • 2021
  • Vegetation development in rivers is one of the important issues not only in academic fields such as geomorphology, ecology, hydraulics, etc., but also in river management practices. The problem of river vegetation is directly connected to the harmony of conflicting values of flood management and ecosystem conservation. In Korea, since the 2000s, the issue of river vegetation and land formation has been continuously raised under various conditions, such as the regulating rivers downstream of the dams, the small eutrophicated tributary rivers, and the floodplain sites for the four major river projects. In this background, this study proposes a method for classifying the distribution of vegetation in rivers based on remote sensing data, and presents the results of applying this to the Naeseong Stream. The Naeseong Stream is a representative example of the river landscape that has changed due to vegetation development from 2014 to the latest. The remote sensing data used in the study are images of Sentinel 1 and 2 satellites, which is operated by the European Aerospace Administration (ESA), and provided by Google Earth Engine. For the ground truth, manually classified dataset on the surface of the Naeseong Stream in 2016 were used, where the area is divided into eight types including water, sand and herbaceous and woody vegetation. The classification method used a random forest classification technique, one of the machine learning algorithms. 1,000 samples were extracted from 10 pre-selected polygon regions, each half of them were used as training and verification data. The accuracy based on the verification data was found to be 82~85%. The model established through training was also applied to images from 2016 to 2020, and the process of changes in vegetation zones according to the year was presented. The technical limitations and improvement measures of this paper were considered. By providing quantitative information of the vegetation distribution, this technique is expected to be useful in practical management of vegetation such as thinning and rejuvenation of river vegetation as well as technical fields such as flood level calculation and flow-vegetation coupled modeling in rivers.

  • PDF

Automated Analyses of Ground-Penetrating Radar Images to Determine Spatial Distribution of Buried Cultural Heritage (매장 문화재 공간 분포 결정을 위한 지하투과레이더 영상 분석 자동화 기법 탐색)

  • Kwon, Moonhee;Kim, Seung-Sep
    • Economic and Environmental Geology
    • /
    • v.55 no.5
    • /
    • pp.551-561
    • /
    • 2022
  • Geophysical exploration methods are very useful for generating high-resolution images of underground structures, and such methods can be applied to investigation of buried cultural properties and for determining their exact locations. In this study, image feature extraction and image segmentation methods were applied to automatically distinguish the structures of buried relics from the high-resolution ground-penetrating radar (GPR) images obtained at the center of Silla Kingdom, Gyeongju, South Korea. The major purpose for image feature extraction analyses is identifying the circular features from building remains and the linear features from ancient roads and fences. Feature extraction is implemented by applying the Canny edge detection and Hough transform algorithms. We applied the Hough transforms to the edge image resulted from the Canny algorithm in order to determine the locations the target features. However, the Hough transform requires different parameter settings for each survey sector. As for image segmentation, we applied the connected element labeling algorithm and object-based image analysis using Orfeo Toolbox (OTB) in QGIS. The connected components labeled image shows the signals associated with the target buried relics are effectively connected and labeled. However, we often find multiple labels are assigned to a single structure on the given GPR data. Object-based image analysis was conducted by using a Large-Scale Mean-Shift (LSMS) image segmentation. In this analysis, a vector layer containing pixel values for each segmented polygon was estimated first and then used to build a train-validation dataset by assigning the polygons to one class associated with the buried relics and another class for the background field. With the Random Forest Classifier, we find that the polygons on the LSMS image segmentation layer can be successfully classified into the polygons of the buried relics and those of the background. Thus, we propose that these automatic classification methods applied to the GPR images of buried cultural heritage in this study can be useful to obtain consistent analyses results for planning excavation processes.

Development of a Clinical Decision Support System Utilizing Support Vector Machine (Support Vector Machine을 이용한 생체 신호 분류기 개발)

  • Hong, Dong-Kwon;Chai, Yong-Yoong
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.13 no.3
    • /
    • pp.661-668
    • /
    • 2018
  • Biomedical signals using skin resistance have different characteristics according to stress diseases. Biological diagnostic devices for diagnosing stress diseases have been developed by using these characteristics, and devices have been developed so that the signals measured by the skin storage meter can be easily analyzed. Experts in the field will look directly at the output signal to determine the likelihood of any stress disorder. However, it is very difficult for a person to accurately determine whether a person to be measured has a stress disorder by analyzing a bio-signal measured by each person to be measured, and the result of the judgment is very likely to be wrong. In order to solve these problems, we implemented the function of determining the signal of a stress disorder by using the machine learning technique. SVM was used as a classification method in consideration of low computing ability of measurement equipment. Training data and test data were randomly generated for each disease using error range 5 based on 13 diseases. Simulation results showed more than 90% decision accuracy. In the future, if the measurement equipment is actually applied to the patients, we can retrain the classifier with the newly generated data.

A Hybrid Under-sampling Approach for Better Bankruptcy Prediction (부도예측 개선을 위한 하이브리드 언더샘플링 접근법)

  • Kim, Taehoon;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.2
    • /
    • pp.173-190
    • /
    • 2015
  • The purpose of this study is to improve bankruptcy prediction models by using a novel hybrid under-sampling approach. Most prior studies have tried to enhance the accuracy of bankruptcy prediction models by improving the classification methods involved. In contrast, we focus on appropriate data preprocessing as a means of enhancing accuracy. In particular, we aim to develop an effective sampling approach for bankruptcy prediction, since most prediction models suffer from class imbalance problems. The approach proposed in this study is a hybrid under-sampling method that combines the k-Reverse Nearest Neighbor (k-RNN) and one-class support vector machine (OCSVM) approaches. k-RNN can effectively eliminate outliers, while OCSVM contributes to the selection of informative training samples from majority class data. To validate our proposed approach, we have applied it to data from H Bank's non-external auditing companies in Korea, and compared the performances of the classifiers with the proposed under-sampling and random sampling data. The empirical results show that the proposed under-sampling approach generally improves the accuracy of classifiers, such as logistic regression, discriminant analysis, decision tree, and support vector machines. They also show that the proposed under-sampling approach reduces the risk of false negative errors, which lead to higher misclassification costs.