• Title/Summary/Keyword: 10-fold cross-validation

Search Result 213, Processing Time 0.026 seconds

Analysis of extreme wind speed and precipitation using copula (코플라함수를 이용한 극단치 강풍과 강수 분석)

  • Kwon, Taeyong;Yoon, Sanghoo
    • Journal of the Korean Data and Information Science Society
    • /
    • v.28 no.4
    • /
    • pp.797-810
    • /
    • 2017
  • The Korean peninsula is exposed to typhoons every year. Typhoons cause huge socioeconomic damage because tropical cyclones tend to occur with strong winds and heavy precipitation. In order to understand the complex dependence structure between strong winds and heavy precipitation, the copula links a set of univariate distributions to a multivariate distribution and has been actively studied in the field of hydrology. In this study, we carried out analysis using data of wind speed and precipitation collected from the weather stations in Busan and Jeju. Log-Normal, Gamma, and Weibull distributions were considered to explain marginal distributions of the copula. Kolmogorov-Smirnov, Cramer-von-Mises, and Anderson-Darling test statistics were employed for testing the goodness-of-fit of marginal distribution. Observed pseudo data were calculated through inverse transformation method for establishing the copula. Elliptical, archimedean, and extreme copula were considered to explain the dependence structure between strong winds and heavy precipitation. In selecting the best copula, we employed the Cramer-von-Mises test and cross-validation. In Busan, precipitation according to average wind speed followed t copula and precipitation just as maximum wind speed adopted Clayton copula. In Jeju, precipitation according to maximum wind speed complied Normal copula and average wind speed as stated in precipitation followed Frank copula and maximum wind speed according to precipitation observed Husler-Reiss copula.

A SVR Based-Pseudo Modified Einstein Procedure Incorporating H-ADCP Model for Real-Time Total Sediment Discharge Monitoring (실시간 총유사량 모니터링을 위한 H-ADCP 연계 수정 아인슈타인 방법의 의사 SVR 모형)

  • Noh, Hyoseob;Son, Geunsoo;Kim, Dongsu;Park, Yong Sung
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.43 no.3
    • /
    • pp.321-335
    • /
    • 2023
  • Monitoring sediment loads in natural rivers is the key process in river engineering, but it is costly and dangerous. In practice, suspended loads are directly measured, and total loads, which is a summation of suspended loads and bed loads, are estimated. This study proposes a real-time sediment discharge monitoring system using the horizontal acoustic Doppler current profiler (H-ADCP) and support vector regression (SVR). The proposed system is comprised of the SVR model for suspended sediment concentration (SVR-SSC) and for total loads (SVR-QTL), respectively. SVR-SSC estimates SSC and SVR-QTL mimics the modified Einstein procedure. The grid search with K-fold cross validation (Grid-CV) and the recursive feature elimination (RFE) were employed to determine SVR's hyperparameters and input variables. The two SVR models showed reasonable cross-validation scores (R2) with 0.885 (SVR-SSC) and 0.860 (SVR-QTL). During the time-series sediment load monitoring period, we successfully detected various sediment transport phenomena in natural streams, such as hysteresis loops and sensitive sediment fluctuations. The newly proposed sediment monitoring system depends only on the gauged features by H-ADCP without additional assumptions in hydraulic variables (e.g., friction slope and suspended sediment size distribution). This method can be applied to any ADCP-installed discharge monitoring station economically and is expected to enhance temporal resolution in sediment monitoring.

Image Mood Classification Using Deep CNN and Its Application to Automatic Video Generation (심층 CNN을 활용한 영상 분위기 분류 및 이를 활용한 동영상 자동 생성)

  • Cho, Dong-Hee;Nam, Yong-Wook;Lee, Hyun-Chang;Kim, Yong-Hyuk
    • Journal of the Korea Convergence Society
    • /
    • v.10 no.9
    • /
    • pp.23-29
    • /
    • 2019
  • In this paper, the mood of images was classified into eight categories through a deep convolutional neural network and video was automatically generated using proper background music. Based on the collected image data, the classification model is learned using a multilayer perceptron (MLP). Using the MLP, a video is generated by using multi-class classification to predict image mood to be used for video generation, and by matching pre-classified music. As a result of 10-fold cross-validation and result of experiments on actual images, each 72.4% of accuracy and 64% of confusion matrix accuracy was achieved. In the case of misclassification, by classifying video into a similar mood, it was confirmed that the music from the video had no great mismatch with images.

Prediction of Key Variables Affecting NBA Playoffs Advancement: Focusing on 3 Points and Turnover Features (미국 프로농구(NBA)의 플레이오프 진출에 영향을 미치는 주요 변수 예측: 3점과 턴오버 속성을 중심으로)

  • An, Sehwan;Kim, Youngmin
    • Journal of Intelligence and Information Systems
    • /
    • v.28 no.1
    • /
    • pp.263-286
    • /
    • 2022
  • This study acquires NBA statistical information for a total of 32 years from 1990 to 2022 using web crawling, observes variables of interest through exploratory data analysis, and generates related derived variables. Unused variables were removed through a purification process on the input data, and correlation analysis, t-test, and ANOVA were performed on the remaining variables. For the variable of interest, the difference in the mean between the groups that advanced to the playoffs and did not advance to the playoffs was tested, and then to compensate for this, the average difference between the three groups (higher/middle/lower) based on ranking was reconfirmed. Of the input data, only this year's season data was used as a test set, and 5-fold cross-validation was performed by dividing the training set and the validation set for model training. The overfitting problem was solved by comparing the cross-validation result and the final analysis result using the test set to confirm that there was no difference in the performance matrix. Because the quality level of the raw data is high and the statistical assumptions are satisfied, most of the models showed good results despite the small data set. This study not only predicts NBA game results or classifies whether or not to advance to the playoffs using machine learning, but also examines whether the variables of interest are included in the major variables with high importance by understanding the importance of input attribute. Through the visualization of SHAP value, it was possible to overcome the limitation that could not be interpreted only with the result of feature importance, and to compensate for the lack of consistency in the importance calculation in the process of entering/removing variables. It was found that a number of variables related to three points and errors classified as subjects of interest in this study were included in the major variables affecting advancing to the playoffs in the NBA. Although this study is similar in that it includes topics such as match results, playoffs, and championship predictions, which have been dealt with in the existing sports data analysis field, and comparatively analyzed several machine learning models for analysis, there is a difference in that the interest features are set in advance and statistically verified, so that it is compared with the machine learning analysis result. Also, it was differentiated from existing studies by presenting explanatory visualization results using SHAP, one of the XAI models.

Motor Imagery EEG Classification Method using EMD and FFT (EMD와 FFT를 이용한 동작 상상 EEG 분류 기법)

  • Lee, David;Lee, Hee-Jae;Lee, Sang-Goog
    • Journal of KIISE
    • /
    • v.41 no.12
    • /
    • pp.1050-1057
    • /
    • 2014
  • Electroencephalogram (EEG)-based brain-computer interfaces (BCI) can be used for a number of purposes in a variety of industries, such as to replace body parts like hands and feet or to improve user convenience. In this paper, we propose a method to decompose and extract motor imagery EEG signal using Empirical Mode Decomposition (EMD) and Fast Fourier Transforms (FFT). The EEG signal classification consists of the following three steps. First, during signal decomposition, the EMD is used to generate Intrinsic Mode Functions (IMFs) from the EEG signal. Then during feature extraction, the power spectral density (PSD) is used to identify the frequency band of the IMFs generated. The FFT is used to extract the features for motor imagery from an IMF that includes mu rhythm. Finally, during classification, the Support Vector Machine (SVM) is used to classify the features of the motor imagery EEG signal. 10-fold cross-validation was then used to estimate the generalization capability of the given classifier., and the results show that the proposed method has an accuracy of 84.50% which is higher than that of other methods.

A Study on Predicting Cryptocurrency Distribution Prices Using Machine Learning Techniques (머신러닝 기법을 활용한 암호화폐 유통 가격 예측 연구)

  • KIM, Han-Min;KIM, Hoik
    • Journal of Distribution Science
    • /
    • v.17 no.11
    • /
    • pp.93-101
    • /
    • 2019
  • Purpose: Blockchain technology suggests ways to solve the problems in the existing industry. Among them, Cryptocurrency system, which is an element of Blockchain technology, is a very important factor for operating Blockchain. While Blockchain cryptocurrency has attracted attention, studies on cryptocurrency prices have been mainly conducted, however previous studies mainly conducted on Bitcoin prices. On the other hand, in the context of the creation and trading of various cryptocurrencies based on the Blockchain system, little research has been done on cryptocurrencies other than Bitcoin. Hence, this study attempts to find variables related to the prices of Dash, Litecoin, and Monero cryptocurrencies using machine learning techniques. We also attempt to find differences in the variables related to the prices for each cryptocurrencies and to examine machine learning techniques that can provide better performance. Research design, data, and methodology: This study performed Dash, Litecoin, and Monero price prediction analysis of cryptocurrency using Blockchain information and machine learning techniques. We employed number of transactions in Blockchain, amount of generated cryptocurrency, transaction fees, number of activity accounts in Blockchain, Block creation difficulty, block size, umber of created blocks as independent variables. This study tried to ensure the reliability of the analysis results through 10-fold cross validation. Blockchain information was hierarchically added for price prediction, and the analysis result was measured as RMSE and MAPE. Results: The analysis shows that the prices of Dash, Litecoin and Monero cryptocurrency are related to Blockchain information. Also, we found that different Blockchain information improves the analysis results for each cryptocurrency. In addition, this study found that the neural network machine learning technique provides better analysis results than support-vector machine in predicting cryptocurrency prices. Conclusion: This study concludes that the information of Blockchain should be considered for the prediction of the price of Dash, Litecoin, and Monero cryptocurrency. It also suggests that Blockchain information related to the price of cryptocurrency differs depending on the type of cryptocurrency. We suggest that future research on various types of cryptocurrencies is needed. The findings of this study can provide a theoretical basis for future cryptocurrency research in distribution management.

Project Failure Main Factors Analysis using Text Mining in Audit Evaluation (감리결과에 텍스트마이닝 기법을 적용한 프로젝트 실패 주요요인 분석)

  • Jang, Kyoungae;Jang, Seong Yong;Kim, Woo-Je
    • Journal of KIISE
    • /
    • v.42 no.4
    • /
    • pp.468-474
    • /
    • 2015
  • Corporations should make efforts to recognize the importance of projects, identify their failure factors, prevent risks in advance, and raise the success rates, because the corporations need to make quick responses to rapid external changes. There are some previous studies on success and failure factors of projects, however, most of them have limitations in terms of objectivity and quantitative analysis based on data gathering through surveys, statistical sampling and analysis. This study analyzes the failure factors of projects based on data mining to find problems with projects in an audit report, which is an objective project evaluation report. To do this, we identified the texts in the paragraph of suggestions about improvement. We made use of the superior classification algorithms in this study, which were NaiveBayes, SMO and J48. They were evaluated in terms of data of Recall and Precision after performing 10-fold-cross validation. In the identified texts, the failure factors of projects were analyzed so that they could be utilized in project implementation.

Named Entity Recognition for Patent Documents Based on Conditional Random Fields (조건부 랜덤 필드를 이용한 특허 문서의 개체명 인식)

  • Lee, Tae Seok;Shin, Su Mi;Kang, Seung Shik
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.9
    • /
    • pp.419-424
    • /
    • 2016
  • Named entity recognition is required to improve the retrieval accuracy of patent documents or similar patents in the claims and patent descriptions. In this paper, we proposed an automatic named entity recognition for patents by using a conditional random field that is one of the best methods in machine learning research. Named entity recognition system has been constructed from the training set of tagged corpus with 660,000 words and 70,000 words are used as a test set for evaluation. The experiment shows that the accuracy is 93.6% and the Kappa coefficient is 0.67 between manual tagging and automatic tagging system. This figure is better than the Kappa coefficient 0.6 for manually tagged results and it shows that automatic named entity tagging system can be used as a practical tagging for patent documents in replacement of a manual tagging.

A Study on Classification Models for Predicting Bankruptcy Based on XAI (XAI 기반 기업부도예측 분류모델 연구)

  • Jihong Kim;Nammee Moon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.12 no.8
    • /
    • pp.333-340
    • /
    • 2023
  • Efficient prediction of corporate bankruptcy is an important part of making appropriate lending decisions for financial institutions and reducing loan default rates. In many studies, classification models using artificial intelligence technology have been used. In the financial industry, even if the performance of the new predictive models is excellent, it should be accompanied by an intuitive explanation of the basis on which the result was determined. Recently, the US, EU, and South Korea have commonly presented the right to request explanations of algorithms, so transparency in the use of AI in the financial sector must be secured. In this paper, an artificial intelligence-based interpretable classification prediction model was proposed using corporate bankruptcy data that was open to the outside world. First, data preprocessing, 5-fold cross-validation, etc. were performed, and classification performance was compared through optimization of 10 supervised learning classification models such as logistic regression, SVM, XGBoost, and LightGBM. As a result, LightGBM was confirmed as the best performance model, and SHAP, an explainable artificial intelligence technique, was applied to provide a post-explanation of the bankruptcy prediction process.

An Automatic Coding System of Korean Standard Industry/Occupation Code Using Example-based Learning (예제기반의 학습을 이용한 한국어 표준 산업/직업 자동 코딩 시스템)

  • Lim Heui-Seok
    • The Journal of the Korea Contents Association
    • /
    • v.5 no.4
    • /
    • pp.169-179
    • /
    • 2005
  • Standard industry and occupation code are usually assigned manually in Korean census. The manual coding is very labor intensive and expensive task. Furthermore, inconsistent coding is resulted from the ability of human experts and their working environments. This paper proposes an automatic code classification system which converts natural language responses on survey questionnaires into corresponding numeric codes by using manually constructed rule base and example-based machine learning. The system was trained with 400,000 records of which standard codes was assigned. It was evaluated with 10-fold cross validation and was tested with three code sets: population occupation set, industry set, and industry survey set. The proposed system showed 76.63%, 82.24 and 99.68% accuracy for each code set.

  • PDF