• Title/Summary/Keyword: Cross validation technique

Search Result 126, Processing Time 0.021 seconds

Restricted support vector quantile regression without crossing

  • Shim, Joo-Yong;Lee, Jang-Taek
    • Journal of the Korean Data and Information Science Society
    • /
    • v.21 no.6
    • /
    • pp.1319-1325
    • /
    • 2010
  • Quantile regression provides a more complete statistical analysis of the stochastic relationships among random variables. Sometimes quantile functions estimated at different orders can cross each other. We propose a new non-crossing quantile regression method applying support vector median regression to restricted regression quantile, restricted support vector quantile regression. The proposed method provides a satisfying solution to estimating non-crossing quantile functions when multiple quantiles for high dimensional data are needed. We also present the model selection method that employs cross validation techniques for choosing the parameters which aect the performance of the proposed method. One real example and a simulated example are provided to show the usefulness of the proposed method.

IRF-k kriging of electrical resistivity data for estimating the extent of saltwater intrusion in a coastal aquifer system

  • Shim B. O.;Chung S. Y.;Kim H. J.;Sung I. H.
    • 한국지구물리탐사학회:학술대회논문집
    • /
    • 2003.11a
    • /
    • pp.352-361
    • /
    • 2003
  • We have evaluated the extent of saltwater intrusion from electrical resistivity distribution in a coastal aquifer system in the southeastern part of Busan, Korea. This aquifer system is divided into four layers according to the hydrogeologic characteristics and the horizontal extent of intruded saltwater is determined at each layer through the geostatistical interpretation of electrical resistivity data. In order to define the statistical structure of electrical resistivity data, variogram analysis is carried out to obtain best generalized covariance models. IRF-k (intrinsic random function of order k) kriging is performed with covariance models to produce the plane of spatial mean resistivities. The kriged estimates are evaluated by cross validation to show a good agreement with the true values and the statistics of cross validation represented low errors for the estimates. In the resistivity contour maps more than 5 m below the surface, we can see a dominant direction of saltwater intrusion beginning from the east side. The area of saltwater intrusion increases with depth. The northeast side has low resistivities less than 5 ohm-m due to the presence of saline water in the depth range of 20 m through 70 m. These results show that the application of geostatistical technique to electrical resistivity data is useful for assessing saltwater intrusion in a coastal aquifer system.

  • PDF

Development of Highway Traffic Information Prediction Models Using the Stacking Ensemble Technique Based on Cross-validation (스태킹 앙상블 기법을 활용한 고속도로 교통정보 예측모델 개발 및 교차검증에 따른 성능 비교)

  • Yoseph Lee;Seok Jin Oh;Yejin Kim;Sung-ho Park;Ilsoo Yun
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.22 no.6
    • /
    • pp.1-16
    • /
    • 2023
  • Accurate traffic information prediction is considered to be one of the most important aspects of intelligent transport systems(ITS), as it can be used to guide users of transportation facilities to avoid congested routes. Various deep learning models have been developed for accurate traffic prediction. Recently, ensemble techniques have been utilized to combine the strengths and weaknesses of various models in various ways to improve prediction accuracy and stability. Therefore, in this study, we developed and evaluated a traffic information prediction model using various deep learning models, and evaluated the performance of the developed deep learning models as a stacking ensemble. The individual models showed error rates within 10% for traffic volume prediction and 3% for speed prediction. The ensemble model showed higher accuracy compared to other models when no cross-validation was performed, and when cross-validation was performed, it showed a uniform error rate in long-term forecasting.

A Study on Random Selection of Pooling Operations for Regularization and Reduction of Cross Validation (정규화 및 교차검증 횟수 감소를 위한 무작위 풀링 연산 선택에 관한 연구)

  • Ryu, Seo-Hyeon
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.19 no.4
    • /
    • pp.161-166
    • /
    • 2018
  • In this paper, we propose a method for the random selection of pooling operations for the regularization and reduction of cross validation in convolutional neural networks. The pooling operation in convolutional neural networks is used to reduce the size of the feature map and for its shift invariant properties. In the existing pooling method, one pooling operation is applied in each pooling layer. Because this method fixes the convolution network, the network suffers from overfitting, which means that it excessively fits the models to the training samples. In addition, to find the best combination of pooling operations to maximize the performance, cross validation must be performed. To solve these problems, we introduce the probability concept into the pooling layers. The proposed method does not select one pooling operation in each pooling layer. Instead, we randomly select one pooling operation among multiple pooling operations in each pooling region during training, and for testing purposes, we use probabilistic weighting to produce the expected output. The proposed method can be seen as a technique in which many networks are approximately averaged using a different pooling operation in each pooling region. Therefore, this method avoids the overfitting problem, as well as reducing the amount of cross validation. The experimental results show that the proposed method can achieve better generalization performance and reduce the need for cross validation.

Spatial-Temporal Modelling of Road Traffic Data in Seoul City

  • Lee, Sang-Yeol;Ahn, Soo-Han;Park, Chang-Yi;Jeon, Jong-Woo
    • Journal of the Korean Data and Information Science Society
    • /
    • v.13 no.2
    • /
    • pp.261-270
    • /
    • 2002
  • Recently, the demand of the Intelligent Transportation System(ITS) has been increased to a large extent, and a real-time traffic information service based on the internet system became very important. When ITS companies carry out real-time traffic services, they find some traffic data missing, and use the conventional method of reconstructing missing values by calculating average time trend. However, the method is found unsatisfactory, so that we develop a new method based the spatial and spatial-temporal models. A cross-validation technique shows that the spatial-temporal model outperforms the others.

  • PDF

Logistic Regression Method in Interval-Censored Data

  • Yun, Eun-Young;Kim, Jin-Mi;Ki, Choong-Rak
    • The Korean Journal of Applied Statistics
    • /
    • v.24 no.5
    • /
    • pp.871-881
    • /
    • 2011
  • In this paper we propose a logistic regression method to estimate the survival function and the median survival time in interval-censored data. The proposed method is motivated by the data augmentation technique with no sacrifice in augmenting data. In addition, we develop a cross validation criterion to determine the size of data augmentation. We compare the proposed estimator with other existing methods such as the parametric method, the single point imputation method, and the nonparametric maximum likelihood estimator through extensive numerical studies to show that the proposed estimator performs better than others in the sense of the mean squared error. An illustrative example based on a real data set is given.

Prediction of retention of uncharged solutes in nanofiltration by means of molecular descriptors

  • Nowaczyk, Alicja;Nowaczyk, Jacek;Koter, Stanislaw
    • Membrane and Water Treatment
    • /
    • v.1 no.3
    • /
    • pp.181-192
    • /
    • 2010
  • A linear quantitative structure-property relationship (QSPR) model is presented for the prediction of rejection in permeation through membrane. The model was produced by using the multiple linear regression (MLR) technique on the database consisting of retention data of 25 pesticides in 4 different membrane separation experiments. Among the 3224 different physicochemical, topological and structural descriptors that were considered as inputs to the model only 50 were selected using several criteria of elimination. The physical meaning of chosen descriptor is discussed in detail. The accuracy of the proposed MLR models is illustrated using the following evaluation techniques: leave-one-out cross validation procedure, leave-many-out cross validation procedure and Y-randomization.

Claims Reserving via Kernel Machine

  • Kim, Mal-Suk;Park, He-Jung;Hwang, Chang-Ha;Shim, Joo-Yong
    • Journal of the Korean Data and Information Science Society
    • /
    • v.19 no.4
    • /
    • pp.1419-1427
    • /
    • 2008
  • This paper shows the kernel Poisson regression which can be applied in the claims reserving, where the row effect is assumed to be a nonlinear function of the row index. The paper concentrates on the chain-ladder technique, within the framework of the chain-ladder linear model. It is shown that the proposed method can provide better reserve estimates than the Poisson model. The cross validation function is introduced to choose optimal hyper-parameters in the procedure. Experimental results are then presented which indicate the performance of the proposed model.

  • PDF

Development of a High Resolution Cinematic Particle Image Velocimetry and Its Application to measurement of Unsteady Complex Turbulent Flows (고분해능 Cinematic PIV 시스템의 개발과 비정상 복잡 난류유동측정에의 응용)

  • Kim, Kyung-Chun;Park, Kyung-Hyun
    • Proceedings of the KSME Conference
    • /
    • 2001.06e
    • /
    • pp.536-541
    • /
    • 2001
  • A high resolution digital cinematic Particle Image Velocimetry(PIV) has been developed. The system consists of a high speed CCD camera, a continuous Ar-ion laser and a computer with camera controller. To improve the spatial resolution, we adopt a Recursive Technique for velocity interrogation. At first, we obtain a velocity vector for a larger interrogation window size based on the conventional two-frame cross-correlation PIV analysis using the FFT algorithm. Based on the knowing velocity information, more spatially resolved velocity vectors are obtained in the next iteration step with smaller interrogation windows. The correct velocity vector at the first step is found to be critical, so we apply a Multiple Correlation Validation(MCV) technique in order to decrease the spurious vectors. The MCV technique turns out to improve SNR(Signal to Noise Ratio) of the correlation table. The developed cinematic PIV method has been applied to the measurement of the unsteady flow characteristics of a Rushton turbine mixer. A total of 3,245 instantaneous velocity vectors were successfully obtained with 4 ms time resolution. The acquired spatial resolution corresponds the performance of the conventional high resolution digital PIV system using a $1K{\times}1K$ CCD camera.

  • PDF

Development of a High Resolution Digital Cinematic Particle Image Velocimetry (고해상도 Cinematic PIV의 개발)

  • Park, Gyeong-Hyeon;Kim, Gyeong-Cheon
    • Transactions of the Korean Society of Mechanical Engineers B
    • /
    • v.25 no.11
    • /
    • pp.1535-1542
    • /
    • 2001
  • A high resolution digital cinematic Particle Image Velocimetry(PIV) has been developed. The system consists of a high speed CCD camera, a continuous Ar-ion laser and a computer with camera controller. To improve the spatial resolution, we adopt a Recursive Technique for velocity interrogation. At first, we obtain a velocity vector fur a larger interrogation window size based on the conventional two-frame cross-correlation PIV analysis using the FFT algorithm. Based on the knowing velocity information, more spatially resolved velocity vectors are obtained in the next iteration step with smaller interrogation windows. When the correct velocity vector at the first step is found to be critical, a Multiple Correlation Validation(MCV) technique is applied to decrease the spurious vectors. The MCV technique turns out to improve SNR(Signal to Noise Ratio) of the correlation table. The developed cinematic PIV method has been applied to the measurement of the unsteady flow characteristics of a Rushton turbine mixer. A total of 3,245 instantaneous velocity vectors were successfully obtained with 4 ms time resolution. The acquired spatial resolution corresponds to the conventional high resolution digital PIV system using a 1K ${\times}$ 1K CCD camera.