• Journal of KIISE:Software and Applications
• /
• v.30 no.3_4
• /
• pp.326-338
• /
• 2003

The objective of a neural network design and model selection is to construct an optimal network with a good generalization performance. However, training data include noises, and the number of training data is not sufficient, which results in the difference between the true probability distribution and the empirical one. The difference makes the teaming parameters to over-fit only to training data and to deviate from the true distribution of data, which is called the overfitting phenomenon. The overfilled neural network shows good approximations for the training data, but gives bad predictions to untrained new data. As the complexity of the neural network increases, this overfitting phenomenon also becomes more severe. In this paper, by taking statistical viewpoint, we proposed an integrative process for neural network design and model selection method in order to improve generalization performance. At first, by using the natural gradient learning with adaptive regularization, we try to obtain optimal parameters that are not overfilled to training data with fast convergence. By adopting the natural pruning to the obtained optimal parameters, we generate several candidates of network model with different sizes. Finally, we select an optimal model among candidate models based on the Bayesian Information Criteria. Through the computer simulation on benchmark problems, we confirm the generalization and structure optimization performance of the proposed integrative process of teaming and model selection.

• Journal of Korea Water Resources Association
• /
• v.54 no.11
• /
• pp.955-968
• /
• 2021

A water resource plan is routinely based on a natural flow and can be estimated using observed streamflow data or a long-term continuous rainfall-runoff model. However, the watershed with the natural flow is very limited to the upstream area of the dam. In particular, for the ungauged watershed, a rainfall-runoff model is established for the gauged watershed, and the model is then applied to the ungauged watershed by transferring the associated parameters. In this study, the GR4J rainfall-runoff model is mainly used to regionalize the parameters that are estimated from the 14 dam watershed via an optimization process. In terms of optimizing the parameters, the Bayesian approach was applied to consider the uncertainty of parameters quantitatively, and a number of parameter samples obtained from the posterior distribution were used for the regionalization. Here, the relationship between the estimated parameters and the topographical factors was first identified, and the dependencies between them are effectively modeled by a Copula function approach to obtain the regionalized parameters. The predicted streamflow with the use of regionalized parameters showed a good agreement with that of the observed with a correlation of about 0.8. It was found that the proposed regionalized framework is able to effectively simulate streamflow for the ungauged watersheds by the use of the regionalized parameters, along with the associated uncertainty, informed by the basin characteristics.

• Journal of Intelligence and Information Systems
• /
• v.25 no.2
• /
• pp.39-55
• /
• 2019

Recently banks and large financial institutions have introduced lots of Robo-Advisor products. Robo-Advisor is a Robot to produce the optimal asset allocation portfolio for investors by using the financial engineering algorithms without any human intervention. Since the first introduction in Wall Street in 2008, the market size has grown to 60 billion dollars and is expected to expand to 2,000 billion dollars by 2020. Since Robo-Advisor algorithms suggest asset allocation output to investors, mathematical or statistical asset allocation strategies are applied. Mean variance optimization model developed by Markowitz is the typical asset allocation model. The model is a simple but quite intuitive portfolio strategy. For example, assets are allocated in order to minimize the risk on the portfolio while maximizing the expected return on the portfolio using optimization techniques. Despite its theoretical background, both academics and practitioners find that the standard mean variance optimization portfolio is very sensitive to the expected returns calculated by past price data. Corner solutions are often found to be allocated only to a few assets. The Black-Litterman Optimization model overcomes these problems by choosing a neutral Capital Asset Pricing Model equilibrium point. Implied equilibrium returns of each asset are derived from equilibrium market portfolio through reverse optimization. The Black-Litterman model uses a Bayesian approach to combine the subjective views on the price forecast of one or more assets with implied equilibrium returns, resulting a new estimates of risk and expected returns. These new estimates can produce optimal portfolio by the well-known Markowitz mean-variance optimization algorithm. If the investor does not have any views on his asset classes, the Black-Litterman optimization model produce the same portfolio as the market portfolio. What if the subjective views are incorrect? A survey on reports of stocks performance recommended by securities analysts show very poor results. Therefore the incorrect views combined with implied equilibrium returns may produce very poor portfolio output to the Black-Litterman model users. This paper suggests an objective investor views model based on Support Vector Machines(SVM), which have showed good performance results in stock price forecasting. SVM is a discriminative classifier defined by a separating hyper plane. The linear, radial basis and polynomial kernel functions are used to learn the hyper planes. Input variables for the SVM are returns, standard deviations, Stochastics %K and price parity degree for each asset class. SVM output returns expected stock price movements and their probabilities, which are used as input variables in the intelligent views model. The stock price movements are categorized by three phases; down, neutral and up. The expected stock returns make P matrix and their probability results are used in Q matrix. Implied equilibrium returns vector is combined with the intelligent views matrix, resulting the Black-Litterman optimal portfolio. For comparisons, Markowitz mean-variance optimization model and risk parity model are used. The value weighted market portfolio and equal weighted market portfolio are used as benchmark indexes. We collect the 8 KOSPI 200 sector indexes from January 2008 to December 2018 including 132 monthly index values. Training period is from 2008 to 2015 and testing period is from 2016 to 2018. Our suggested intelligent view model combined with implied equilibrium returns produced the optimal Black-Litterman portfolio. The out of sample period portfolio showed better performance compared with the well-known Markowitz mean-variance optimization portfolio, risk parity portfolio and market portfolio. The total return from 3 year-period Black-Litterman portfolio records 6.4%, which is the highest value. The maximum draw down is -20.8%, which is also the lowest value. Sharpe Ratio shows the highest value, 0.17. It measures the return to risk ratio. Overall, our suggested view model shows the possibility of replacing subjective analysts's views with objective view model for practitioners to apply the Robo-Advisor asset allocation algorithms in the real trading fields.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.14 no.2
• /
• pp.130-135
• /
• 2004

This paper presents a new design methods of the short-term load forecasting system (STLFS) using the data mining. The structure of the proposed STLFS is divided into two parts: the Takagi-Sugeno (T-S) fuzzy model-based classifier and predictor The proposed classifier is composed of the Gaussian fuzzy sets in the premise part and the linearized Bayesian classifier in the consequent part. The related parameters of the classifier are easily obtained from the statistic information of the training set. The proposed predictor takes form of the convex combination of the linear time series predictors for each inputs. The problem of estimating the consequent parameters is formulated by the convex optimization problem, which is to minimize the norm distance between the real load and the output of the linear time series estimator. The problem of estimating the premise parameters is to find the parameter value minimizing the error between the real load and the overall output. Finally, to show the feasibility of the proposed method, this paper provides the short-term load forecasting example.

• Journal of Korea Water Resources Association
• /
• v.54 no.8
• /
• pp.617-628
• /
• 2021

Drought is a major natural disaster that causes serious social and economic losses. Local drought forecasts can provide important information for drought preparedness. In this study, we propose a new machine learning model that predicts drought by using historical drought indices and meteorological data from 10 sites from 1981 to 2020 in the southeastern part of the Korean Peninsula, Busan-Ulsan-Gyeongnam. Using Bayesian optimization techniques, a hyper-parameter-tuned Random Forest, XGBoost, and Light GBM model were constructed to predict the evaporative demand drought index on a 6-month time scale after 1-month. The model performance was compared by constructing a single site model and a regional model, respectively. In addition, the possibility of improving the model performance was examined by constructing a fine-tuned model using data from a individual site based on the regional model.

• Nuclear Engineering and Technology
• /
• v.53 no.7
• /
• pp.2334-2340
• /
• 2021

Radioactive particle tracking (RPT) is a minimally invasive nuclear technique that tracks a radioactive particle inside a volume of interest by means of a mathematical location algorithm. During the past decades, many algorithms have been developed including ones based on artificial intelligence techniques. In this study, RPT technique is applied in a simulated test section that employs a simplified mixer filled with concrete, six scintillator detectors and a¹³⁷Cs radioactive particle emitting gamma rays of 662 keV. The test section was developed using MCNPX code, which is a mathematical code based on Monte Carlo simulation, and 3516 different radioactive particle positions (x,y,z) were simulated. Novelty of this paper is the use of a location algorithm based on a deep learning model, more specifically a 6-layers deep rectifier neural network (DRNN), in which hyperparameters were defined using a Bayesian optimization method. DRNN is a type of deep feedforward neural network that substitutes the usual sigmoid based activation functions, traditionally used in vanilla Multilayer Perceptron Networks, for rectified activation functions. Results show the great accuracy of the DRNN in a RPT tracking system. Root mean squared error for x, y and coordinates of the radioactive particle is, respectively, 0.03064, 0.02523 and 0.07653.

• Wind and Structures
• /
• v.36 no.6
• /
• pp.393-404
• /
• 2023

Skewness and kurtosis are important higher-order statistics for simulating non-Gaussian wind pressure series on low-rise buildings, but their predictions are less studied in comparison with those of the low order statistics as mean and rms. The distribution gradients of skewness and kurtosis on roofs are evidently higher than those of mean and rms, which increases their prediction difficulty. The conventional artificial neural networks (ANNs) used for predicting mean and rms show unsatisfactory accuracy in predicting skewness and kurtosis owing to the limited capacity of shallow learning of ANNs. In this work, the deep neural networks (DNNs) model with the ability of deep learning is introduced to predict the skewness and kurtosis on a low-rise building. For obtaining the optimal generalization of the DNNs model, the hyper parameters are automatically determined by Bayesian Optimization (BO). Moreover, for providing a benchmark for future studies on predicting higher order statistics, the data sets for training and testing the DNNs model are extracted from the internationally open NIST-UWO database, and the prediction errors of all taps are comprehensively quantified by various error metrices. The results show that the prediction accuracy in this study is apparently better than that in the literature, since the correlation coefficient between the predicted and experimental results is 0.99 and 0.75 in this paper and the literature respectively. In the untrained cornering wind direction, the distributions of skewness and kurtosis are well captured by DNNs on the whole building including the roof corner with strong non-normality, and the correlation coefficients between the predicted and experimental results are 0.99 and 0.95 for skewness and kurtosis respectively.

• The Journal of Engineering Geology
• /
• v.26 no.4
• /
• pp.485-495
• /
• 2016

To reinforce and improve the soft ground under a breakwater while using materials efficiently, the replacement ratio and leaving periods of surcharge load are optimized probabilistically. The results of Bayesian updating of the random variables using prior information decrease uncertainty by up to 39.8%, and using prior information with more samples results in a sharp decrease in uncertainty. Replacement ratios of 15%-40% are analyzed using First Order Reliability Method and Monte Carlo simulation to optimize the replacement ratio. The results show that replacement ratios of 20% and 25% are acceptable at the column jet grouting area and the granular compaction pile area, respectively. Life cycle costs are also compared to optimize the replacement ratios within allowable ranges. The results show that a range of 20%-30% is the most economical during the total life cycle. This means that initial construction cost, maintenance cost and failure loss cost are minimized during total life cycle. Probabilistic analysis for leaving periods of shows that three months acceptable. Design optimization with respect to life cycle cost is important to minimize maintenance costs and retain the performance of the structures for the required period. Therefore, more case studies that consider the maintenance costs of soil structures are necessary to establish relevant design codes.

• Journal of the Institute of Electronics Engineers of Korea TC
• /
• v.46 no.5
• /
• pp.70-77
• /
• 2009

In an RFID network consisting of a single reader and many tags, a framed and slotted ALOHA, which provides a number of slots for the tags to respond, was introduced for arbitrating a collision among tags' responses. In a framed and slotted ALOHA, the number of slots in each frame should be optimized to attain the maximal efficiency in tag cognizance. While such an optimization necessitates the knowledge about the number of tags, the reader hardly knows it. In this paper, we propose a tag cognizance scheme based on framed and slotted ALOHA, which is characterized by directly taking a Bayes action on the number of slots without estimating the number of tags separately. Specifically, a Bayes action is yielded by solving a decision problem which incorporates the prior distribution the number of tags, the observation on the number of slots in which no tag responds and the loss function reflecting the cognizance rate. Also, a Bayes action in each frame is supported by an evolution of prior distribution for the number of tags. From the simulation results, we observe that the pair of evolving prior distribution and Bayes action forms a robust scheme which attains a certain level of cognizance rate in spite of a high discrepancy between the Due and initially believed numbers of tags. Also, the proposed scheme is confirmed to be able to achieve higher cognizance completion probability than a scheme using classical estimate of the number of tags separately.