• Title/Summary/Keyword: gradient algorithm

Search Result 1,168, Processing Time 0.025 seconds

Dropout Genetic Algorithm Analysis for Deep Learning Generalization Error Minimization

  • Park, Jae-Gyun;Choi, Eun-Soo;Kang, Min-Soo;Jung, Yong-Gyu
    • International Journal of Advanced Culture Technology
    • /
    • v.5 no.2
    • /
    • pp.74-81
    • /
    • 2017
  • Recently, there are many companies that use systems based on artificial intelligence. The accuracy of artificial intelligence depends on the amount of learning data and the appropriate algorithm. However, it is not easy to obtain learning data with a large number of entity. Less data set have large generalization errors due to overfitting. In order to minimize this generalization error, this study proposed DGA(Dropout Genetic Algorithm) which can expect relatively high accuracy even though data with a less data set is applied to machine learning based genetic algorithm to deep learning based dropout. The idea of this paper is to determine the active state of the nodes. Using Gradient about loss function, A new fitness function is defined. Proposed Algorithm DGA is supplementing stochastic inconsistency about Dropout. Also DGA solved problem by the complexity of the fitness function and expression range of the model about Genetic Algorithm As a result of experiments using MNIST data proposed algorithm accuracy is 75.3%. Using only Dropout algorithm accuracy is 41.4%. It is shown that DGA is better than using only dropout.

A Study of Digital Image Restoration for Modified PEM Gradient Algorithm (변형된 PEM 그래디언트 알고리즘을 이용한 디지털화상처리에 관한 연구)

  • Song, Min-Koo
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.1
    • /
    • pp.313-320
    • /
    • 2000
  • PEM algorithm cannot expend repeated algorithm, if penalty function is transcendental function. However, OSL algorithm has an advantage that repeated algorithm is easily derived, even though penalty function which has a complicated transcendental function. In spite of this advantage, this algorithm is restricted in convergence region of smoothing constant which increase penalized log-likelihood, so we cannot get the optimal image restoration because it cannot provide us with a various smoothing constant value for the digital image restoration. In this paper, in order to resolve the disadvantage of OSL algorithm, we would like to suggest the algorithm with smoothing constant enlarge the tolerance limit range of convergence and to find not only properties of its convergence but also usefulness of suggested algorithm through digital image simulation.

  • PDF

Simultaneous optimization method of feature transformation and weighting for artificial neural networks using genetic algorithm : Application to Korean stock market

  • Kim, Kyoung-jae;Ingoo Han
    • Proceedings of the Korea Inteligent Information System Society Conference
    • /
    • 1999.10a
    • /
    • pp.323-335
    • /
    • 1999
  • In this paper, we propose a new hybrid model of artificial neural networks(ANNs) and genetic algorithm (GA) to optimal feature transformation and feature weighting. Previous research proposed several variants of hybrid ANNs and GA models including feature weighting, feature subset selection and network structure optimization. Among the vast majority of these studies, however, ANNs did not learn the patterns of data well, because they employed GA for simple use. In this study, we incorporate GA in a simultaneous manner to improve the learning and generalization ability of ANNs. In this study, GA plays role to optimize feature weighting and feature transformation simultaneously. Globally optimized feature weighting overcome the well-known limitations of gradient descent algorithm and globally optimized feature transformation also reduce the dimensionality of the feature space and eliminate irrelevant factors in modeling ANNs. By this procedure, we can improve the performance and enhance the generalisability of ANNs.

  • PDF

Time-Varying Two-Phase Optimization and its Application to neural Network Learning (시변 2상 최적화 및 이의 신경회로망 학습에의 응용)

  • Myeong, Hyeon;Kim, Jong-Hwan
    • Journal of the Korean Institute of Telematics and Electronics B
    • /
    • v.31B no.7
    • /
    • pp.179-189
    • /
    • 1994
  • A two-phase neural network finds exact feasible solutions for a constrained optimization programming problem. The time-varying programming neural network is a modified steepest-gradient algorithm which solves time-varying optimization problems. In this paper, we propose a time-varying two-phase optimization neural network which incorporates the merits of the two-phase neural network and the time-varying neural network. The proposed algorithm is applied to system identification and function approximation using a multi-layer perceptron. Particularly training of a multi-layer perceptrion is regarded as a time-varying optimization problem. Our algorithm can also be applied to the case where the weights are constrained. Simulation results prove the proposed algorithm is efficient for solving various optimization problems.

  • PDF

Detection of View Reversal in a Stereo Video

  • Son, Ji Deok;Song, Byung Cheol
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.2 no.5
    • /
    • pp.317-321
    • /
    • 2013
  • This paper proposes a detection algorithm for view reversal in a stereoscopic video using a disparity map and motion vector field. We obtain the disparity map of a stereo image was obtained using a specific stereo matching algorithm and classify the image into the foreground and background. Next, the motion vector field of the image on a block basis was produced using a full search algorithm. Finally, the stereo image was considered to be reversed when the foreground moved toward the background and the covered region was in the foreground. The proposed algorithm achieved a good detection rate when the background was covered sufficiently by its moving foreground.

  • PDF

Application of Self-Organizing Fuzzy Logic Controller to Nuclear Steam Generator Level Control

  • Park, Gee-Yong;Park, Jae-Chang;Kim, Chang-Hwoi;Kim, Jung-So;Jung, Chul-Hwan;Seong, Poong-Hyun
    • Proceedings of the Korean Nuclear Society Conference
    • /
    • 1996.11a
    • /
    • pp.85-90
    • /
    • 1996
  • In this paper, the self-organizing fuzzy logic controller is developed for water level control of steam generator. In comparison with conventional fuzzy logic controllers, this controller performs control task with no control rules at initial and creates control rules as control behavior goes on, and also modifies its control structure when uncertain disturbance is suspected. Selected parameters in the fuzzy logic controller are updated on-line by the gradient descent loaming algorithm based on the performance cost function. This control algorithm is applied to water level control of steam generator model developed by Lee, et al. The computer simulation results confirm good performance of this control algorithm in all power ranges. This control algorithm can be expected to be used for automatic control of feedwater control system in the nuclear power plant with digital instrumentation and control systems.

  • PDF

Lineament Extraction from DEM Using Raindrop Tracing Algorithm

  • Yun, Sang-ho
    • Proceedings of the KSRS Conference
    • /
    • 1999.11a
    • /
    • pp.290-295
    • /
    • 1999
  • Lineament extraction from mountain area often provides valuable geological information. In many cases, the lineaments correspond to a series of continuous large valleys. This paper introduces a new lineament extraction method from Digital Elevation Model (DEM) using Raindrop Tracing Algorithm (RTA). The main advantage of this algorithm over conventional Segment Tracing Algorithm (STA) is that it utilizes DEM directly unlike the STA Which utilizes the shaded relief of DEM. The RTA simulates the real life of raindrops that converge into a large valley. The simulation has been done by sprinkling the randomized raindrops over DEM and counting the number of raindrop path that follows the negative gradient of the DEM. The large counting number indicates the location of a big valley where the raindrops converge. With the help of the counting number array (accumulator array) recording the flowing path information, RTA can produce perfectly unbiased binary image of the lineament.

  • PDF

A Learning Algorithm for Optimal Fuzzy Control Rules (최적의 퍼지제어규칙을 얻기위한 퍼지학습법)

  • Chung, Byeong-Mook
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.20 no.2
    • /
    • pp.399-407
    • /
    • 1996
  • A fuzzy learning algorithm to get the optimal fuzzy rules is presented in this paper. The algorithm introduces a reference model to generate a desired output and a performance index funtion instead of the performance index table. The performance index funtion is a cost function based on the error and error-rate between the reference and plant output. The cost function is minimized by a gradient method and the control input is also updated. In this case, the control rules which generate the desired response can be obtained by changing the portion of the error-rate in the cost funtion. In SISO(Single-Input Single- Output)plant, only by the learning delay, it is possible to experss the plant model and to get the desired control rules. In the long run, this algorithm gives us the good control rules with a minimal amount of prior informaiton about the environment.

A two-level parallel algorithm for material nonlinearity problems

  • Lee, Jeeho;Kim, Min Seok
    • Structural Engineering and Mechanics
    • /
    • v.38 no.4
    • /
    • pp.405-416
    • /
    • 2011
  • An efficient two-level domain decomposition parallel algorithm is suggested to solve large-DOF structural problems with nonlinear material models generating unsymmetric tangent matrices, such as a group of plastic-damage material models. The parallel version of the stabilized bi-conjugate gradient method is developed to solve unsymmetric coarse problems iteratively. In the present approach the coarse DOF system is solved parallelly on each processor rather than the whole system equation to minimize the data communication between processors, which is appropriate to maintain the computing performance on a non-supercomputer level cluster system. The performance test results show that the suggested algorithm provides scalability on computing performance and an efficient approach to solve large-DOF nonlinear structural problems on a cluster system.

A Study on the Retrieval Algorithms for Atmospheric Parameters from FORMOSAT-3/COSMIC Occultation Data

  • Yeh, Wen-Hao;Chiu, Tsen-Chieh;Huang, Cheng-Yung;Liou, Yuei-An
    • Proceedings of the KSRS Conference
    • /
    • v.1
    • /
    • pp.312-315
    • /
    • 2006
  • Radio occultation technique has been used in planetary science to obtain reliable and accurate temperature profiles of the other planets' atmosphere for decades. It relies on the fact that radio waves are bent and delayed due to the gradient of atmospheric refractivity along-ray-path. With the advent of Global Positioning System (GPS), it becomes possible to retrieve the refractivity and temperature profiles of the Earth's atmosphere from the occultation data. We have developed a retrieval algorithm and compared the results of our algorithm with the data of CHAMP to verify the accuracy of our algorithm is good enough. In our algorithm, there are some smoothing steps when retrieving. We analysis the data of FORMOSAT-3 and compare the results with and without smoothing and the results of TACC to see is there any phenomenon deleted after smoothing.

  • PDF