• Title/Summary/Keyword: Stochastic Gradient descent

Search Result 42, Processing Time 0.029 seconds

Hybrid CMA-ES/SPGD Algorithm for Phase Control of a Coherent Beam Combining System and its Performance Analysis by Numerical Simulations (CMA-ES/SPGD 이중 알고리즘을 통한 결맞음 빔 결합 시스템 위상제어 및 동작성능에 대한 전산모사 분석)

  • Minsu, Yeo;Hansol, Kim;Yoonchan, Jeong
    • Korean Journal of Optics and Photonics
    • /
    • v.34 no.1
    • /
    • pp.1-12
    • /
    • 2023
  • In this study, we propose a hybrid phase-control algorithm for multi-channel coherent beam combining (CBC) system by combining the covariant matrix adaption evolution strategy (CMA-ES) and stochastic parallel gradient descent (SPGD) algorithms and analyze its operational performance. The proposed hybrid CMA-ES/SPGD algorithm is a sequential process which initially runs the CMA-ES algorithm until the combined final output intensity reaches a preset interim value, and then switches to running the SPGD algorithm to the end of the whole process. For ideal 7-channel and 19-channel all-fiber-based CBC systems, we have found that the mean convergence time can be reduced by about 10% in comparison with the case when the SPGD algorithm is implemented alone. Furthermore, we analyzed a more realistic situation in which some additional phase noise was introduced in the same CBC system. As a result, it is shown that the proposed algorithm reduces the mean convergence time by about 17% for a 7-channel CBC system and 16-27% for a 19-channel system compared to the existing SPGD alone algorithm. We expect that for implementing a CBC system in a real outdoor environment where phase noise cannot be ignored, the hybrid CMA-ES/SPGD algorithm proposed in this study will be exploited very usefully.

Proof-of-principle Experimental Study of the CMA-ES Phase-control Algorithm Implemented in a Multichannel Coherent-beam-combining System (다채널 결맞음 빔결합 시스템에서 CMA-ES 위상 제어 알고리즘 구현에 관한 원리증명 실험적 연구)

  • Minsu Yeo;Hansol Kim;Yoonchan Jeong
    • Korean Journal of Optics and Photonics
    • /
    • v.35 no.3
    • /
    • pp.107-114
    • /
    • 2024
  • In this study, the feasibility of using the covariance-matrix-adaptation-evolution-strategy (CMA-ES) algorithm in a multichannel coherent-beam-combining (CBC) system was experimentally verified. We constructed a multichannel CBC system utilizing a spatial light modulator (SLM) as a multichannel phase-modulator array, along with a coherent light source at 635 nm, implemented the stochastic-parallel-gradient-descent (SPGD) and CMA-ES algorithms on it, and compared their performances. In particular, we evaluated the characteristics of the CMA-ES and SPGD algorithms in the CBC system in both 16-channel rectangular and 19-channel honeycomb formats. The results of the evaluation showed that the performances of the two algorithms were similar on average, under the given conditions; However, it was verified that under the given conditions the CMA-ES algorithm was able to operate with more stable performance than the SPGD algorithm, as the former had less operational variation with the initial phase setting than the latter. It is emphasized that this study is the first proof-of-principle demonstration of the CMA-ES phase-control algorithm in a multichannel CBC system, to the best of our knowledge, and is expected to be useful for future experimental studies of the effects of additional channel-number increments, or external-phase-noise effects, in multichannel CBC systems based on the CMA-ES phase-control algorithm.

Mean Life Assessment and Prediction of the Failure Probability of Combustion Turbine Generating Unit with Data Analytic Method Based on Aging Failure Data (통계적 분석방법을 이용한 복합화력 발전설비의 평균수명 계산 및 고장확률 예측)

  • Lee, Sung-Hoon;Lee, Seung-Hyuk;Kim, Jin-O
    • The Transactions of the Korean Institute of Electrical Engineers A
    • /
    • v.54 no.10
    • /
    • pp.480-486
    • /
    • 2005
  • This paper proposes a method to consider an aging failure probability and survival probability of power system components, though only aging failure probability has been considered in existing mean life calculation. The estimates of the mean and its standard deviation is calculated by using Weibull distribution, and each estimated parameters is obtained from Data Analytic Method (Type H Censoring). The parameter estimation using Data Analytic Method is simpler and faster than the traditional calculation method using gradient descent algorithm. This paper shows calculation procedure of the mean life and its standard deviation by the proposed method and illustrates that the estimated results are close enough to real historical data of combustion turbine generating units in Korean systems. Also, this paper shows the calculation procedures of a probabilistic failure prediction through a stochastic data analysis. Consequently, the proposed methods would be likely to permit that the new deregulated environment forces utilities to reduce overall costs while maintaining an are-related reliability index.

A Quick Hybrid Atmospheric-interference Compensation Method in a WFS-less Free-space Optical Communication System

  • Cui, Suying;Zhao, Xiaohui;He, Xu;Gu, Haijun
    • Current Optics and Photonics
    • /
    • v.2 no.6
    • /
    • pp.612-622
    • /
    • 2018
  • In wave-front-sensor-less adaptive optics (WFS-less AO) systems, the Jacopo Antonello (JA) method belongs to the model-based class and requires few iterations to achieve acceptable distortion correction. However, this method needs a lot of measurements, especially when it deals with moderate or severe aberration, which is undesired in free-space optical communication (FSOC). On the contrary, the stochastic parallel gradient descent (SPGD) algorithm only requires three time measurements in each iteration, and is widely applied in WFS-less AO systems, even though plenty of iterations are necessary. For better and faster compensation, we propose a WFS-less hybrid approach, borrowing from the JA method to compensate for low-order wave front and from the SPGD algorithm to compensate for residual low-order wave front and high-order wave front. The correction results for this proposed method are provided by simulations to show its superior performance, through comparison of both the Strehl ratio and the convergence speed of the WFS-less hybrid approach to those of the JA method and SPGD algorithm.

Pragmatic Assessment of Optimizers in Deep Learning

  • Ajeet K. Jain;PVRD Prasad Rao ;K. Venkatesh Sharma
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.10
    • /
    • pp.115-128
    • /
    • 2023
  • Deep learning has been incorporating various optimization techniques motivated by new pragmatic optimizing algorithm advancements and their usage has a central role in Machine learning. In recent past, new avatars of various optimizers are being put into practice and their suitability and applicability has been reported on various domains. The resurgence of novelty starts from Stochastic Gradient Descent to convex and non-convex and derivative-free approaches. In the contemporary of these horizons of optimizers, choosing a best-fit or appropriate optimizer is an important consideration in deep learning theme as these working-horse engines determines the final performance predicted by the model. Moreover with increasing number of deep layers tantamount higher complexity with hyper-parameter tuning and consequently need to delve for a befitting optimizer. We empirically examine most popular and widely used optimizers on various data sets and networks-like MNIST and GAN plus others. The pragmatic comparison focuses on their similarities, differences and possibilities of their suitability for a given application. Additionally, the recent optimizer variants are highlighted with their subtlety. The article emphasizes on their critical role and pinpoints buttress options while choosing among them.

A Novel Road Segmentation Technique from Orthophotos Using Deep Convolutional Autoencoders

  • Sameen, Maher Ibrahim;Pradhan, Biswajeet
    • Korean Journal of Remote Sensing
    • /
    • v.33 no.4
    • /
    • pp.423-436
    • /
    • 2017
  • This paper presents a deep learning-based road segmentation framework from very high-resolution orthophotos. The proposed method uses Deep Convolutional Autoencoders for end-to-end mapping of orthophotos to road segmentations. In addition, a set of post-processing steps were applied to make the model outputs GIS-ready data that could be useful for various applications. The optimization of the model's parameters is explained which was conducted via grid search method. The model was trained and implemented in Keras, a high-level deep learning framework run on top of Tensorflow. The results show that the proposed model with the best-obtained hyperparameters could segment road objects from orthophotos at an average accuracy of 88.5%. The results of optimization revealed that the best optimization algorithm and activation function for the studied task are Stochastic Gradient Descent (SGD) and Exponential Linear Unit (ELU), respectively. In addition, the best numbers of convolutional filters were found to be 8 for the first and second layers and 128 for the third and fourth layers of the proposed network architecture. Moreover, the analysis on the time complexity of the model showed that the model could be trained in 4 hours and 50 minutes on 1024 high-resolution images of size $106{\times}106pixels$, and segment road objects from similar size and resolution images in around 14 minutes. The results show that the deep learning models such as Convolutional Autoencoders could be a best alternative to traditional machine learning models for road segmentation from aerial photographs.

Movie Recommendation System based on Latent Factor Model (잠재요인 모델 기반 영화 추천 시스템)

  • Ma, Chen;Kim, Kang-Chul
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.16 no.1
    • /
    • pp.125-134
    • /
    • 2021
  • With the rapid development of the film industry, the number of films is significantly increasing and movie recommendation system can help user to predict the preferences of users based on their past behavior or feedback. This paper proposes a movie recommendation system based on the latent factor model with the adjustment of mean and bias in rating. Singular value decomposition is used to decompose the rating matrix and stochastic gradient descent is used to optimize the parameters for least-square loss function. And root mean square error is used to evaluate the performance of the proposed system. We implement the proposed system with Surprise package. The simulation results shows that root mean square error is 0.671 and the proposed system has good performance compared to other papers.

Optimal Algorithm and Number of Neurons in Deep Learning (딥러닝 학습에서 최적의 알고리즘과 뉴론수 탐색)

  • Jang, Ha-Young;You, Eun-Kyung;Kim, Hyeock-Jin
    • Journal of Digital Convergence
    • /
    • v.20 no.4
    • /
    • pp.389-396
    • /
    • 2022
  • Deep Learning is based on a perceptron, and is currently being used in various fields such as image recognition, voice recognition, object detection, and drug development. Accordingly, a variety of learning algorithms have been proposed, and the number of neurons constituting a neural network varies greatly among researchers. This study analyzed the learning characteristics according to the number of neurons of the currently used SGD, momentum methods, AdaGrad, RMSProp, and Adam methods. To this end, a neural network was constructed with one input layer, three hidden layers, and one output layer. ReLU was applied to the activation function, cross entropy error (CEE) was applied to the loss function, and MNIST was used for the experimental dataset. As a result, it was concluded that the number of neurons 100-300, the algorithm Adam, and the number of learning (iteraction) 200 would be the most efficient in deep learning learning. This study will provide implications for the algorithm to be developed and the reference value of the number of neurons given new learning data in the future.

A Unicode based Deep Handwritten Character Recognition model for Telugu to English Language Translation

  • BV Subba Rao;J. Nageswara Rao;Bandi Vamsi;Venkata Nagaraju Thatha;Katta Subba Rao
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.2
    • /
    • pp.101-112
    • /
    • 2024
  • Telugu language is considered as fourth most used language in India especially in the regions of Andhra Pradesh, Telangana, Karnataka etc. In international recognized countries also, Telugu is widely growing spoken language. This language comprises of different dependent and independent vowels, consonants and digits. In this aspect, the enhancement of Telugu Handwritten Character Recognition (HCR) has not been propagated. HCR is a neural network technique of converting a documented image to edited text one which can be used for many other applications. This reduces time and effort without starting over from the beginning every time. In this work, a Unicode based Handwritten Character Recognition(U-HCR) is developed for translating the handwritten Telugu characters into English language. With the use of Centre of Gravity (CG) in our model we can easily divide a compound character into individual character with the help of Unicode values. For training this model, we have used both online and offline Telugu character datasets. To extract the features in the scanned image we used convolutional neural network along with Machine Learning classifiers like Random Forest and Support Vector Machine. Stochastic Gradient Descent (SGD), Root Mean Square Propagation (RMS-P) and Adaptative Moment Estimation (ADAM)optimizers are used in this work to enhance the performance of U-HCR and to reduce the loss function value. This loss value reduction can be possible with optimizers by using CNN. In both online and offline datasets, proposed model showed promising results by maintaining the accuracies with 90.28% for SGD, 96.97% for RMS-P and 93.57% for ADAM respectively.

Classification of Transport Vehicle Noise Events in Magnetotelluric Time Series Data in an Urban area Using Random Forest Techniques (Random Forest 기법을 이용한 도심지 MT 시계열 자료의 차량 잡음 분류)

  • Kwon, Hyoung-Seok;Ryu, Kyeongho;Sim, Ickhyeon;Lee, Choon-Ki;Oh, Seokhoon
    • Geophysics and Geophysical Exploration
    • /
    • v.23 no.4
    • /
    • pp.230-242
    • /
    • 2020
  • We performed a magnetotelluric (MT) survey to delineate the geological structures below the depth of 20 km in the Gyeongju area where an earthquake with a magnitude of 5.8 occurred in September 2016. The measured MT data were severely distorted by electrical noise caused by subways, power lines, factories, houses, and farmlands, and by vehicle noise from passing trains and large trucks. Using machine-learning methods, we classified the MT time series data obtained near the railway and highway into two groups according to the inclusion of traffic noise. We applied three schemes, stochastic gradient descent, support vector machine, and random forest, to the time series data for the highspeed train noise. We formulated three datasets, Hx, Hy, and Hx & Hy, for the time series data of the large truck noise and applied the random forest method to each dataset. To evaluate the effect of removing the traffic noise, we compared the time series data, amplitude spectra, and apparent resistivity curves before and after removing the traffic noise from the time series data. We also examined the frequency range affected by traffic noise and whether artifact noise occurred during the traffic noise removal process as a result of the residual difference.