• Title/Summary/Keyword: Q-learning algorithm

Search Result 155, Processing Time 0.03 seconds

Development of Autonomous Algorithm Using an Online Feedback-Error Learning Based Neural Network for Nonholonomic Mobile Robots (온라인 피드백 에러 학습을 이용한 이동 로봇의 자율주행 알고리즘 개발)

  • Lee, Hyun-Dong;Myung, Byung-Soo
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.21 no.5
    • /
    • pp.602-608
    • /
    • 2011
  • In this study, a method of designing a neurointerface using neural network (NN) is proposed for controlling nonholonomic mobile robots. According to the concept of virtual master-slave robots, in particular, a partially stable inverse dynamic model of the master robot is acquired online through the NN by applying a feedback-error learning method, in which the feedback controller is assumed to be based on a PD compensator for such a nonholonomic robot. The NN for the online feedback-error learning can composed that the input layer consists of six units for the inputs $x_i$, i=1~6, the hidden layer consists of two hidden units for hidden outputs $o_j$, j=1~2, and the output layer consists of two units for the outputs ${\tau}_k$, k=1~2. A tracking control problem is demonstrated by some simulations for a nonholonomic mobile robot with two-independent driving wheels. The initial q value was set to [0, 5, ${\pi}$].

Development and Validation of a Machine Learning-based Differential Diagnosis Model for Patients with Mild Cognitive Impairment using Resting-State Quantitative EEG (안정 상태에서의 정량 뇌파를 이용한 기계학습 기반의 경도인지장애 환자의 감별 진단 모델 개발 및 검증)

  • Moon, Kiwook;Lim, Seungeui;Kim, Jinuk;Ha, Sang-Won;Lee, Kiwon
    • Journal of Biomedical Engineering Research
    • /
    • v.43 no.4
    • /
    • pp.185-192
    • /
    • 2022
  • Early detection of mild cognitive impairment can help prevent the progression of dementia. The purpose of this study was to design and validate a machine learning model that automatically differential diagnosed patients with mild cognitive impairment and identified cognitive decline characteristics compared to a control group with normal cognition using resting-state quantitative electroencephalogram (qEEG) with eyes closed. In the first step, a rectified signal was obtained through a preprocessing process that receives a quantitative EEG signal as an input and removes noise through a filter and independent component analysis (ICA). Frequency analysis and non-linear features were extracted from the rectified signal, and the 3067 extracted features were used as input of a linear support vector machine (SVM), a representative algorithm among machine learning algorithms, and classified into mild cognitive impairment patients and normal cognitive adults. As a result of classification analysis of 58 normal cognitive group and 80 patients in mild cognitive impairment, the accuracy of SVM was 86.2%. In patients with mild cognitive impairment, alpha band power was decreased in the frontal lobe, and high beta band power was increased in the frontal lobe compared to the normal cognitive group. Also, the gamma band power of the occipital-parietal lobe was decreased in mild cognitive impairment. These results represented that quantitative EEG can be used as a meaningful biomarker to discriminate cognitive decline.

A DQN-based Two-Stage Scheduling Method for Real-Time Large-Scale EVs Charging Service

  • Tianyang Li;Yingnan Han;Xiaolong Li
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.3
    • /
    • pp.551-569
    • /
    • 2024
  • With the rapid development of electric vehicles (EVs) industry, EV charging service becomes more and more important. Especially, in the case of suddenly drop of air temperature or open holidays that large-scale EVs seeking for charging devices (CDs) in a short time. In such scenario, inefficient EV charging scheduling algorithm might lead to a bad service quality, for example, long queueing times for EVs and unreasonable idling time for charging devices. To deal with this issue, this paper propose a Deep-Q-Network (DQN) based two-stage scheduling method for the large-scale EVs charging service. Fine-grained states with two delicate neural networks are proposed to optimize the sequencing of EVs and charging station (CS) arrangement. Two efficient algorithms are presented to obtain the optimal EVs charging scheduling scheme for large-scale EVs charging demand. Three case studies show the superiority of our proposal, in terms of a high service quality (minimized average queuing time of EVs and maximized charging performance at both EV and CS sides) and achieve greater scheduling efficiency. The code and data are available at THE CODE AND DATA.

An efficient hybrid TLBO-PSO-ANN for fast damage identification in steel beam structures using IGA

  • Khatir, S.;Khatir, T.;Boutchicha, D.;Le Thanh, C.;Tran-Ngoc, H.;Bui, T.Q.;Capozucca, R.;Abdel-Wahab, M.
    • Smart Structures and Systems
    • /
    • v.25 no.5
    • /
    • pp.605-617
    • /
    • 2020
  • The existence of damages in structures causes changes in the physical properties by reducing the modal parameters. In this paper, we develop a two-stages approach based on normalized Modal Strain Energy Damage Indicator (nMSEDI) for quick applications to predict the location of damage. A two-dimensional IsoGeometric Analysis (2D-IGA), Machine Learning Algorithm (MLA) and optimization techniques are combined to create a new tool. In the first stage, we introduce a modified damage identification technique based on frequencies using nMSEDI to locate the potential of damaged elements. In the second stage, after eliminating the healthy elements, the damage index values from nMSEDI are considered as input in the damage quantification algorithm. The hybrid of Teaching-Learning-Based Optimization (TLBO) with Artificial Neural Network (ANN) and Particle Swarm Optimization (PSO) are used along with nMSEDI. The objective of TLBO is to estimate the parameters of PSO-ANN to find a good training based on actual damage and estimated damage. The IGA model is updated using experimental results based on stiffness and mass matrix using the difference between calculated and measured frequencies as objective function. The feasibility and efficiency of nMSEDI-PSO-ANN after finding the best parameters by TLBO are demonstrated through the comparison with nMSEDI-IGA for different scenarios. The result of the analyses indicates that the proposed approach can be used to determine correctly the severity of damage in beam structures.

MCMC Algorithm for Dirichlet Distribution over Gridded Simplex (그리드 단체 위의 디리슐레 분포에서 마르코프 연쇄 몬테 칼로 표집)

  • Sin, Bong-Kee
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.1
    • /
    • pp.94-99
    • /
    • 2015
  • With the recent machine learning paradigm of using nonparametric Bayesian statistics and statistical inference based on random sampling, the Dirichlet distribution finds many uses in a variety of graphical models. It is a multivariate generalization of the gamma distribution and is defined on a continuous (K-1)-simplex. This paper presents a sampling method for a Dirichlet distribution for the problem of dividing an integer X into a sequence of K integers which sum to X. The target samples in our problem are all positive integer vectors when multiplied by a given X. They must be sampled from the correspondingly gridded simplex. In this paper we develop a Markov Chain Monte Carlo (MCMC) proposal distribution for the neighborhood grid points on the simplex and then present the complete algorithm based on the Metropolis-Hastings algorithm. The proposed algorithm can be used for the Markov model, HMM, and Semi-Markov model for accurate state-duration modeling. It can also be used for the Gamma-Dirichlet HMM to model q the global-local duration distributions.

Smart Anti-jamming Mobile Communication for Cloud and Edge-Aided UAV Network

  • Li, Zhiwei;Lu, Yu;Wang, Zengguang;Qiao, Wenxin;Zhao, Donghao
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.12
    • /
    • pp.4682-4705
    • /
    • 2020
  • The Unmanned Aerial Vehicles (UAV) networks consisting of low-cost UAVs are very vulnerable to smart jammers that can choose their jamming policies based on the ongoing communication policies accordingly. In this article, we propose a novel cloud and edge-aided mobile communication scheme for low-cost UAV network against smart jamming. The challenge of this problem is to design a communication scheme that not only meets the requirements of defending against smart jamming attack, but also can be deployed on low-cost UAV platforms. In addition, related studies neglect the problem of decision-making algorithm failure caused by intermittent ground-to-air communication. In this scheme, we use the policy network deployed on the cloud and edge servers to generate an emergency policy tables, and regularly update the generated policy table to the UAVs to solve the decision-making problem when communications are interrupted. In the operation of this communication scheme, UAVs need to offload massive computing tasks to the cloud or the edge servers. In order to prevent these computing tasks from being offloaded to a single computing resource, we deployed a lightweight game algorithm to ensure that the three types of computing resources, namely local, edge and cloud, can maximize their effectiveness. The simulation results show that our communication scheme has only a small decrease in the SINR of UAVs network in the case of momentary communication interruption, and the SINR performance of our algorithm is higher than that of the original Q-learning algorithm.

Interface Establishment between Reinforcement Learning Algorithm and External Analysis Program for AI-based Automation of Bridge Design Process (AI기반 교량설계 프로세스 자동화를 위한 강화학습 알고리즘과 외부 해석프로그램 간 인터페이스 구축)

  • Kim, Minsu;Choi, Sanghyun
    • Journal of the Computational Structural Engineering Institute of Korea
    • /
    • v.34 no.6
    • /
    • pp.403-408
    • /
    • 2021
  • Currently, in the design process of civil structures such as bridges, it is common to make final products by repeating the process of redesigning, if the initial design is found to not meet the standards after a structural review. This iterative process extends the design time, and causes inefficient consumption of engineering manpower, which should be put into higher-level design, on simple repetitive mechanical work. This problem can be resolved by automating the design process, but the external analysis program used in the design process has been the biggest obstacle to such automation. In this study, we constructed an AI-based automation system for the bridge design process, including an interface that could control both a reinforcement learning algorithm, and an external analysis program, to replace the repetitive tasks in the current design process. The prototype of the system built in this study was developed for a 2-span RC Rahmen bridge, which is one of the simplest bridge systems. In the future, it is expected that the developed interface system can be utilized as a basic technology for linking the latest AI with other types of bridge designs.

Development of EEG Signals Measurement and Analysis Method based on Timbre (음색 기반 뇌파측정 및 분석기법 개발)

  • Park, Seung-Min;Lee, Young-Hwan;Ko, Kwang-Eun;Sim, Kwee-Bo
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.20 no.3
    • /
    • pp.388-393
    • /
    • 2010
  • Cultural Content Technology(CT, Culture Technology) for the development of cultural industry and the commercialization of technology, cultural contents, media, mount, pass the value chain process and increase the added value of cultural products that are good for all forms of intangible technology. In the field of Culture Technology, Music by analyzing the characteristics of the development of a variety of applications has been studied. Associated with EEG measures and the results of their research in response to musical stimuli are used to detect and study is getting attention. In this paper, the musical stimuli in EEG signals by amplifying the corresponding reaction to the averaging method, ERP (Event-Related Potentials) experiments based on the process of extracting sound methods for removing noise from the ICA algorithm to extract the tone and noise removal according to the results are applied to analyze the characteristics of EEG.

Efficiency Optimization Control of SynRM Drive using Multi-AFLC (다중 AFLC를 이용한 SynRM 드라이브의 효율 최적화 제어)

  • Choi, Jung-Sik;Ko, Jae-Sub;Jang, Mi-Geum;Chung, Dong-Hwa
    • Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
    • /
    • v.24 no.5
    • /
    • pp.44-54
    • /
    • 2010
  • Optimal efficiency control of synchronous reluctance motor(SynRM) is very important in the sense of energy saving and conservation of natural environment because the efficiency of the SynRM is generally lower than that of other types of AC motors. This paper is proposed a novel efficiency optimization control of SynRM considering iron loss using multi adaptive fuzzy learning controller(AFLC). The optimal current ratio between torque current and exciting current is analytically derived to drive SynRM at maximum efficiency. This paper is proposed an efficiency optimization control for the SynRM which minimizes the copper and iron losses. There exists a variety of combinations of d and q-axis current which provide a specific motor torque. The objective of the efficiency optimization control is to seek a combination of d and q-axis current components, which provides minimum losses at a certain operating point in steady state. The control performance of the proposed controller is evaluated by analysis for various operating conditions. Analysis results are presented to show the validity of the proposed algorithm.

Nuclear Magnetic Resonance (NMR)-Based Quantification on Flavor-Active and Bioactive Compounds and Application for Distinguishment of Chicken Breeds

  • Kim, Hyun Cheol;Yim, Dong-Gyun;Kim, Ji Won;Lee, Dongheon;Jo, Cheorun
    • Food Science of Animal Resources
    • /
    • v.41 no.2
    • /
    • pp.312-323
    • /
    • 2021
  • The purpose of this study was to use 1H nuclear magnetic resonance (1H NMR) to quantify taste-active and bioactive compounds in chicken breasts and thighs from Korean native chicken (KNC) [newly developed KNCs (KNC-A, -C, and -D) and commercial KNC-H] and white-semi broiler (WSB) used in Samgye. Further, each breed was differentiated using multivariate analyses, including a machine learning algorithm designed to use metabolic information from each type of chicken obtained using 1H-13C heteronuclear single quantum coherence (2D NMR). Breast meat from KNC-D chickens were superior to those of conventional KNC-H and WSB chickens in terms of both taste-active and bioactive compounds. In the multivariate analysis, meat portions (breast and thigh) and chicken breeds (KNCs and WSB) could be clearly distinguished based on the outcomes of the principal component analysis and partial least square-discriminant analysis (R2=0.945; Q2=0.901). Based on this, we determined the receiver operating characteristic (ROC) curve for each of these components. AUC analysis identified 10 features which could be consistently applied to distinguish between all KNCs and WSB chickens in both breast (0.988) and thigh (1.000) meat without error. Here, both 1H NMR and 2D NMR could successfully quantify various target metabolites which could be used to distinguish between different chicken breeds based on their metabolic profile.