• Title/Summary/Keyword: Q learning

Search Result 426, Processing Time 0.035 seconds

Reliability-aware service chaining mapping in NFV-enabled networks

  • Liu, Yicen;Lu, Yu;Qiao, Wenxin;Chen, Xingkai
    • ETRI Journal
    • /
    • v.41 no.2
    • /
    • pp.207-223
    • /
    • 2019
  • Network function virtualization can significantly improve the flexibility and effectiveness of network appliances via a mapping process called service function chaining. However, the failure of any single virtualized network function causes the breakdown of the entire chain, which results in resource wastage, delays, and significant data loss. Redundancy can be used to protect network appliances; however, when failures occur, it may significantly degrade network efficiency. In addition, it is difficult to efficiently map the primary and backups to optimize the management cost and service reliability without violating the capacity, delay, and reliability constraints, which is referred to as the reliability-aware service chaining mapping problem. In this paper, a mixed integer linear programming formulation is provided to address this problem along with a novel online algorithm that adopts the joint protection redundancy model and novel backup selection scheme. The results show that the proposed algorithm can significantly improve the request acceptance ratio and reduce the consumption of physical resources compared to existing backup algorithms.

Hyper-parameter Optimization for Monte Carlo Tree Search using Self-play

  • Lee, Jin-Seon;Oh, Il-Seok
    • Smart Media Journal
    • /
    • v.9 no.4
    • /
    • pp.36-43
    • /
    • 2020
  • The Monte Carlo tree search (MCTS) is a popular method for implementing an intelligent game program. It has several hyper-parameters that require an optimization for showing the best performance. Due to the stochastic nature of the MCTS, the hyper-parameter optimization is difficult to solve. This paper uses the self-playing capability of the MCTS-based game program for optimizing the hyper-parameters. It seeks a winner path over the hyper-parameter space while performing the self-play. The top-q longest winners in the winner path compete for the final winner. The experiment using the 15-15-5 game (Omok in Korean name) showed a promising result.

Process for Automatic Requirement Generation in Korean Requirements Documents using NLP Machine Learning (NLP 기계 학습을 사용한 한글 요구사항 문서에서의 요구사항 자동 생성 프로세스)

  • Young Yun Baek;Soo Jin Park;Young Bum Park
    • Journal of the Semiconductor & Display Technology
    • /
    • v.22 no.1
    • /
    • pp.88-93
    • /
    • 2023
  • In software engineering, requirement analysis is an important task throughout the process and takes up a high proportion. However, factors that fail to analyze requirements include communication failure, different understanding of the meaning of requirements, and failure to perform requirements normally. To solve this problem, we derived actors and behaviors using morpheme analysis and BERT algorithms in the Korean requirement document and constructed them as ontologies. A chatbot system with ontology data is constructed to derive a final system event list through Q&A with users. The chatbot system generates the derived system event list as a requirement diagram and a requirement specification and provides it to the user. Through the above system, diagrams and specifications with a level of coverage complied with Korean requirement documents were created.

  • PDF

Predictions of dam inflow on Han-river basin using LSTM (LSTM을 이용한 한강유역 댐유입량 예측)

  • Kim, Jongho;Tran, Trung Duc
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2020.06a
    • /
    • pp.319-319
    • /
    • 2020
  • 최근 데이터 과학의 획기적인 발전 덕분에 딥러닝 (Deep Learning) 알고리즘이 개발되어 다양한 분야에 널리 적용되고 있다. 본 연구에서는 인공신경망 중 하나인 LSTM(Long-Short Term Memory) 네트워크를 사용하여 댐 유입량을 예측하였다. 구체적인 내용으로, (1) LSTM에 필요한 입력 데이터를 효율적으로 사전 처리하는 방법, (2) LSTM의 하이퍼 매개변수를 결정하는 방법 및 (3) 다양한 손실 함수(Loss function)를 선택하고 그 영향을 평가하는 방법 등을 다루었다. 제안된 LSTM 모델은 강우량(R), 댐유입량(Q) 기온(T), 기저유량(BF) 등을 포함한 다양한 입력 변수들의 함수로 가정하였으며, CCF(Cross Correlations), ACF(Autocorrelations) 및 PACF(Partial Autocorrelations) 등의 기법을 사용하여 입력 변수를 결정하였다. 다양한 sequence length를 갖는 (즉 t, t-1, … t-n의 시간 지연을 갖는) 입력 변수를 적용하여 데이터 학습에 최적의 시퀀스 길이를 결정하였다. LSTM 네트워크 모델을 적용하여 2014년부터 2020년까지 한강 유역 9개의 댐 유입량을 추정하였다. 본 연구로부터 댐 유입량을 예측하는 것은 홍수 및 가뭄 통제를 위한 필수 요건들 중 하나이며 수자원 계획 및 관리에 도움이 될 것이다.

  • PDF

Performance Comparison of Deep Reinforcement Learning based Computation Offloading in MEC (MEC 환경에서 심층 강화학습을 이용한 오프로딩 기법의 성능비교)

  • Moon, Sungwon;Lim, Yujin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2022.05a
    • /
    • pp.52-55
    • /
    • 2022
  • 5G 시대에 스마트 모바일 기기가 기하급수적으로 증가하면서 멀티 액세스 엣지 컴퓨팅(MEC)이 유망한 기술로 부상했다. 낮은 지연시간 안에 계산 집약적인 서비스를 제공하기 위해 MEC 서버로 오프로딩하는 특히, 태스크 도착률과 무선 채널의 상태가 확률적인 MEC 시스템 환경에서의 오프로딩 연구가 주목받고 있다. 본 논문에서는 차량의 전력과 지연시간을 최소화하기 위해 로컬 실행을 위한 연산 자원과 오프로딩을 위한 전송 전력을 할당하는 심층 강화학습 기반의 오프로딩 기법을 제안하였다. Deep Deterministic Policy Gradient (DDPG) 기반 기법과 Deep Q-network (DQN) 기반 기법을 차량의 전력 소비량과 큐잉 지연시간 측면에서 성능을 비교 분석하였다.

Development of An Autonomous Medicine Delivery Robot Using Facial Recognition for Unlocking Mechanisms (얼굴인식 알고리즘을 활용한 잠금해제 및 자율주행 약제배송로봇 개발)

  • Yu-Kyeong Kim;Ye-Rin Kim
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.11a
    • /
    • pp.874-875
    • /
    • 2023
  • 본 논문은 COVID-19와 같은 전염병 확산 방지를 위해 비대면 약제배송로봇을 제안한다. 제안한 로봇은 OpenCV와 Q-Learning기반의 모델을 사용하여 실시간 영상처리로 사람의 얼굴을 식별한다. 환자의 얼굴, 나이, 전달 약제 등을 환자 데이터베이스에 등록한다. 카메라로 인식된 환자의 얼굴과 데이터베이스 내 환자의 얼굴이 일치할 경우 잠금장치를 해제시켜 환자의 약제 수령을 허용한다. 또한 어플리케이션을 통해 약제가 올바르게 전달되었는지 2차적으로 확인한다. 따라서 본 논문에서 제안한 로봇은 비대면으로 환자에게 약을 전달함으로써 입원병동에서 발생할 수 있는 전염병 확상의 방지에 효과적으로 기여할 수 있을 것이다.

Novel Reward Function for Autonomous Drone Navigating in Indoor Environment

  • Khuong G. T. Diep;Viet-Tuan Le;Tae-Seok Kim;Anh H. Vo;Yong-Guk Kim
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.11a
    • /
    • pp.624-627
    • /
    • 2023
  • Unmanned aerial vehicles are gaining in popularity with the development of science and technology, and are being used for a wide range of purposes, including surveillance, rescue, delivery of goods, and data collection. In particular, the ability to avoid obstacles during navigation without human oversight is one of the essential capabilities that a drone must possess. Many works currently have solved this problem by implementing deep reinforcement learning (DRL) model. The essential core of a DRL model is reward function. Therefore, this paper proposes a new reward function with appropriate action space and employs dueling double deep Q-Networks to train a drone to navigate in indoor environment without collision.

Classification of Query E-Mail Using Neural Network (신경망을 이용한 사용자 질의 전자 메일 분류)

  • 변영철;홍영보
    • Journal of Korea Multimedia Society
    • /
    • v.7 no.3
    • /
    • pp.438-449
    • /
    • 2004
  • More and more users are using the query e-mail according to the increment of use of internet. The operator of internet site desires the users to check the FAQ and Q&A contents first before sending the query e-mail to the operator However the users try to get the solution for a problem easily by simply sending a query e-mail. Therefore the increment of query e-mail is inevitable, and the site operator is suffering from too heavy loads and spending too much time and cost to reply the query e-mail. In this paper, we are proposing an efficient method of classifying the query e-mail of users automatically by using a neural network. To verify the reasonability of our work, the query e-mails of KORNET are used as the test data, which is actually gathered in KT. A total of 210 learning data and 280 test data were used to test the performance of the proposed approach. From the experiments we got the encouraging result from the view point of application in real life. The proposed approach satisfied the request of users who wanted rapid response for their query e-mail.

  • PDF

Inference of Context-Free Grammars using Binary Third-order Recurrent Neural Networks with Genetic Algorithm (이진 삼차 재귀 신경망과 유전자 알고리즘을 이용한 문맥-자유 문법의 추론)

  • Jung, Soon-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.3
    • /
    • pp.11-25
    • /
    • 2012
  • We present the method to infer Context-Free Grammars by applying genetic algorithm to the Binary Third-order Recurrent Neural Networks(BTRNN). BTRNN is a multiple-layered architecture of recurrent neural networks, each of which is corresponding to an input symbol, and is combined with external stack. All parameters of BTRNN are represented as binary numbers and each state transition is performed with any stack operation simultaneously. We apply Genetic Algorithm to BTRNN chromosomes and obtain the optimal BTRNN inferring context-free grammar of positive and negative input patterns. This proposed method infers BTRNN, which includes the number of its states equal to or less than those of existing methods of Discrete Recurrent Neural Networks, with less examples and less learning trials. Also BTRNN is superior to the recent method of chromosomes representing grammars at recognition time complexity because of performing deterministic state transitions and stack operations at parsing process. If the number of non-terminals is p, the number of terminals q, the length of an input string k, and the max number of BTRNN states m, the parallel processing time is O(k) and the sequential processing time is O(km).

Efficiency Optimization Control of SynRM Drive using Multi-AFLC (다중 AFLC를 이용한 SynRM 드라이브의 효율 최적화 제어)

  • Choi, Jung-Sik;Ko, Jae-Sub;Jang, Mi-Geum;Chung, Dong-Hwa
    • Journal of the Korean Institute of Illuminating and Electrical Installation Engineers
    • /
    • v.24 no.5
    • /
    • pp.44-54
    • /
    • 2010
  • Optimal efficiency control of synchronous reluctance motor(SynRM) is very important in the sense of energy saving and conservation of natural environment because the efficiency of the SynRM is generally lower than that of other types of AC motors. This paper is proposed a novel efficiency optimization control of SynRM considering iron loss using multi adaptive fuzzy learning controller(AFLC). The optimal current ratio between torque current and exciting current is analytically derived to drive SynRM at maximum efficiency. This paper is proposed an efficiency optimization control for the SynRM which minimizes the copper and iron losses. There exists a variety of combinations of d and q-axis current which provide a specific motor torque. The objective of the efficiency optimization control is to seek a combination of d and q-axis current components, which provides minimum losses at a certain operating point in steady state. The control performance of the proposed controller is evaluated by analysis for various operating conditions. Analysis results are presented to show the validity of the proposed algorithm.