• Title/Summary/Keyword: markov decision problem

Search Result 70, Processing Time 0.022 seconds

Korean Speech Segmentation and Recognition by Frame Classification via GMM (GMM을 이용한 프레임 단위 분류에 의한 우리말 음성의 분할과 인식)

  • 권호민;한학용;고시영;허강인
    • Proceedings of the Korea Institute of Convergence Signal Processing
    • /
    • 2003.06a
    • /
    • pp.18-21
    • /
    • 2003
  • In general it has been considered to be the difficult problem that we divide continuous speech into short interval with having identical phoneme quality. In this paper we used Gaussian Mixture Model (GMM) related to probability density to divide speech into phonemes, an initial, medial, and final sound. From them we peformed continuous speech recognition. Decision boundary of phonemes is determined by algorithm with maximum frequency in a short interval. Recognition process is performed by Continuous Hidden Markov Model(CHMM), and we compared it with another phoneme divided by eye-measurement. For the experiments result we confirmed that the method we presented is relatively superior in auto-segmentation in korean speech.

  • PDF

Design and Elucidation of Integrated Forecasting Model for Information Factor Analysis (정보인자분석(情報因子分析)을 위한 통합예측(統合豫測)모델의 설계(設計) 및 해석(解析))

  • Kim, Hong-Jae;Lee, Tae-Hui
    • Journal of Korean Society for Quality Management
    • /
    • v.21 no.1
    • /
    • pp.181-189
    • /
    • 1993
  • Over the past two decades, forecasting has gained widespread acceptance as an integral part of business planning and decision making. Accurate forecasting is a prerequisite to successful planning. Accordingly, recent advances in forecasting techniques are of exceptional value to corporate planners. But most of forecasting mothods are reveal its limit and problem for precision and reliability duing to each relationship for raw data and possibility of explanation for each variable. Therefore, to construct the Integrated Forecasting Model(IFM) for Information Factor Analysis, it shoud be considered that whether law data has time lag and variables are explained. For this. following several method can be used : Least Square Method, Markov Process, Fibonacci series, Auto-Correlation, Cross-Correlation, Serial Correlation and Random Walk Theory. Thus, the unified property of these several functions scales the safety and growth of the system which may be varied time-to-time.

  • PDF

Deep Reinforcement Learning-based Distributed Routing Algorithm for Minimizing End-to-end Delay in MANET (MANET에서 종단간 통신지연 최소화를 위한 심층 강화학습 기반 분산 라우팅 알고리즘)

  • Choi, Yeong-Jun;Seo, Ju-Sung;Hong, Jun-Pyo
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.25 no.9
    • /
    • pp.1267-1270
    • /
    • 2021
  • In this paper, we propose a distributed routing algorithm for mobile ad hoc networks (MANET) where mobile devices can be utilized as relays for communication between remote source-destination nodes. The objective of the proposed algorithm is to minimize the end-to-end communication delay caused by transmission failure with deep channel fading. In each hop, the node needs to select the next relaying node by considering a tradeoff relationship between the link stability and forward link distance. Based on such feature, we formulate the problem with partially observable Markov decision process (MDP) and apply deep reinforcement learning to derive effective routing strategy for the formulated MDP. Simulation results show that the proposed algorithm outperforms other baseline schemes in terms of the average end-to-end delay.

A Study on Deep Reinforcement Learning Framework for DME Pulse Design

  • Lee, Jungyeon;Kim, Euiho
    • Journal of Positioning, Navigation, and Timing
    • /
    • v.10 no.2
    • /
    • pp.113-120
    • /
    • 2021
  • The Distance Measuring Equipment (DME) is a ground-based aircraft navigation system and is considered as an infrastructure that ensures resilient aircraft navigation capability during the event of a Global Navigation Satellite System (GNSS) outage. The main problem of DME as a GNSS back up is a poor positioning accuracy that often reaches over 100 m. In this paper, a novel approach of applying deep reinforcement learning to a DME pulse design is introduced to improve the DME distance measuring accuracy. This method is designed to develop multipath-resistant DME pulses that comply with current DME specifications. In the research, a Markov Decision Process (MDP) for DME pulse design is set using pulse shape requirements and a timing error. Based on the designed MDP, we created an Environment called PulseEnv, which allows the agent representing a DME pulse shape to explore continuous space using the Soft Actor Critical (SAC) reinforcement learning algorithm.

Dynamic Computation Offloading Based on Q-Learning for UAV-Based Mobile Edge Computing

  • Shreya Khisa;Sangman Moh
    • Smart Media Journal
    • /
    • v.12 no.3
    • /
    • pp.68-76
    • /
    • 2023
  • Emerging mobile edge computing (MEC) can be used in battery-constrained Internet of things (IoT). The execution latency of IoT applications can be improved by offloading computation-intensive tasks to an MEC server. Recently, the popularity of unmanned aerial vehicles (UAVs) has increased rapidly, and UAV-based MEC systems are receiving considerable attention. In this paper, we propose a dynamic computation offloading paradigm for UAV-based MEC systems, in which a UAV flies over an urban environment and provides edge services to IoT devices on the ground. Since most IoT devices are energy-constrained, we formulate our problem as a Markov decision process considering the energy level of the battery of each IoT device. We also use model-free Q-learning for time-critical tasks to maximize the system utility. According to our performance study, the proposed scheme can achieve desirable convergence properties and make intelligent offloading decisions.

A Study-on Context-Dependent Acoustic Models to Improve the Performance of the Korea Speech Recognition (한국어 음성인식 성능향상을 위한 문맥의존 음향모델에 관한 연구)

  • 황철준;오세진;김범국;정호열;정현열
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.2 no.4
    • /
    • pp.9-15
    • /
    • 2001
  • In this paper we investigate context dependent acoustic models to improve the performance of the Korean speech recognition . The algorithm are using the Korean phonological rules and decision tree, By Successive State Splitting(SSS) algorithm the Hidden Merkov Netwwork(HM-Net) which is an efficient representation of phoneme-context-dependent HMMs, can be generated automatically SSS is powerful technique to design topologies of tied-state HMMs but it doesn't treat unknown contexts in the training phoneme contexts environment adequately In addition it has some problem in the procedure of the contextual domain. In this paper we adopt a new state-clustering algorithm of SSS, called Phonetic Decision Tree-based SSS (PDT-SSS) which includes contexts splits based on the Korean phonological rules. This method combines advantages of both the decision tree clustering and SSS, and can generated highly accurate HM-Net that can express any contexts To verify the effectiveness of the adopted methods. the experiments are carried out using KLE 452 word database and YNU 200 sentence database. Through the Korean phoneme word and sentence recognition experiments. we proved that the new state-clustering algorithm produce better phoneme, word and continuous speech recognition accuracy than the conventional HMMs.

  • PDF

Integrated Inventory Allocation and Customer Order Admission Control in a Two-stage Supply Chain with Make-to-stock and Make-to-order Facilities (계획생산과 주문생산 시설들로 이루어진 두 단계 공급망에서 재고 할당과 고객주문 수용 통제의 통합적 관리)

  • Kim, Eun-Gab
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.35 no.1
    • /
    • pp.83-95
    • /
    • 2010
  • This paper considers a firm that operates make-to-stock and make-to-order facilities in successive stages. The make-to-stock facility produces components which are consumed by the external market demand as well as the internal make-to-order operation. The make-to-order facility processes customer orders with the option of acceptance or rejection. In this paper, we address the problem of coordinating how to allocate the capacity of the make-to-stock facility to internal and external demands and how to control incoming customer orders at the make-to-order facility so as to maximize the firm's profit subject to the system costs. To deal with this issue, we formulate the problem as a Markov decision process and characterize the structure of the optimal inventory allocation and customer order control. In a numerical experiment, we compare the performance of the optimal policy to the heuristic with static inventory allocation and admission control under different operating conditions of the system.

Optimal Call Control Strategies in a Cellular Mobile Communication System with a Buffer for New Calls (신규호에 대한 지체가 허용된 셀룰라 이동통신시스템에서 최적 호제어 연구)

  • Paik, Chun-hyun;Chung, Yong-joo;Cha, Dong-wan
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.23 no.3
    • /
    • pp.135-151
    • /
    • 1998
  • The demand of large capacity in coming cellular systems makes inevitable the deployment of small cells, rendering more frequent handoff occurrences of calls than in the conventional system. The key issue is then how effectively to reduce the chance of unsuccessful handoffs, since the handoff failure is less desirable than that of a new call attempt. In this study, we consider the control policies which give priority to handoff calls by limiting channel assignment for the originating new calls, and allow queueing the new calls which are rejected at their first attempts. On this system. we propose the problem of finding an optimal call control strategy which optimizes the objective function value, while satisfying the requirements on the handoff/new call blocking probabilities and the new call delay. The objective function takes the most general form to include such well-known performance measures as the weighted average carried traffic and the handoff call blocking probability. The problem is formulated into two different linear programming (LP) models. One is based on the direct employment of steady state equations, and the other uses the theory of semi-Markov decision process. Two LP formulations are competitive each other, having its own strength in the numbers of variables and constraints. Extensive experiments are also conducted to show which call control strategy is optimal under various system environments having different objective functions and traffic patterns.

  • PDF

Resource Allocation Strategy of Internet of Vehicles Using Reinforcement Learning

  • Xi, Hongqi;Sun, Huijuan
    • Journal of Information Processing Systems
    • /
    • v.18 no.3
    • /
    • pp.443-456
    • /
    • 2022
  • An efficient and reasonable resource allocation strategy can greatly improve the service quality of Internet of Vehicles (IoV). However, most of the current allocation methods have overestimation problem, and it is difficult to provide high-performance IoV network services. To solve this problem, this paper proposes a network resource allocation strategy based on deep learning network model DDQN. Firstly, the method implements the refined modeling of IoV model, including communication model, user layer computing model, edge layer offloading model, mobile model, etc., similar to the actual complex IoV application scenario. Then, the DDQN network model is used to calculate and solve the mathematical model of resource allocation. By decoupling the selection of target Q value action and the calculation of target Q value, the phenomenon of overestimation is avoided. It can provide higher-quality network services and ensure superior computing and processing performance in actual complex scenarios. Finally, simulation results show that the proposed method can maintain the network delay within 65 ms and show excellent network performance in high concurrency and complex scenes with task data volume of 500 kbits.

Semi-Supervised Recursive Learning of Discriminative Mixture Models for Time-Series Classification

  • Kim, Minyoung
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.13 no.3
    • /
    • pp.186-199
    • /
    • 2013
  • We pose pattern classification as a density estimation problem where we consider mixtures of generative models under partially labeled data setups. Unlike traditional approaches that estimate density everywhere in data space, we focus on the density along the decision boundary that can yield more discriminative models with superior classification performance. We extend our earlier work on the recursive estimation method for discriminative mixture models to semi-supervised learning setups where some of the data points lack class labels. Our model exploits the mixture structure in the functional gradient framework: it searches for the base mixture component model in a greedy fashion, maximizing the conditional class likelihoods for the labeled data and at the same time minimizing the uncertainty of class label prediction for unlabeled data points. The objective can be effectively imposed as individual mixture component learning on weighted data, hence our mixture learning typically becomes highly efficient for popular base generative models like Gaussians or hidden Markov models. Moreover, apart from the expectation-maximization algorithm, the proposed recursive estimation has several advantages including the lack of need for a pre-determined mixture order and robustness to the choice of initial parameters. We demonstrate the benefits of the proposed approach on a comprehensive set of evaluations consisting of diverse time-series classification problems in semi-supervised scenarios.