• Title/Summary/Keyword: dynamic programming approach

Search Result 127, Processing Time 0.02 seconds

Investigations on data-driven stochastic optimal control and approximate-inference-based reinforcement learning methods (데이터 기반 확률론적 최적제어와 근사적 추론 기반 강화 학습 방법론에 관한 고찰)

  • Park, Jooyoung;Ji, Seunghyun;Sung, Keehoon;Heo, Seongman;Park, Kyungwook
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.25 no.4
    • /
    • pp.319-326
    • /
    • 2015
  • Recently in the fields o f stochastic optimal control ( SOC) and reinforcemnet l earning (RL), there have been a great deal of research efforts for the problem of finding data-based sub-optimal control policies. The conventional theory for finding optimal controllers via the value-function-based dynamic programming was established for solving the stochastic optimal control problems with solid theoretical background. However, they can be successfully applied only to extremely simple cases. Hence, the data-based modern approach, which tries to find sub-optimal solutions utilizing relevant data such as the state-transition and reward signals instead of rigorous mathematical analyses, is particularly attractive to practical applications. In this paper, we consider a couple of methods combining the modern SOC strategies and approximate inference together with machine-learning-based data treatment methods. Also, we apply the resultant methods to a variety of application domains including financial engineering, and observe their performance.

CAPACITY EXPANSION MODELING OF WATER SUPPLY IN A PLANNING SUPPORT SYSTEM FOR URBAN GROWTH MANAGEMENT (도시성장관리를 위한 계획지원체계에서 상수도의 시설확장 모델링)

  • Hyong-Bok, Kim
    • Proceedings of the Korean Association of Geographic Inforamtion Studies Conference
    • /
    • 1995.12a
    • /
    • pp.9-21
    • /
    • 1995
  • A planning support system enhances our ability to use water capacity expansion as an urban growth management strategy. This paper reports the development of capacity expansion modeling of water supply as part of the continuing development of such a planning support system (PEGASUS: Planning Environment for Generation and Analysis of Spatial Urban Systems) to incorporate water supply, This system is designed from the understanding that land use and development drive the demand for infrastructure and infrastructure can have a significant influence on the ways in which land is developed and used. Capacity expansion Problems of water supply can be solved in two ways: 1) optimal control theory, and 2) mixed integer nonlinear programming (MINLP). Each method has its strengths and weaknesses. In this study the MINLP approach is used because of its strength of determining expansion sizing and timing simultaneously. A dynamic network optimization model and a water-distribution network analysis model can address the dynamic interdependence between water planning and land use planning. While the water-distribution network analysis model evaluates the performance of generated networks over time, the dynamic optimization model chooses alternatives to meet expanding water needs. In addition, the user and capacity expansion modeling-to-generate-alternatives (MGA) can generate alternatives. A cost benefit analysis module using a normalization technique helps in choosing the most economical among those alternatives. GIS provide a tool for estimating the volume of demanded water and showing results of the capacity expansion model.

  • PDF

MarSel : LD based tagSNP Selection System for Large-scale SNP Haplotype Dataset (MarSel : 대용량 SNP 일배체형 데이터에 대한 연관불균형기반의 tagSNP 선택 시스템)

  • Kim Sang-Jun;Yeo Sang-Soo;Kim Sung-Kwon
    • The KIPS Transactions:PartA
    • /
    • v.13A no.1 s.98
    • /
    • pp.79-86
    • /
    • 2006
  • Recently the tagSNP selection problem has been researched for reducing the cost of association studies between human's diversities and SNPs. General approach for this problem is that all of SNPs are separated into appropriate blocks and then tagSNPs are chosen in each block. Marsel in this paper is the system that involved the concept of linkage disequilibrium for overcoming the problem that the existing block partitioning approaches have short of biological meanings. In most approaches, the contiguous regions, which recombinations have LD coefficient |D'| and then tagSNP selection step is performed. And MarSel guarantees the minimum tagSNP selection using entropy-based optimal selection algorithm when tagSNPs are chosen in each block, and enables chromosome-level association studies using efficient memory management technique when input is very large-scale dataset that is impossible to be processed in the existing systems.

Development of Part Sales Agent Coupled with Virtual Manufacturing in Internet Environment (인터넷상의 가상생산 기반 부품판매 에이전트 개발)

  • Choi, Hyung-Rim;Kim, Hyun-Soo;Park, Young-Jae;Park, Byoung-Joo;Lee, Kyoung-Jun
    • Asia pacific journal of information systems
    • /
    • v.12 no.4
    • /
    • pp.193-213
    • /
    • 2002
  • The sales activity of most of small manufacturing companies is based on orders of buyers. The process of promotion, negotiation, receipt and selection of orders of the manufacturers is closely coupled with the load status of the production lines. The decision on whether to accept an order or not, or the selection of optimal order set among excessive orders is entirely dependent on the schedule of production lines. However, in the real world, since the production scheduling activity is mainly performed by human experts, most of small manufacturers are suffer from being unable to meet due dates, lack of rapid decision on the acceptance of new order. Recently, Internet based Electronic Commerce is recognized as one of the alternatives for strengthening sales power of small and medium companies. However, small and medium manufacturers can't adjust properly to the new environment because they are in short of money, personnel, and technology. To cope with this problem, this paper deals with development of part sales agent coupled with virtual manufacturing in Internet environment that consist of selection agent, advertisement agent, selection agent, negotiation agent, and virtual manufacturing system. This paper develops a time-bounded negotiation mechanism for small and medium manufacturers in agent-based automated negotiation between customers and negotiation agents. Furthermore, to select optimal order set maximized profit, we first formulate the order selection problem with mixed integer programming, but the computation time of IP is not acceptable for real world scale problem. To overcome this problem and dynamic nature of virtual manufacturing, we suggest a genetic algorithm approach, which shows a reasonable computation time for real world case and good incremental problem solving capability.

An Optimal Investment Planning Model for Improving the Reliability of Layered Air Defense System based on a Network Model (다층 대공방어 체계의 신뢰도 향상을 위한 네트워크 모델 기반의 최적 투자 계획 모델)

  • Lee, Jinho;Chung, Suk-Moon
    • Journal of the Korea Society for Simulation
    • /
    • v.26 no.3
    • /
    • pp.105-113
    • /
    • 2017
  • This study considers an optimal investment planning for improving survivability from an air threat in the layered air defense system. To establish an optimization model, we first represent the layered air defense system as a network model, and then, present two optimization models minimizing the failure probability of counteracting an air threat subject to budget limitation, in which one deals with whether to invest and the other enables continuous investment on the subset of nodes. Nonlinear objective functions are linearized using log function, and we suggest dynamic programming algorithm and linear programing for solving the proposed models. After designing a layered air defense system based on a virtual scenario, we solve the two optimization problems and analyze the corresponding optimal solutions. This provides necessity and an approach for an effective investment planning of the layered air defense system.

A Semantic-Based Mashup Development Tool Supporting Various Open API Types (다양한 Open API 타입들을 지원하는 시맨틱 기반 매쉬업 개발 툴)

  • Lee, Yong-Ju
    • Journal of Internet Computing and Services
    • /
    • v.13 no.3
    • /
    • pp.115-126
    • /
    • 2012
  • Mashups have become very popular over the last few years, and their use also varies for IT convergency services. In spite of their popularity, there are several challenging issues when combining Open APIs into mashups, First, since portal sites may have a large number of APIs available for mashups, manually searching and finding compatible APIs can be a tedious and time-consuming task. Second, none of the existing portal sites provides a way to leverage semantic techniques that have been developed to assist users in locating and integrating APIs like those seen in traditional SOAP-based web services. Third, although suitable APIs have been discovered, the integration of these APIs is required for in-depth programming knowledge. To solve these issues, we first show that existing techniques and algorithms used for finding and matching SOAP-based web services can be reused, with only minor changes. Next, we show how the characteristics of APIs can be syntactically defined and semantically described, and how to use the syntactic and semantic descriptions to aid the easy discovery and composition of Open APIs. Finally, we propose a goal-directed interactive approach for the dynamic composition of APIs, where the final mashup is gradually generated by a forward chaining of APIs. At each step, a new API is added to the composition.

Analytical Determination of Optimal Transit Stop Spacing (최적 정류장 간격의 해석적 연구)

  • Park, Jun-Sik;Go, Seung-Yeong;Lee, Cheong-Won;Kim, Jeom-San
    • Journal of Korean Society of Transportation
    • /
    • v.25 no.3
    • /
    • pp.145-154
    • /
    • 2007
  • Determining stop spacing is a very important process in transit system planning. This study is involved in an analytical approach to decide the transit stop spacing. Transit stop spacing should be longer as 1) user access speed, 2) user travel time, and 3) dwell time increase, and shorter as 1) passengers (boardings and alightings) and 2) headway increase. In this study, a methodology is proposed to determine transit stop spacing to minimize total cost (user cost plus operator cost) with irregular passenger distribution (boardings and alightings) Without considering in-vehicle passengers, the transit stop spacing should be shorter in the concentrated sections of the passenger distribution than in others to minimize total cost. Through the conceptual analysis, it is verified that the transit stop spacing could be longer as the in-vehicle passengers increase in certain sections. This study proposes a simple practical method to determine transit stop spacing and locations instead of a dynamic programming method which generally includes a complex and difficult calculation. If the space axis is changed to a time axis. the methodology of this study could be expanded to analyze a solution for the transit service (or headway) schedule problem.