• Title/Summary/Keyword: Dynamic storage management

Search Result 104, Processing Time 0.024 seconds

Dynamic Cluster Management of Hadoop Distributed Filesystem (하둡 분산 파일시스템의 동적 클러스터 관리 기법)

  • Ryu, Wooseok
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2016.10a
    • /
    • pp.435-437
    • /
    • 2016
  • Hadoop Distributed File System(HDFS) is a file system for distributed processing of big data by replicating data to distributed data nodes. HDFS cluster shows a great scalability up to thousands of nodes, but it assumes a exclusive node cluster with numerous nodes for the big data processing. Various operational-purpose worker systems used by office are hardly considered as a part of cluster. This paper discusses this problem and proposes a dynamic cluster management technique to increase storage capability and analytic performance of hadoop cluster. The propsed technique can add legacy systems to the cluster and can remove them from the cluster dynamically depending on their availability.

  • PDF

Reservoir Operating System Using Sampling Stochastic Dynamic Programming for the Han River Basin (표본 추계학적 동적계획법을 사용한 한강수계 저수지 운영시스템 개발)

  • Eum, Hyung-Il;Park, Myung-Ky
    • Journal of Korea Water Resources Association
    • /
    • v.43 no.1
    • /
    • pp.67-79
    • /
    • 2010
  • Korea water resources corporation (K-Water) has developed the real-time water resources management system for the Nakdong and the Geum River basin to efficiently operate multi-purpose dams in the basins. This study has extended to the Han River basin for providing an effective ending target storage of a month to the real-time water resources management system using Sampling Stochastic Dynamic Programming (SSDP), consequently increasing the efficiency of the reservoir system. The optimization model were developed for three reservoirs, named Soyang, Chungju, and Hwacheon, with high priority in terms of the amounts of effective capacity and water supply for the basin. The number of storage state variable for each dam to set an optimization problem has been assigned from the results of sensitivity analysis. Compared with the K-water operating policy with the target water supply elevations, the optimization model suggested in this study showed that the shortfalls are decreased by 37.22 MCM/year for the required water demands in the basin, even increasing 171 GWh in hydro electronic power generation. In addition, the result of a reservoir operating system during the drawdown period applied to real situation demonstrates that additional releases for water quality or hydro electronic power generation would be possible during the drawdown period between 2007 and 2008. On the basis of these simulation results, the applicability of the SSDP model and the reservoir operating system is proved. Therefore, the more efficient reservoir operation can be achieved if the reservoir operating system is extended further to other Korean basins.

Optimal LNG Procurement Policy in a Spot Market Using Dynamic Programming (동적 계획법을 이용한 LNG 현물시장에서의 포트폴리오 구성방법)

  • Ryu, Jong-Hyun
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.41 no.3
    • /
    • pp.259-266
    • /
    • 2015
  • Among many energy resources, natural gas has recently received a remarkable amount of attention, particularly from the electrical generation industry. This is in part due to increasing shale gas production, providing an environment-friendly fossil fuel, and high risk of nuclear power. Because South Korea, the world's second largest LNG importing nation after Japan, has no international natural gas pipelines and relies on imports in the form of LNG, the natural gas has been traditionally procured by long term LNG contracts at relatively high price. Thus, there is a need of developing an Asian LNG trading hub, where LNG can be traded at more competitive spot prices. In a natural gas spot market, the amount of natural gas to be bought should be carefully determined considering a limited storage capacity and future pricing dynamics. In this work, the problem to find the optimal amount of natural gas in a spot market is formulated as a Markov decision process (MDP) in risk neutral environment and the optimal base stock policy which depends on a stage and price is established. Taking into account price and demand uncertainties, the basestock target levels are simply approximated from dynamic programming. The simulation results show that the basestock policy can be one of effective ways for procurement of LNG in a spot market.

A Data Mining Approach for a Dynamic Development of an Ontology-Based Statistical Information System

  • Mohamed Hachem Kermani;Zizette Boufaida;Amel Lina Bensabbane;Besma Bourezg
    • Journal of Information Science Theory and Practice
    • /
    • v.11 no.2
    • /
    • pp.67-81
    • /
    • 2023
  • This paper presents a dynamic development of an ontology-based statistical information system supporting the collection, storage, processing, analysis, and the presentation of statistical knowledge at the national scale. To accomplish this, we propose a data mining technique to dynamically collect data relating to citizens from publicly available data sources; the collected data will then be structured, classified, categorized, and integrated into an ontology. Moreover, an intelligent platform is proposed in order to generate quantitative and qualitative statistical information based on the knowledge stored in the ontology. The main aims of our proposed system are to digitize administrative tasks and to provide reliable statistical information to governmental, economic, and social actors. The authorities will use the ontology-based statistical information system for strategic decision-making as it easily collects, produces, analyzes, and provides both quantitative and qualitative knowledge that will help to improve the administration and management of national political, social, and economic life.

Flash-Conscious Storage Management Method for DBMS using Dynamic Log Page Allocation (동적 로그 페이지 할당을 이용한 플래시-고려 DBMS의 스토리지 관리 기법)

  • Song, Seok-Il;Khil, Ki-Jeong;Choi, Kil-Seong
    • Journal of Advanced Navigation Technology
    • /
    • v.14 no.5
    • /
    • pp.767-774
    • /
    • 2010
  • Due to advantages of NAND flash memory such as non-volatility, low access latency, low energy consumption, light weight, small size and shock resistance, it has become a better alternative over traditional magnetic disk drives, and has been widely used. Traditional DBMSs including mobile DBMSs may run on flash memory without any modification by using Flash Translation Layer (FTL), which emulates a random access block device to hide the characteristics of flash memory such as "erase-before-update". However, most existing FTLs are optimized for file systems, not for DBMSs, and traditional DBMSs are not aware of them. Also, traditional DBMSs do not consider the characteristics of flash memory. In this paper, we propose a flash-conscious storage system for DBMSs that utilizes flash memory as a main storage medium, and carefully put the characteristics of flash memory into considerations. The proposed flash-conscious storage system exploits log records to avoid costly update operations. It is shown that the proposed storage system outperforms the state.

상하분해 단체법에서 수정 Forrest-Tomlin 방법의 효율적인 구현

  • 김우제;임성묵;박순달
    • Proceedings of the Korean Operations and Management Science Society Conference
    • /
    • 1998.10a
    • /
    • pp.63-66
    • /
    • 1998
  • In the implementation of the simplex method program, the representation and the maintenance of basis matrix is very important, In the experimental study, we investigates Suhl's idea in the LU factorization and LU update of basis matrix. First, the triangularization of basis matrix is implemented and its efficiency is shown. Second, various technique in the dynamic Markowitz's ordering and threshold pivoting are presented. Third, modified Forrest-Tomlin LU update method exploiting sparsity is presented. Fourth, as a storage scheme of LU factors, Gustavson data structure is explained. Fifth, efficient timing of reinversion is developed. Finally, we show that modified Forrest-Tomlin method with Gustavson data structure is superior more than 30% to the Reid method with linked list data structure.

  • PDF

Development of a decision support system for the yard assignment planning of the import and export containers (수출입 컨테이너 장치장 배정을 위한 소프트웨어의 개발)

  • 김갑환;김홍배;홍봉희;김기영;배종욱;최진오;김두열;이영기;박영만
    • Korean Management Science Review
    • /
    • v.12 no.3
    • /
    • pp.1-15
    • /
    • 1995
  • The Pusan Container Terminal faces a rapid increase in berthing time of container ships as well as in waiting time of external trucks, which is due to an absolute lack of yard space. This research is focused on the development of a decision support system for the planning of the container terminal yard assignment so that the yard space would be utilized most effectively. Efforts should be given to the reasonable assignment of the yard storage and the dynamic adaptation to the ever changing environment. The software introduced here is based on the know-how of the field exports and its framework takes the approach of the hierarchical decision making.

  • PDF

Development of a Packet-Switched Public computer Communication Network -PART 2: KORNET Design and Development of Network Node Processor(NNP) (Packet Switching에 의한 공중 Computer 통신망 개발 연구 -제2부: KORNET의 설계 및 Network Node Processor(NNP)의 개발)

  • 조유제;김희동
    • Journal of the Korean Institute of Telematics and Electronics
    • /
    • v.22 no.6
    • /
    • pp.114-123
    • /
    • 1985
  • This is the second part of the four-part paper describing the development of a packet-switched computer network named the cORNET In this paper, following the first par paper that describes the concepts of the KORNET and the development of the network management center (NMC), wc present the design of the KORNET and the development of the network node processor (NNP) The initial configuration of the KORNET consists of three NNP's and one NMC. We have developed each NNP as a microprocessor-based (Mc68000) multiprocessor system, and implemented the NMC using a super-mini computer (Mv/8000) For the KORNET we use the virtual circuit (VC) method as the packet service strategy and the distributed adaptive routing algorithm to adapt efficiently the variation of node and link status. Also, we use a dynamic buffer management algorithm for efficient storage management. Thc hardware of the NNP system has been designed with emphasis on modularity so that it may be expanded esily . Also, the software of the NNP system has been developed according to the CCITT recommendations X.25, X.3, X.28 and X.29.

  • PDF

Implementation and Application of PML Server for Vehicle Information Management in RFID Environment (RFID 기반의 차량 정보 관리용 PML 서버 구현 및 적용)

  • Oh, Won-Rock;Lee, Sung-Keun
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.2 no.2
    • /
    • pp.83-91
    • /
    • 2007
  • The purpose of this paper is developing a PML server which is capable of saving and managing effectively the information of vehicles using RFID technology in the logistical and distributive environment. The PML server logs and manages the real-time static and dynamic information about the vehicles in EPC code format which is represented by XML schema and transmitted by the RFID reader to the server. It receives information from SAVANT the RFID middleware and transmits the data to the traffic information management application The main system we developed consists of the EPC data registry, query processor, data converter, vehicle product and instance data storage, system management tool and application API parts.

  • PDF

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.