• Title/Summary/Keyword: Data Provisioning

Search Result 112, Processing Time 0.019 seconds

An Adaptive Workflow Scheduling Scheme Based on an Estimated Data Processing Rate for Next Generation Sequencing in Cloud Computing

  • Kim, Byungsang;Youn, Chan-Hyun;Park, Yong-Sung;Lee, Yonggyu;Choi, Wan
    • Journal of Information Processing Systems
    • /
    • v.8 no.4
    • /
    • pp.555-566
    • /
    • 2012
  • The cloud environment makes it possible to analyze large data sets in a scalable computing infrastructure. In the bioinformatics field, the applications are composed of the complex workflow tasks, which require huge data storage as well as a computing-intensive parallel workload. Many approaches have been introduced in distributed solutions. However, they focus on static resource provisioning with a batch-processing scheme in a local computing farm and data storage. In the case of a large-scale workflow system, it is inevitable and valuable to outsource the entire or a part of their tasks to public clouds for reducing resource costs. The problems, however, occurred at the transfer time for huge dataset as well as there being an unbalanced completion time of different problem sizes. In this paper, we propose an adaptive resource-provisioning scheme that includes run-time data distribution and collection services for hiding the data transfer time. The proposed adaptive resource-provisioning scheme optimizes the allocation ratio of computing elements to the different datasets in order to minimize the total makespan under resource constraints. We conducted the experiments with a well-known sequence alignment algorithm and the results showed that the proposed scheme is efficient for the cloud environment.

An Efficient Resource Optimization Method for Provisioning on Flash Memory-Based Storage (플래시 메모리 기반 저장장치에서 프로비저닝을 위한 효율적인 자원 최적화 기법)

  • Hyun-Seob Lee
    • Journal of Internet of Things and Convergence
    • /
    • v.9 no.4
    • /
    • pp.9-14
    • /
    • 2023
  • Recently, resource optimization research has been actively conducted in enterprises and data centers to manage the rapid growth of big data. In particular, thin provisioning, which allocates a large number of resources compared to fixedly allocated storage resources, has the effect of reducing initial costs, but as the number of resources actually used increases, the cost effectiveness decreases and the management cost for allocating resources increases. In this paper, we propose a technique that divides the physical blocks of flash memory into single-bit cells and multi-bit cells, formats them with a hybrid technique, and manages them by dividing frequently used hot data and infrequently used cold data. The proposed technique has the advantage that the physical and allocated resources are the same, such as thick provisioning, and can be used without additional cost increase, and the underutilized resources can be managed in multi-bit cell blocks, such as thin provisioning, which can allocate more resources than typical storage devices. Finally, we estimated the resource optimization effectiveness of the proposed technique through experiments based on simulations.

Auto Regulated Data Provisioning Scheme with Adaptive Buffer Resilience Control on Federated Clouds

  • Kim, Byungsang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.10 no.11
    • /
    • pp.5271-5289
    • /
    • 2016
  • On large-scale data analysis platforms deployed on cloud infrastructures over the Internet, the instability of the data transfer time and the dynamics of the processing rate require a more sophisticated data distribution scheme which maximizes parallel efficiency by achieving the balanced load among participated computing elements and by eliminating the idle time of each computing element. In particular, under the constraints that have the real-time and limited data buffer (in-memory storage) are given, it needs more controllable mechanism to prevent both the overflow and the underflow of the finite buffer. In this paper, we propose an auto regulated data provisioning model based on receiver-driven data pull model. On this model, we provide a synchronized data replenishment mechanism that implicitly avoids the data buffer overflow as well as explicitly regulates the data buffer underflow by adequately adjusting the buffer resilience. To estimate the optimal size of buffer resilience, we exploits an adaptive buffer resilience control scheme that minimizes both data buffer space and idle time of the processing elements based on directly measured sample path analysis. The simulation results show that the proposed scheme provides allowable approximation compared to the numerical results. Also, it is suitably efficient to apply for such a dynamic environment that cannot postulate the stochastic characteristic for the data transfer time, the data processing rate, or even an environment where the fluctuation of the both is presented.

The Selection of Measurement Indicators by Spatial Levels for Ecosystem Services Assessment - Focused on the Provisioning Service - (생태계서비스 평가를 위한 공간 수준별 측정지표 선정 - 공급서비스를 중심으로 -)

  • Jung, Pil-Mo;Kim, Jung-In;Yeo, Inae;Joo, Wooyeong;Lee, Kyungeun
    • Journal of the Korean Society of Environmental Restoration Technology
    • /
    • v.24 no.6
    • /
    • pp.67-87
    • /
    • 2021
  • Provisioning service, which is one of the ecosystem service functions, means goods and services such as food and fuel that people get from ecosystem. Provisioning functions are closely related to the primary industry, a sector of economy. Excessive demand and use of human society can cause trade-offs among regulation, cultural, and supporting services. Therefore, it is important to perform evaluation ecosystem services periodically and to monitor the time series fluctuations to identify the impact of provisioning services on other ecosystem services (trade-off) and to maintain sustainable provisioning service. When it comes to the precise assessment of provisioning service, it is necessary to get the statistical data and standardize indicators and methods. In this study, indicators and methods, which are applicable to the spatial level of national-local-protected areas, were derived through literature analysis and expert survey. The result of this study implies that provisioning services measurement by spatial level improve the efficiency of the establishment of environmental conservation plans by whose purpose.

Resource Management Strategies in Fog Computing Environment -A Comprehensive Review

  • Alsadie, Deafallah
    • International Journal of Computer Science & Network Security
    • /
    • v.22 no.4
    • /
    • pp.310-328
    • /
    • 2022
  • Internet of things (IoT) has emerged as the most popular technique that facilitates enhancing humans' quality of life. However, most time sensitive IoT applications require quick response time. So, processing these IoT applications in cloud servers may not be effective. Therefore, fog computing has emerged as a promising solution that addresses the problem of managing large data bandwidth requirements of devices and quick response time. This technology has resulted in processing a large amount of data near the data source compared to the cloud. However, efficient management of computing resources involving balancing workload, allocating resources, provisioning resources, and scheduling tasks is one primary consideration for effective computing-based solutions, specifically for time-sensitive applications. This paper provides a comprehensive review of the source management strategies considering resource limitations, heterogeneity, unpredicted traffic in the fog computing environment. It presents recent developments in the resource management field of the fog computing environment. It also presents significant management issues such as resource allocation, resource provisioning, resource scheduling, task offloading, etc. Related studies are compared indifferent mentions to provide promising directions of future research by fellow researchers in the field.

Scalable Data Provisioning Scheme on Large-Scale Distributed Computing Environment (대규모 분산 컴퓨팅 환경에서 확장성을 고려한 실시간 데이터 공급 기법)

  • Kim, Byungs-Sang;Youn, Chan-Hyun
    • The KIPS Transactions:PartA
    • /
    • v.18A no.4
    • /
    • pp.123-128
    • /
    • 2011
  • As the global grid has grown in size, large-scale distributed data analysis schemes have gained momentum. Over the last few years, a number of methods have been introduced for allocating data intensive tasks across distributed and heterogeneous computing platforms. However, these approaches have a limited potential for scaling up computing nodes so that they can serve more tasks simultaneously. This paper tackles the scalability and communication delay for computing nodes. We propose a distributed data node for storing and allocating the data. This paper also provides data provisioning method based on the steady states for minimizing the communication delay between the data source and the computing nodes. The experimental results show that scalability and communication delay can be achieved in our system.

Time-Efficient Event Processing Using Provisioning-to-Signaling Method in Data Transport Systems Requiring Multiple Processors

  • Kim, Bup-Joong;Ryoo, Jeong-dong;Cho, Kyoungrok
    • ETRI Journal
    • /
    • v.39 no.1
    • /
    • pp.41-50
    • /
    • 2017
  • In connection-oriented data transport services, data loss can occur when a service experiences a problem in its end-to-end path. To resolve the problem promptly, the data transport systems providing the service must quickly modify their internal configurations, which are distributed among different locations within each system. The configurations are modified through a series of problem (event) handling procedures, which are carried out by multiple control processors in the system. This paper proposes a provisioning-to-signaling method for inter-control-processor messaging to improve the time efficiency of event processing. This method simplifies the sharing of the runtime event, and minimizes the time variability caused by the amount of event data, which results in a decrease in the latency time and an increase in the time determinacy when processing global events. The proposed method was tested for an event that required 4,000 internal path changes, and was found to lessen the latency time of global event processing by about 50% compared with the time required for general methods to do the same; in addition, it reduced the impact of the event data on the event processing time to about 30%.

PCIA Cloud Service Modeling and Performance Analysis of Physical & Logical Resource Provisioning (PCIA 클라우드 서비스 모델링 및 자원 구성에 따른 성능 영향도 분석)

  • Yin, Binfeng;Kwak, Jong Wook
    • Journal of the Korea Society of Computer and Information
    • /
    • v.19 no.2
    • /
    • pp.1-10
    • /
    • 2014
  • Cloud computing provides flexible and efficient mass data analysis platform. In this paper, we define a new resource provisioning architecture in the public cloud, named PCIA. In addition, we provide a service model of PCIA and its new naming scheme. Our model selects the proper number of physical or virtual resources based on the requirements of clients. By the analysis of performance variation in the PCIA, we evaluate the relationship between performance variation and resource provisioning, and we present key standards for cloud system constructions, which can be an important resource provisioning criteria for both cloud service providers and clients.

Centralized Scheme for the Provisioning Control in the Synchronous Optical Transport Network (SDH 기반 광 전달망에서 연결 설정을 위한 집중형 제어 체계에 관한 연구)

  • Kang Dongwoo;Kim Dalwon;Cho Kyuseob;Yae Byungho
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.30 no.11B
    • /
    • pp.707-721
    • /
    • 2005
  • Recently, there has been a dramatic increase in data traffic, driven primarily by the explosive growth of the Internet. Optical networking is believed as a key solution to keep up with the growth, thus, the most pressing issue is how to manage and control large optical networks. Currently, provisioning end-to-end connections across the transport network has involved the network operator, leading to long provisioning times in an era when customers are demanding shorter provisioning time. To address this critical issue, new control intelligence is being studied for use within optical networks to shorten provisioning time. Both the IETF and the ITU-T have been aggressively defining many aspects of a control plane for the next generation convergence transport network. Basically, they are based on the distributed control scheme. In this Paper, we suey the applicability of the centralized control scheme for the provisioning control of optical transport network to utilize its inherent advantages over the distributed control scheme. We discuss new central control architecture, and control procedure. Also, we examine the applicability of the existing IETF routing and signaling protocols to the new control concepts, and then, we propose the additional routing and signaling information elements.

Resource Prediction Technique based on Expected Value in Cloud Computing (클라우드 환경에서 기대 값 기반의 동적 자원 예측 기법)

  • Choi, Yeongho;Lim, Yujin
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.4 no.3
    • /
    • pp.81-84
    • /
    • 2015
  • Cloud service is one of major technologies in modern IT business. Due to the dynamics of user demands, service providers need VM(Virtual Machine) provisioning mechanism to predict the amount of resources demanded by cloud users for the next service and to prepare the resources. VM provisioning provides the QoS to cloud user and maximize the revenue of a service provider by minimizing the expense. In this paper, we propose a new VM provisioning technique to minimize the total expense of a service provider by minimizing the expected value of the expense based on the predicted demands of users. To evaluate the effectiveness of our prediction technique, we compare the total expense of our technique with these of the other prediction techniques with a series of real trace data.