• Title/Summary/Keyword: data cache

Search Result 487, Processing Time 0.027 seconds

A Global Buffer Manager for a Shared Disk File System in SAN Clusters (SAN 환경에서 공유 디스크 파일 시스템을 위한 전역 버퍼 관리자)

  • 박선영;손덕주;신범주;김학영;김명준
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.2
    • /
    • pp.134-145
    • /
    • 2004
  • With rapid growth in the amount of data transferred on the Internet, traditional storage systems have reached the limits of their capacity and performance. SAN (Storage Area Network), which connects hosts to disk with the Fibre Channel switches, provides one of the powerful solutions to scale the data storage and servers. In this environment, the maintenance of data consistency among hosts is an important issue because multiple hosts share the files on disks attached to the SAN. To preserve data consistency, each host can execute the disk I/O whenever disk read and write operations are requested. However, frequent disk I/O requests cause the deterioration of the overall performance of a SAN cluster. In this paper, we introduce a SANtopia global buffer manager to improve the performance of a SAN cluster reducing the number of disk I/Os. We describe the design and algorithms of the SANtopia global buffer manager, which provides a buffer cache sharing mechanism among the hosts in the SAN cluster. Micro-benchmark results to measure the performance of block I/O operations show that the global buffer manager achieves speed-up by the factor of 1.8-12.8 compared with the existing method using disk I/O operations. Also, File system micro-benchmark results show that SANtopia file system with the global buffer manager improves performance by the factor of 1.06 in case of directories and 1.14 in case of files compared with the file system without a global buffer manager.

The development of the high effective and stoppageless file system for high performance computing (High Performance Computing 환경을 위한 고성능, 무정지 파일시스템 구현)

  • Park, Yeong-Bae;Choe, Seung-Hwan;Lee, Sang-Ho;Kim, Gyeong-Su;Gong, Yong-Jun
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2004.11a
    • /
    • pp.395-401
    • /
    • 2004
  • In the current high network-centralized computing and enterprising environment, it is getting essential to transmit data reliably at very high rates. Until now previous client/server model based NFS(Network File System) or AFS(Andrew's Files System) have met the various demands but from now couldn't satisfy those of the today's scalable high-performance computing environment. Not only performance but data sharing service redundancy have risen as a serious problem. In case of NFS, the locking issue and cache cause file system to reboot and make problem when it is used simply as ip-take over for H/A service. In case of AFS, it provides file sharing redundancy but it is not possible until the storage supporting redundancy and equipments are prepared. Lustre is an open source based cluster file system developed to meet both demands. Lustre consists of three types of subsystems : MDS(Meta-Data Server) which offers the meta-data services, OST(Objec Storage Targets) which provide file I/O, and Lustre Clients which interact with OST and MDS. These subsystems with message exchanging and pursuing scalable high-performance file system service. In this paper, we compare the transmission speed of gigabytes file between Lustre and NFS on the basis of concurrent users and also present the high availability of the file system by removing more than one OST in operation.

  • PDF

A Caching Strategy Considering Characteristics of Broadcast Algorithm in Hybrid-based Data Broadcast Systems (혼합 데이터 방송 시스템에서 방송 알고리즘의 특성을 고려한 캐싱 전략)

  • Shin Dong Cheon
    • The KIPS Transactions:PartC
    • /
    • v.12C no.2 s.98
    • /
    • pp.243-250
    • /
    • 2005
  • To introduce the cache in a client is one of the methods to migrate the performance degradation of broadcast systems doe to the inherent restrictions of wireless communication environments such as low bandwidth or frequent disconnections. In this paper, we propose a pull-based broadcast strategy in hybrid-based data broadcast systems using bit vectors in order to effectively broadcast data recently requested by clients. Then, we propose a caching strategy considering the characteristics of data broadcast algorithm and then evaluate the performance of the system. According to the result of evaluation, the system employing the proposed strategies shows the better performance in terms of response time.

A New File System for Multimedia Data Stream (멀티미디어 데이터 스트림을 위한 파일 시스템의 설계 및 구현)

  • Lee, Minsuk;Song, Jin-Seok
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.1 no.2
    • /
    • pp.90-103
    • /
    • 2006
  • There are many file systems in various operating systems. Those are usually designed for server environments, where the common cases are usually 'multiple active users', 'great many small files' And they assume a big main memory to be used as buffer cache. So the existing file systems are not suitable for resource hungry embedded systems that process multimedia data streams. In this study, we designed and implemented a new file system which efficiently stores and retrieves multimedia data steams. The proposed file system has a very simple disk layout, which guarantees a quick disk initialization and file system recovery. And we introduced a new indexing-scheme, called the time-based indexing scheme, with the file system. With the indexing scheme, the file system maintains the relation between time and the location for all the multimedia streams. The scheme is useful in searching and playing the compressed multimedia streams by locating exact frame position with given time, resulting in reduction of CPU processing and power consumption. The proposed file system and its APIs utilizing the time-based indexing schemes were implemented firstly on a Linux environment, though it is operating system independent. In the performance evaluation on a real DVR system, which measured the execution time of multi-threaded reading and writing, we found the proposed file system is maximum 38.7% faster than EXT2 file system.

  • PDF

Parallel Cell-Connectivity Information Extraction Algorithm for Ray-casting on Unstructured Grid Data (비정렬 격자에 대한 광선 투사를 위한 셀 사이 연결정보 추출 병렬처리 알고리즘)

  • Lee, Jihun;Kim, Duksu
    • Journal of the Korea Computer Graphics Society
    • /
    • v.26 no.1
    • /
    • pp.17-25
    • /
    • 2020
  • We present a novel multi-core CPU based parallel algorithm for the cell-connectivity information extraction algorithm, which is one of the preprocessing steps for volume rendering of unstructured grid data. We first check the synchronization issues when parallelizing the prior serial algorithm naively. Then, we propose a 3-step parallel algorithm that achieves high parallelization efficiency by removing synchronization in each step. Also, our 3-step algorithm improves the cache utilization efficiency by increasing the spatial locality for the duplicated triangle test process, which is the core operation of building cell-connectivity information. We further improve the efficiency of our parallel algorithm by employing a memory pool for each thread. To check the benefit of our approach, we implemented our method on a system consisting of two octa-core CPUs and measured the performance. As a result, our method shows continuous performance improvement as we add threads. Also, it achieves up to 82.9 times higher performance compared with the prior serial algorithm when we use thirty-two threads (sixteen physical cores). These results demonstrate the high parallelization efficiency and high cache utilization efficiency of our method. Also, it validates the suitability of our algorithm for large-scale unstructured data.

Access Frequency Based Selective Buffer Cache Management Strategy For Multimedia News Data (접근 요청 빈도에 기반한 멀티미디어 뉴스 데이터의 선별적 버퍼 캐쉬 관리 전략)

  • Park, Yong-Un;Seo, Won-Il;Jeong, Gi-Dong
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.9
    • /
    • pp.2524-2532
    • /
    • 1999
  • In this paper, we present a new buffer pool management scheme designed for video type news objects to build a cost-effective News On Demand storage server for serving users requests beyond the limitation of disk bandwidth. In a News On Demand Server where many of users request for video type news objects have to be serviced keeping their playback deadline, the maximum numbers of concurrent users are limited by the maximum disk bandwidth the server provides. With our proposed buffer cache management scheme, a requested data is checked to see whether or not it is worthy of caching by checking its average arrival interval and current disk traffic density. Subsequently, only granted news objects are permitted to get into the buffer pool, where buffer allocation is made not on the block basis but on the object basis. We evaluated the performance of our proposed caching algorithm through simulation. As a result of the simulation, we show that by using this caching scheme to support users requests for real time news data, compared with serving those requests only by disks, 30% of extra requests are served without additional cost increase.

  • PDF

A Data Prefetching Scheme Exploiting the Grain Size in Parallel Programs using Data Arrays (데이타 배열을 사용하는 병렬 프로그램에서 그레인 크기를 이용한 데이타 선인출 기법)

  • Jung, In-Bum;Lee, Joon-Won
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.27 no.1
    • /
    • pp.101-108
    • /
    • 2000
  • The data prefetching scheme is an effective technique to reduce the main memory access latency by exploiting the overlap of processor computations with data accesses. However, if the prefetched data replicate the useful existing data in the cache memory and they are not being used in computations. performances of programs are aggravated. This phenomenon results from the lack of correct predictions for data being used in the future. When parallel programs exploit the data arrays for computations, the grain size is useful information for data prefetching scheme because it implies the range of data using in computations. Based on this information, we suggest a new data prefetching scheme exploited by the grain size of the parallel program. Simulation results show that the suggested prefetching scheme improves the performance of the simulated parallel programs due to the reduction of bus transactions as well as useful prefetching operations.

  • PDF

UTrustDisk: An Efficient Data Protection Scheme for Building Trusted USB Flash Disk

  • Cheng, Yong;Ma, Jun;Ren, Jiangchun;Mei, Songzhu;Wang, Zhiying
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.4
    • /
    • pp.2276-2291
    • /
    • 2017
  • Data protection of removable storage devices is an important issue in information security. Unfortunately, most existing data protection mechanisms are aimed at protecting computer platform which is not suitable for ultra-low-power devices. To protect the flash disk appropriately and efficiently, we propose a trust based USB flash disk, named UTrustDisk. The data protection technologies in UTrustDisk include data authentication protocol, data confidentiality protection and data leakage prevention. Usually, the data integrity protection scheme is the bottleneck in the whole system and we accelerate it by WH universal hash function and speculative caching. The speculative caching will cache the potential hot chunks for reducing the memory bandwidth pollution. We adopt the symmetric encryption algorithm to protect data confidentiality. Before mounting the UTrustDisk, we will run a trusted virtual domain based lightweight virtual machine for preventing information leakage. Besides, we prove formally that UTrustDisk can prevent sensitive data from leaking out. Experimental results show that our scheme's average writing throughput is 44.8% higher than that of NH scheme, and 316% higher than that of SHA-1 scheme. And the success rate of speculative caching mechanism is up to 94.5% since the access pattern is usually sequential.

Time Window based Cache Replacement Strategy using Popularity and Life of News-Demand Data (NOD(News On Demand) 데이터의 인기도와 생명주기를 이용하는 시간 윈도우에 기반한 캐시 재배치 기법)

  • 최태욱;박성호;김영주;정기동
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 1998.10a
    • /
    • pp.101-103
    • /
    • 1998
  • 뉴스기사를 구성하는 NOD데이터는 VOD(Video on Demand) 데이터와는 달리 미디어의 종류 및 크기, 시간적인 접근 지역성, 사용자와 상호 작용성 등의 차이점을 가질 뿐만 아니라 새로운 뉴스기사가 수시로 생성되고 사용자가 인기도가 높은 기사와 최신의 뉴스기사에 더 많이 접근하는 특성을 가진다. 본 논문에서는 현재 서비스중인 전자신문의 로그파일을 분석하여 NOD 뉴스기사의 인기도가 Zipf분포와 다름을 보이고, 뉴스기사의 생명주기Lifr Cycle)에 따른 접근 확률분포 제시한다. NOD 데이터의 접근 편기성으로 인하여 데이터 캐싱을 통한 NOD 서버의 성능 향상을 기대할 수 있으나 뉴스기사의 생명주기가 짧고 접근시간대별로 사용자 접근형태가 변하는 등의 이유로 단순히 인기도만 고려한 캐싱은 빈번한 데이터 재배치 문제로 인해 높은 캐시 관리비용을 야기한다. 따라서 본 논문에서는 뉴스 기사의 접근 편기성에 나타나는 인기도(popularity)와 생명주기를 조합한 척도를 제안하고 이를 이용한 재배치를 제안한다.

  • PDF

공유 메모리를 갖는 다중 프로세서 컴퓨터 시스팀의 설계 및 성능분석

  • Choe, Chang-Yeol;Park, Byeong-Gwan;Park, Seong-Gyu;O, Gil-Rok
    • ETRI Journal
    • /
    • v.10 no.3
    • /
    • pp.83-91
    • /
    • 1988
  • This paper describes the architecture and the performance analysis of a multiprocessor system, which is based on the shared memory and single system bus. The system bus provides the pended protocol for the multiprocessor environment. Analyzing the processor utilization, address/data bus utilization and memory conflicts, we use a simulation model. The hit ratio of private cache memory is a major factor on the linear increase of the performance of a shared memory based multiprocessor system.

  • PDF