• Title/Summary/Keyword: cache storage

Search Result 138, Processing Time 0.033 seconds

Data De-duplication and Recycling Technique in SSD-based Storage System for Increasing De-duplication Rate and I/O Performance (SSD 기반 스토리지 시스템에서 중복률과 입출력 성능 향상을 위한 데이터 중복제거 및 재활용 기법)

  • Kim, Ju-Kyeong;Lee, Seung-Kyu;Kim, Deok-Hwan
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.49 no.12
    • /
    • pp.149-155
    • /
    • 2012
  • SSD is a storage device of having high-performance controller and cache buffer and consists of many NAND flash memories. Because NAND flash memory does not support in-place update, valid pages are invalidated when update and erase operations are issued in file system and then invalid pages are completely deleted via garbage collection. However, garbage collection performs many erase operations of long latency and then it reduces I/O performance and increases wear leveling in SSD. In this paper, we propose a new method of de-duplicating valid data and recycling invalid data. The method de-duplicates valid data and then recycles invalid data so that it improves de-duplication ratio. Due to reducing number of writes and garbage collection, the method could increase I/O performance and decrease wear leveling in SSD. Experimental result shows that it can reduce maximum 20% number of garbage collections and 9% I/O latency than those of general case.

Enhancing LRU Buffer Replacement Policy with Delayed Write of Not-cold-dirty-pages for Flash Memory (플래시 메모리를 위한 Not-cold-Page 쓰기지연을 통한 LRU 버퍼교체 정책 개선)

  • Jung Ho-Young;Park Sung-Min;Cha Jae-Hyuk;Kang Soo-Yong
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.33 no.9
    • /
    • pp.634-641
    • /
    • 2006
  • Flash memory has many advantages like non-volatility and fast I/O speed, but it has also disadvantages such as not-in-place-update data and asymmetric read/write/erase speed. For the performance of flash memory storage, it is essential for the buffer replacement algorithms to reduce the number of write operations that also affects the number of erase operations. A new buffer replacement algorithm is proposed in this paper, that delays the writes of not-cold-dirty pages in the buffer cache of flash storage. We show that this algorithm effectively decreases the number of write operations and erase operations without much degradation of hit ratio. As a result overall performance of flash I/O speed is improved.

K-Box: Ontology Management System based on Topic Maps (K-Box: 토픽맵 기반의 온톨로지 관리 시스템)

  • 김정민;박철만;정준원;이한준;민경섭;김형주
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.1
    • /
    • pp.1-13
    • /
    • 2004
  • The Semantic Web introduces the next generation of the Web by establishing a semantic layer of machine-understandable data to enable machines (i.e intelligent agents) retrieve more relevant information and execute automated web services using semantic information. Ontology-related technologies are very important to evolve the World Wide Web of today into the Semantic Web in representation and share of semantic data. In this paper, we proposed and implemented the efficient ontology management system, K-Box, which constructs and manages ontologies using topic maps. We can use K-Box system to construct, store and retrieve ontologies. K-Box system has several components: Topicmap Factory, Topicmap Provider, Topicmap Query Processor, Topicmap Object Wrapper, Topicmap Cache Manager, Topicmap Storage Wrapper.

Designing Hybrid HDD using SLC/MLC combined Flash Memory (SLC/MLC 혼합 플래시 메모리를 이용한 하이브리드 하드디스크 설계)

  • Hong, Seong-Cheol;Shin, Dong-Kun
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.16 no.7
    • /
    • pp.789-793
    • /
    • 2010
  • Recently, flash memory-based non-volatile cache (NVC) is emerging as an effective solution to enhance both I/O performance and energy consumption of storage systems. To get significant performance and energy gains by NVC, it would be better to use multi-level-cell (MLC) flash memories since it can provide a large capacity of NVC with low cost. However, the number of available program/erase cycles of MLC flash memory is smaller than that of single-level-cell (SLC) flash memory limiting the lifespan of NVC. To overcome such a limitation, SLC/MLC combined flash memory is a promising solution for NVC. In this paper, we propose an effective management scheme for heterogeneous SLC and MLC regions of the combined flash memory.

Mechanism to Select the Data Source of HDFS with SSD Cache Based on Storage I / O Cost (SSD 캐시를 적용한 HDFS의 I/O 비용 기반 데이터 선택 기법)

  • Kim, Minkyung;Shin, Mincheol;Park, Sanghyun
    • Annual Conference of KIPS
    • /
    • 2015.04a
    • /
    • pp.676-679
    • /
    • 2015
  • 빅데이터 분석을 위한 Hadoop 환경에서 고성능 저장장치인 SSD에 대한 중요성이 증가하면서 일반적으로 사용되는 저장장치인 HDD와 혼합하여 사용하는 연구들이 주목 받고 있다. 특히 SSD를 HDD의 캐시로 사용했을 때 저장장치에 대한 I/O 성능을 향상할 수 있다는 연구 결과들이 있다. 본 연구는 이를 바탕으로 SSD를 HDD의 캐시로 사용한다. HDFS는 저장장치에 접근하여 I/O를 수행하는데 기존에는 로컬 서버에서 캐시 미스가 발생한 경우 로컬 HDD로 접근한다. 이러한 방식은 접근하는 데이터에 따라 SSD의 높은 Bandwidth를 활용하지 못하게 되는 경우를 발생시키고 그 결과 특정 서버의 I/O 지연으로 전체 분산 처리의 성능을 저하시킬 수 있다. 이를 해결하기 위해 본 연구는 HDFS 레벨에서 로컬 서버의 HDD와 데이터 복제본들이 저장된 원격 서버의 SSD에서 I/O를 수행하는 경우에 대해 수식을 통해 비용을 비교한다. 그 결과 항상 기대 성능이 높은 저장 장치를 선택하여 데이터를 읽어오게 함으로써 기존 방식보다 성능이 개선될 수 있음을 입증한다.

A Deep Learning Approach for Identifying User Interest from Targeted Advertising

  • Kim, Wonkyung;Lee, Kukheon;Lee, Sangjin;Jeong, Doowon
    • Journal of Information Processing Systems
    • /
    • v.18 no.2
    • /
    • pp.245-257
    • /
    • 2022
  • In the Internet of Things (IoT) era, the types of devices used by one user are becoming more diverse and the number of devices is also increasing. However, a forensic investigator is restricted to exploit or collect all the user's devices; there are legal issues (e.g., privacy, jurisdiction) and technical issues (e.g., computing resources, the increase in storage capacity). Therefore, in the digital forensics field, it has been a challenge to acquire information that remains on the devices that could not be collected, by analyzing the seized devices. In this study, we focus on the fact that multiple devices share data through account synchronization of the online platform. We propose a novel way of identifying the user's interest through analyzing the remnants of targeted advertising which is provided based on the visited websites or search terms of logged-in users. We introduce a detailed methodology to pick out the targeted advertising from cache data and infer the user's interest using deep learning. In this process, an improved learning model considering the unique characteristics of advertisement is implemented. The experimental result demonstrates that the proposed method can effectively identify the user interest even though only one device is examined.

Distributed In-Memory Caching Method for ML Workload in Kubernetes (쿠버네티스에서 ML 워크로드를 위한 분산 인-메모리 캐싱 방법)

  • Dong-Hyeon Youn;Seokil Song
    • Journal of Platform Technology
    • /
    • v.11 no.4
    • /
    • pp.71-79
    • /
    • 2023
  • In this paper, we analyze the characteristics of machine learning workloads and, based on them, propose a distributed in-memory caching technique to improve the performance of machine learning workloads. The core of machine learning workload is model training, and model training is a computationally intensive task. Performing machine learning workloads in a Kubernetes-based cloud environment in which the computing framework and storage are separated can effectively allocate resources, but delays can occur because IO must be performed through network communication. In this paper, we propose a distributed in-memory caching technique to improve the performance of machine learning workloads performed in such an environment. In particular, we propose a new method of precaching data required for machine learning workloads into the distributed in-memory cache by considering Kubflow pipelines, a Kubernetes-based machine learning pipeline management tool.

  • PDF

The Efficient Merge Operation in Log Buffer-Based Flash Translation Layer for Enhanced Random Writing (임의쓰기 성능향상을 위한 로그블록 기반 FTL의 효율적인 합병연산)

  • Lee, Jun-Hyuk;Roh, Hong-Chan;Park, Sang-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.19D no.2
    • /
    • pp.161-186
    • /
    • 2012
  • Recently, the flash memory consistently increases the storage capacity while the price of the memory is being cheap. This makes the mass storage SSD(Solid State Drive) popular. The flash memory, however, has a lot of defects. In order that these defects should be complimented, it is needed to use the FTL(Flash Translation Layer) as a special layer. To operate restrictions of the hardware efficiently, the FTL that is essential to work plays a role of transferring from the logical sector number of file systems to the physical sector number of the flash memory. Especially, the poor performance is attributed to Erase-Before-Write among the flash memory's restrictions, and even if there are lots of studies based on the log block, a few problems still exists in order for the mass storage flash memory to be operated. If the FAST based on Log Block-Based Flash often is generated in the wide locality causing the random writing, the merge operation will be occur as the sectors is not used in the data block. In other words, the block thrashing which is not effective occurs and then, the flash memory's performance get worse. If the log-block makes the overwriting caused, the log-block is executed like a cache and this technique contributes to developing the flash memory performance improvement. This study for the improvement of the random writing demonstrates that the log block is operated like not only the cache but also the entire flash memory so that the merge operation and the erase operation are diminished as there are a distinct mapping table called as the offset mapping table for the operation. The new FTL is to be defined as the XAST(extensively-Associative Sector Translation). The XAST manages the offset mapping table with efficiency based on the spatial locality and temporal locality.

A Design of Fractional Motion Estimation Engine with 4×4 Block Unit of Interpolator & SAD Tree for 8K UHD H.264/AVC Encoder (8K UHD(7680×4320) H.264/AVC 부호화기를 위한 4×4블럭단위 보간 필터 및 SAD트리 기반 부화소 움직임 추정 엔진 설계)

  • Lee, Kyung-Ho;Kong, Jin-Hyeung
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.50 no.6
    • /
    • pp.145-155
    • /
    • 2013
  • In this paper, we proposed a $4{\times}4$ block parallel architecture of interpolation for high-performance H.264/AVC Fractional Motion Estimation in 8K UHD($7680{\times}4320$) video real time processing. To improve throughput, we design $4{\times}4$ block parallel interpolation. For supplying the $10{\times}10$ reference data for interpolation, we design 2D cache buffer which consists of the $10{\times}10$ memory arrays. We minimize redundant storage of the reference pixel by applying the Search Area Stripe Reuse scheme(SASR), and implement high-speed plane interpolator with 3-stage pipeline(Horizontal Vertical 1/2 interpolation, Diagonal 1/2 interpolation, 1/4 interpolation). The proposed architecture was simulated in 0.13um standard cell library. The gate count is 436.5Kgates. The proposed H.264/AVC Fractional Motion Estimation can support 8K UHD at 30 frames per second by running at 187MHz.

Segment-based Cache Replacement Policy in Transcoding Proxy (트랜스코딩 프록시에서 세그먼트 기반 캐쉬 교체 정책)

  • Park, Yoo-Hyun;Kim, Hag-Young;Kim, Kyong-Sok
    • The KIPS Transactions:PartA
    • /
    • v.15A no.1
    • /
    • pp.53-60
    • /
    • 2008
  • Streaming media has contributed to a significant amount of today's Internet Traffic. Like traditional web objects, rich media objects can benefit from proxy caching, but caching streaming media is more of challenging than caching simple web objects, because the streaming media have features such as huge size and high bandwidth. And to support various bandwidth requirements for the heterogeneous ubiquitous devices, a transcoding proxy is usually necessary to provide not only adapting multimedia streams to the client by transcoding, but also caching them for later use. The traditional proxy considers only a single version of the objects, whether they are to be cached or not. However the transcoding proxy has to evaluate the aggregate effect from caching multiple versions of the same object to determine an optimal set of cache objects. And recent researches about multimedia caching frequently store initial parts of videos on the proxy to reduce playback latency and archive better performance. Also lots of researches manage the contents with segments for efficient storage management. In this paper, we define the 9-events of transcoding proxy using 4-atomic events. According to these events, the transcoding proxy can define the next actions. Then, we also propose the segment-based caching policy for the transcoding proxy system. The performance results show that the proposing policy have a low delayed start time, high byte-hit ratio and less transcoding data.