• Title/Summary/Keyword: column-oriented database

Search Result 17, Processing Time 0.024 seconds

A Column-Aware Index Management Using Flash Memory for Read-Intensive Databases

  • Byun, Si-Woo;Jang, Seok-Woo
    • Journal of Information Processing Systems
    • /
    • v.11 no.3
    • /
    • pp.389-405
    • /
    • 2015
  • Most traditional database systems exploit a record-oriented model where the attributes of a record are placed contiguously in a hard disk to achieve high performance writes. However, for read-mostly data warehouse systems, the column-oriented database has become a proper model because of its superior read performance. Today, flash memory is largely recognized as the preferred storage media for high-speed database systems. In this paper, we introduce a column-oriented database model based on flash memory and then propose a new column-aware flash indexing scheme for the high-speed column-oriented data warehouse systems. Our index management scheme, which uses an enhanced $B^+$-Tree, achieves superior search performance by indexing an embedded segment and packing an unused space in internal and leaf nodes. Based on the performance results of two test databases, we concluded that the column-aware flash index management outperforms the traditional scheme in the respect of the mixed operation throughput and its response time.

Performance Comparison of Column-Oriented and Row-Oriented Database Systems for Star Schema Join Processing (스타 스키마 조인 처리에 대한 세로-지향 데이터베이스 시스템과 가로-지향 데이터베이스 시스템의 성능 비교)

  • Oh, Byung-Jung;Ahn, Soo-Min;Kim, Kyung-Chang
    • Journal of the Korea Society of Computer and Information
    • /
    • v.16 no.8
    • /
    • pp.29-38
    • /
    • 2011
  • Unlike in traditional row-oriented database systems, a column-oriented database system stores data in column-oriented and not row-oriented order. Recently, research results revealed the effectiveness of column-oriented databases for applications such as data warehouse and decision support systems that access large volumes of data in a read only manner. In this paper, we investigate the join strategies for column-oriented databases and prove the effectiveness of column-oriented databases in data warehouse systems. For unbiased comparison, the two database systems are analyzed using the star schema benchmark and the performance analysis of a star schema join query is carried out. We experimented with well-known join algorithms and considered early materialization and late materialization join strategies for column-oriented databases. The performance results confirm that star schema join queries perform better in terms of disk I/O cost in column-oriented databases than in row-oriented databases. In addition, the late materialization strategy showed more performance gain than the early materialization strategy in column-oriented databases.

Column-aware Polarization Scheme for High-Speed Database Systems (고속 데이터베이스 시스템을 위한 컬럼-인지 양분화 기법)

  • Byun, Si-Woo
    • Journal of Internet Computing and Services
    • /
    • v.13 no.3
    • /
    • pp.83-91
    • /
    • 2012
  • Recently, column-oriented storage has become a progressive model for high-speed database systems because of its superior I/O performance. In this paper, we analysis traditional raw-oriented storage model and then propose a new column-aware storage management model using flash memory drive and assist drive to improve the effective performance of the high-speed column-oriented database system. Our storage management scheme called column-aware polarization improves the performance of update operation by dividing and compressing table columns into active-columns or inactive-columns, and balancing congested update operations using a assist drive in high workload periods. The results obtained from experimental tests show that our scheme improves the update throughput of column-oriented storage by 19 percent, and the response time by up to 49 percent.

Search Performance Improvement of Column-oriented Flash Storages using Segmented Compression Index (분할된 압축 인덱스를 이용한 컬럼-지향 플래시 스토리지의 검색 성능 개선)

  • Byun, Siwoo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.1
    • /
    • pp.393-401
    • /
    • 2013
  • Most traditional databases exploit record-oriented storage model where the attributes of a record are placed contiguously in hard disk to achieve high performance writes. However, for search-mostly datawarehouse systems, column-oriented storage has become a proper model because of its superior read performance. Today, flash memory is largely recognized as the preferred storage media for high-speed database systems. In this paper, we introduce fast column-oriented database model and then propose a new column-aware index management scheme for the high-speed column-oriented datawarehouse system. Our index management scheme which is based on enhanced $B^+$-Tree achieves high search performance by embedded flash index and unused space compression in internal and leaf nodes. Based on the results of the performance evaluation, we conclude that our index management scheme outperforms the traditional scheme in the respect of the search throughput and response time.

Cross Compressed Replication Scheme for Large-Volume Column Storages (대용량 컬럼 저장소를 위한 교차 압축 이중화 기법)

  • Byun, Siwoo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.14 no.5
    • /
    • pp.2449-2456
    • /
    • 2013
  • The column-oriented database storage is a very advanced model for large-volume data analysis systems because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly datawarehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using MLC flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. In this paper, we introduce fast column-oriented data storage model and then propose a new storage management scheme using a cross compressed replication for the high-speed column-oriented datawarehouse system. Our storage management scheme which is based on two MLC SSD achieves superior performance and reliability by the cross replication of the uncompressed segment and the compressed segment under high workloads of CPU and I/O. Based on the results of the performance evaluation, we conclude that our storage management scheme outperforms the traditional scheme in the respect of update throughput and response time of the column segments.

Shadow Recovery for Column-based Databases (컬럼-기반 데이터베이스를 위한 그림자 복구)

  • Byun, Si-Woo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.16 no.4
    • /
    • pp.2784-2790
    • /
    • 2015
  • The column-oriented database storage is a very advanced model for large-volume data transactions because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly data warehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. In this research, we propose a new transaction recovery scheme for a column-oriented database environment which is based on a flash media file system. We improved traditional shadow paging schemes by reusing old data pages which are supposed to be invalidated in the course of writing a new data page in the flash file system environment. In order to reuse these data pages, we exploit reused shadow list structure in our column-oriented shadow recovery(CoSR) scheme. CoSR scheme minimizes the additional storage overhead for keeping shadow pages and minimizes the I/O performance degradation caused by column data compression of traditional recovery schemes. Based on the results of the performance evaluation, we conclude that CoSR outperforms the traditional schemes by 17%.

Column-aware Transaction Management Scheme for Column-Oriented Databases (컬럼-지향 데이터베이스를 위한 컬럼-인지 트랜잭션 관리 기법)

  • Byun, Si-Woo
    • Journal of Internet Computing and Services
    • /
    • v.15 no.4
    • /
    • pp.125-133
    • /
    • 2014
  • The column-oriented database storage is a very advanced model for large-volume data analysis systems because of its superior I/O performance. Traditional data storages exploit row-oriented storage where the attributes of a record are placed contiguously in hard disk for fast write operations. However, for search-mostly datawarehouse systems, column-oriented storage has become a more proper model because of its superior read performance. Recently, solid state drive using MLC flash memory is largely recognized as the preferred storage media for high-speed data analysis systems. The features of non-volatility, low power consumption, and fast access time for read operations are sufficient grounds to support flash memory as major storage components of modern database servers. However, we need to improve traditional transaction management scheme due to the relatively slow characteristics of column compression and flash operation as compared to RAM memory. In this research, we propose a new scheme called Column-aware Multi-Version Locking (CaMVL) scheme for efficient transaction processing. CaMVL improves transaction performance by using compression lock and multi version reads for efficiently handling slow flash write/erase operation in lock management process. We also propose a simulation model to show the performance of CaMVL. Based on the results of the performance evaluation, we conclude that CaMVL scheme outperforms the traditional scheme.

Axial compressive behaviour of circular CFFT: Experimental database and design-oriented model

  • Khan, Qasim S.;Sheikh, M. Neaz;Hadi, Muhammad N.S.
    • Steel and Composite Structures
    • /
    • v.21 no.4
    • /
    • pp.921-947
    • /
    • 2016
  • Concrete Filled Fibre Reinforced Polymer Tube (CFFT) for new columns construction has attracted significant research attention in recent years. The CFFT acts as a formwork for new columns and a barrier to corrosion accelerating agents. It significantly increases both the strength capacity (Strength enhancement ratio) and the ductility (Strain enhancement ratio) of reinforced concrete columns. In this study, based on predefined selection criteria, experimental investigation results of 134 circular CFFT columns under axial compression have been compiled and analysed from 599 CFFT specimens available in the literature. It has been observed that actual confinement ratio (expressed as a function of material properties of fibres, diameter of CFFT and compressive strength of concrete) has significant influence on the strength and ductility of circular CFFT columns. Design oriented models have been proposed to compute the strength and strain enhancement ratios of circular CFFT columns. The proposed strength and strain enhancement ratio models have significantly reduced Average Absolute Error (AAE), Mean Square Error (MSE), Relative Standard Error of Estimate (RSEE) and Standard Deviation (SD) as compared to other available strength and strain enhancement ratios of circular CFFT column models. The predictions of the proposed strength and strain enhancement ratio models match well with the experimental strength and strain enhancement ratios investigation results in the compiled database.

Livestock Disease Forecasting and Smart Livestock Farm Integrated Control System based on Cloud Computing (클라우드 컴퓨팅기반 가축 질병 예찰 및 스마트 축사 통합 관제 시스템)

  • Jung, Ji-sung;Lee, Meong-hun;Park, Jong-kweon
    • Smart Media Journal
    • /
    • v.8 no.3
    • /
    • pp.88-94
    • /
    • 2019
  • Livestock disease is a very important issue in the livestock industry because if livestock disease is not responded quickly enough, its damage can be devastating. To solve the issues involving the occurrence of livestock disease, it is necessary to diagnose in advance the status of livestock disease and develop systematic and scientific livestock feeding technologies. However, there is a lack of domestic studies on such technologies in Korea. This paper, therefore, proposes Livestock Disease Forecasting and Livestock Farm Integrated Control System using Cloud Computing to quickly manage livestock disease. The proposed system collects a variety of livestock data from wireless sensor networks and application. Moreover, it saves and manages the data with the use of the column-oriented database Hadoop HBase, a column-oriented database management system. This provides livestock disease forecasting and livestock farm integrated controlling service through MapReduce Model-based parallel data processing. Lastly, it also provides REST-based web service so that users can receive the service on various platforms, such as PCs or mobile devices.

Experimental behavior of VHSC encased composite stub column under compression and end moment

  • Huang, Zhenyu;Huang, Xinxiong;Li, Weiwen;Mei, Liu;Liew, J.Y. Richard
    • Steel and Composite Structures
    • /
    • v.31 no.1
    • /
    • pp.69-83
    • /
    • 2019
  • This paper investigates the structural behavior of very high strength concrete encased steel composite columns via combined experimental and analytical study. The experimental programme examines stub composite columns under pure compression and eccentric compression. The experimental results show that the high strength encased concrete composite column exhibits brittle post peak behavior and low ductility but has acceptable compressive resistance. The high strength concrete encased composite column subjected to early spalling and initial flexural cracking due to its brittle nature that may degrade the stiffness and ultimate resistance. The analytical study compares the current code methods (ACI 318, Eurocode 4, AISC 360 and Chinese JGJ 138) in predicting the compressive resistance of the high strength concrete encased composite columns to verify the accuracy. The plastic design resistance may not be fully achieved. A database including the concrete encased composite column under concentered and eccentric compression is established to verify the predictions using the proposed elastic, elastoplastic and plastic methods. Image-oriented intelligent recognition tool-based fiber element method is programmed to predict the load resistances. It is found that the plastic method can give an accurate prediction of the load resistance for the encased composite column using normal strength concrete (20-60 MPa) while the elastoplastic method provides reasonably conservative predictions for the encased composite column using high strength concrete (60-120 MPa).