• Title/Summary/Keyword: data materialization

Search Result 37, Processing Time 0.034 seconds

Performance Comparison of Column-Oriented and Row-Oriented Database Systems for Star Schema Join Processing (스타 스키마 조인 처리에 대한 세로-지향 데이터베이스 시스템과 가로-지향 데이터베이스 시스템의 성능 비교)

  • Oh, Byung-Jung;Ahn, Soo-Min;Kim, Kyung-Chang
    • Journal of the Korea Society of Computer and Information
    • /
    • v.16 no.8
    • /
    • pp.29-38
    • /
    • 2011
  • Unlike in traditional row-oriented database systems, a column-oriented database system stores data in column-oriented and not row-oriented order. Recently, research results revealed the effectiveness of column-oriented databases for applications such as data warehouse and decision support systems that access large volumes of data in a read only manner. In this paper, we investigate the join strategies for column-oriented databases and prove the effectiveness of column-oriented databases in data warehouse systems. For unbiased comparison, the two database systems are analyzed using the star schema benchmark and the performance analysis of a star schema join query is carried out. We experimented with well-known join algorithms and considered early materialization and late materialization join strategies for column-oriented databases. The performance results confirm that star schema join queries perform better in terms of disk I/O cost in column-oriented databases than in row-oriented databases. In addition, the late materialization strategy showed more performance gain than the early materialization strategy in column-oriented databases.

A Study on the Selective Materialization of Spatial Data Cube (공간 데이타 큐브의 선택적 실체화에 관한 연구)

  • 이기영
    • Journal of the Korea Society of Computer and Information
    • /
    • v.4 no.4
    • /
    • pp.69-76
    • /
    • 1999
  • Recently, it has been studied the methods to materialize and precompute the query results for complexed spatial aggregation queries with high response time and the popular use in spatial data warehouse. In this paper, we propose extended selective materialization algorithm and present the way to materialize selectively which is considered access frequency and computation time of spatial operation according to spatial measures of spatial views for improvement of existing selective materialization algorithms.

  • PDF

Fine Granule View Materialization in Data Cubes (데이타 큐브에서 세분화된 뷰 실체화 기법)

  • Kim, Min-Jeong;Jeong, Yeon-Dong;Park, Ung-Je;Kim, Myeong-Ho
    • Journal of KIISE:Databases
    • /
    • v.28 no.4
    • /
    • pp.587-595
    • /
    • 2001
  • Precomputation and materialization of parts. commonly called views of a data cube is a common technique in data warehouses The view is defined as the result of a query which is defined through aggregate functions In this paper we introduce the concept of fine granule view. The fine granule view is the result of a query defined through aggregate functions and the range on each dimension, where the subdivision of each dimension is based on queries access patterns. For the representation and selection of fine granule views to materialize, we define the ANO-OR cube graph and AND-OR minimum cost graph. With these structures, we propose a fine granule view materialization method. And through experiments, we evaluate the performance of the proposed method.

  • PDF

An Indexing Technique for Range Sum Queries in Spatio - Temporal Databases (시공간 데이타베이스에서 영역 합 질의를 위한 색인 기법)

  • Cho Hyung-Ju;Choi Yong-Jin;Min Jun-Ki;Chung Chin-Wan
    • Journal of KIISE:Databases
    • /
    • v.32 no.2
    • /
    • pp.129-141
    • /
    • 2005
  • Although spatio-temporal databases have received considerable attention recently, there has been little work on processing range sum queries on the historical records of moving objects despite their importance. Since to answer range sum queries, the direct access to a huge amount of data incurs prohibitive computation cost, materialization techniques based on existing index structures are recently suggested. A simple but effective solution is to apply the materialization technique to the MVR-tree known as the most efficient structure for window queries with spatio-temporal conditions. However, the MVR-tree has a difficulty in maintaining pre-aggregated results inside its internal nodes due to cyclic paths between nodes. Aggregate structures based on other index structures such as the HR-tree and the 3DR-tree do not provide satisfactory query performance. In this paper, we propose a new indexing technique called the Adaptive Partitioned Aggregate R-Tree (APART) and query processing algorithms to efficiently process range sum queries in many situations. Experimental results show that the performance of the APART is typically above 2 times better than existing aggregate structures in a wide range of scenarios.

XML Vicw Indexing (XML 뷰 인덱싱)

  • 김영성;강현철
    • Journal of KIISE:Databases
    • /
    • v.30 no.3
    • /
    • pp.252-272
    • /
    • 2003
  • The view mechanism provides users with appropriate portions of database through data filtering and integration. In the Web era where information proliferates, the view concept is also useful for XML, a future standard for data exchange on the Web. This paper proposes a method of implementing XML views called XML view indexing, whereby XML view xv is represented as an XML view index(XVI) which is a structure containing the identifiers of xv's underlying XML elements as well as the information on xv. Since XVI for xv stores just the identifiers of the XML elements but not the elements themselves, when a user requests to retrieve xv, its XVI should be materialized against xv's underlying XML documents. Also an efficient algorithm to incrementally maintain consistency of XVI given a update of xv's underlying XML documents is required. This paper proposes and implements data structures and algorithms for XML view indexing. The performance experiments on XML view indexing reveal that it outperforms view recomputation for repeated accesses to the view, and requires as much as about 30 times less storage space compared to XML view materialization though the latter takes less time for repeated accesses to the view due to no need of materialization.

A Model-Based Method for Information Alignment: A Case Study on Educational Standards

  • Choi, Namyoun;Song, Il-Yeol;Zhu, Yongjun
    • Journal of Computing Science and Engineering
    • /
    • v.10 no.3
    • /
    • pp.85-94
    • /
    • 2016
  • We propose a model-based method for information alignment using educational standards as a case study. Discrepancies and inconsistencies in educational standards across different states/cities hinder the retrieval and sharing of educational resources. Unlike existing educational standards alignment systems that only give binary judgments (either "aligned" or "not-aligned"), our proposed system classifies each pair of educational standard statements in one of seven levels of alignments: Strongly Fully-aligned, Weakly Fully-aligned, Partially-$aligned^{***}$, Partially-$aligned^{**}$, Partially-$aligned^*$, Poorly-aligned, and Not-aligned. Such a 7-level categorization extends the notion of binary alignment and provides a finer-grained system for comparing educational standards that can broaden categories of resource discovery and retrieval. This study continues our previous use of mathematics education as a domain, because of its generally unambiguous concepts. We adopt a materialization pattern (MP) model developed in our earlier work to represent each standard statement as a verb-phrase graph and a noun-phrase graph; we align a pair of statements using graph matching based on Bloom's Taxonomy, WordNet, and taxonomy of mathematics concepts. Our experiments on data sets of mathematics educational standards show that our proposed system can provide alignment results with a high degree of agreement with domain expert's judgments.

Application of Data Processing Technology on Large Clusters to Distribution Automation System (대용량 데이터 처리기술을 배전자동화 시스템에 적용)

  • Lee, Sung-Woo;Ha, Bok-Nam;Seo, In-Yong;Jang, Moon-Jong
    • The Transactions of The Korean Institute of Electrical Engineers
    • /
    • v.60 no.2
    • /
    • pp.245-251
    • /
    • 2011
  • Quantities of data in the DMS (Distribution management system) or SCADA (Supervisory control and data acquisition) system is enormously large as illustrated by the usage of term flooding of data. This enormous quantity of data is transmitted to the status data or event data of the on-site apparatus in real-time. In addition, if GIS (Geographic information system) and AMR (Automatic meter reading), etc are integrated, the quantity of data to be processed in real-time increases unimaginably. Increase in the quantity of data due to addition of system or increase in the on-site facilities cannot be handled through the currently used Single Thread format of data processing technology. However, if Multi Thread technology that utilizes LF-POOL (Leader Follower -POOL) is applied in processing large quantity of data, large quantity of data can be processed in short period of time and the load on the server can be minimized. In this Study, the actual materialization and functions of LF POOL technology are examined.

Design and materialization of remote monitoring integrate mod for efficient TCS(Toll Collection System) maintenance (요금수납시스템의 효율적인 유지관리를 위한 원격모니터링 통합모델 설계 및 구현)

  • Park, Dong-Yeon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2007.05a
    • /
    • pp.531-534
    • /
    • 2007
  • 원격모니터링 통합모델은 고속도로나 유료도로의 영업소 요금수납 체계의 하부시스템 (요금수납시스템, ETCS시스템, 위반차량촬영시스템, 축중시스템)의 구성 및 장애 현황을 감시하고, 장애 발생에 대한 장애처리 체계를 수행하는 모델이다. 본 논문에서는 실시간 장애감시, 효율적 WorkFlow 구성 및 차별화된 정보를 제공하기 위해서 감시 S/W 엔진과 DBMS(Database Managemnet System) 기반의 C/S (Client /Server)시스템을 구성하고 WEB기반의 환경을 설계한다. 본 통합모델을 구현함으로써 체계적인 구성관리/장애관리를 통한 효율적인 유지관리 업무가 가능하며, 현장 장비의 상태 파악이 조기에 용이하여 장애 시간을 최소화 할 수 있다. 또한 Data의 지식 DB화를 통하여 유사 장애에 대한 조치가 용이하다.

  • PDF

The Spatial View Creation and Materialization Technique in Multi-Server Environments (멀티서버 환경에서의 공간 뷰 생성 및 실체화 기법)

  • 김태연;정보홍;조숙경;배해영
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2001.10a
    • /
    • pp.112-114
    • /
    • 2001
  • 지금까지 구축 된 공간 데이터들을 각 서버끼리 공유하여 유사한 내용의 공간정보구축을 위만 중복투자를 막고 공간데이터의 활용도를 높이는 방안이 시급하다. 멀티서버 환경에서의 공간데이터의 공유는 데이터의 보안과 사용자 편의성을 고려하여 뷰로 제공되고 관리되어질 수 있다. 분산 된 대용량의 공간데이터에 대해 복잡한 공간연산 처리를 감안한다면 멀티서버 환경에서는 공간 뷰의 실체화가 필수적이다. 본 논문에서는 멀티서버 환경에서의 공간데이터의 유통을 지원하고 공유 된 공간데이터에 대한 사용자 편의성과 네트웍 부하를 고려한 공간 뷰 실체화 기법을 제안한다. 확장 된 공간데이터 디렉토리 서비스(Extended Spatial Data Diretory Service)를 이용하여 각 서버가 공유하는 공간테이블에 대란 정보를 검색하고, 검색 된 징보를 바탕으로 로컬서버는 각 서버에에 직접 접속하여 필요한 영역의 데이터를 얻어 뷰를 생성한다. 이때 공유 공간데이터가 대용량 데이터임을 고려하여 로컬서버는 공간 뷰 성의에 대한 술어를 분리하여 뷰 질의 처리를 각각의 원격서버로 분산시켜 네트웍 부하와 서버의 질의처리 비용을 줄이고 공간 뷰 서버에 실제화 시킴으로써 뷰에 대한 질의처리시 각 서버로의 접근을 데비터의 접근을 줄여 사용자 응-답시간을 최소화한다.

  • PDF