• Title/Summary/Keyword: multidimensional data processing

Search Result 93, Processing Time 0.026 seconds

A Data Dependency Elimination Method for Multidimensional Subscript Loop by Outer Loop Unrolling (외부루프 펼침에 의한 다중첨자 루프의 종속성 제거 기법)

  • Park, Sang-Il;Park, Weol-Seon;Park, Hyun-Ho;Youn, Sung-Dae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2000.04a
    • /
    • pp.557-561
    • /
    • 2000
  • 본 논문에서는 외부 루프를 펼침으로서 불변 종속거리를 가지는 다중 첨자 루프에서의 병렬화를 이룰 수 있는 새로운 알고리즘을 제시한다. 루프는 프로그램의 수행 시간중 많은 부분을 차지하고, 병렬성 추출의 기본이 되는 구조이다. 루프에서 병렬성을 추출하는 기존의 연구는 종속성이 단일 첨자 또는 복수 첨자에 영향을 받는 경우에만 한정되었다. 다중 첨자를 가지는 루프는 이중 또는 그 이상의 첨자 때문에 기존의 방법을 이용해서 루프의 종속성을 제거하는데 필요한 종속거리를 결정할 수 없다. 그러므로 본 논문에서는 종속거리를 측정하기 위한 새로운 기법을 제안하고, 제안된 알고리즘을 모의 실험에 의해 타당성을 확인한다.

  • PDF

Modeling a Business Performance Information System with Knowledge Discovery in Databases (데이터베이스 지식발견체계에 기반한 경영성과 정보시스템의 구축)

  • Cho, Seong-Hoon;Chung, Min-Yong;Kim, Jong-Hwa
    • IE interfaces
    • /
    • v.14 no.2
    • /
    • pp.164-171
    • /
    • 2001
  • We suggest a Business Performance Information System with Knowledge Discovery in Databases(KDD) as a key component of integrated information and knowledge management system. The proposed system measures business performance by considering both VA(Value-Added), which represents stakeholder's point of view and EVA(Economic Value-Added), which represents shareholder's point of view. In modeling of Business Performance Information System, we apply the following KDD processes : Data Warehouse for consistent management of a performance data, On-Line Analytic Processing(OLAP) for multidimensional analysis, Genetic Algorithms for exploring and finding dominant managing factors and Analytic Hierarchy Process(AHP) for applying expert's knowledge and experience. To demonstrate the performance of the system, we conducted a case study using financial data of Korean automobile industry over 16 years from 1981 to 1996, which is taken from database of KISFAS(Korea Investors Services Financial Analysis System).

  • PDF

Multidimensional Optimization Model of Music Recommender Systems (음악추천시스템의 다차원 최적화 모형)

  • Park, Kyong-Su;Moon, Nam-Me
    • The KIPS Transactions:PartB
    • /
    • v.19B no.3
    • /
    • pp.155-164
    • /
    • 2012
  • This study aims to identify the multidimensional variables and sub-variables and study their relative weight in music recommender systems when maximizing the rating function R. To undertake the task, a optimization formula and variables for a research model were derived from the review of prior works on recommender systems, which were then used to establish the research model for an empirical test. With the research model and the actual log data of real customers obtained from an on line music provider in Korea, multiple regression analysis was conducted to induce the optimal correlation of variables in the multidimensional model. The results showed that the correlation value against the rating function R for Items was highest, followed by Social Relations, Users and Contexts. Among sub-variables, popular music from Social Relations, genre, latest music and favourite artist from Items were high in the correlation with the rating function R. Meantime, the derived multidimensional recommender systems revealed that in a comparative analysis, it outperformed two dimensions(Users, Items) and three dimensions(Users, Items and Contexts, or Users, items and Social Relations) based recommender systems in terms of adjusted $R^2$ and the correlation of all variables against the values of the rating function R.

Research on Mining Technology for Explainable Decision Making (설명가능한 의사결정을 위한 마이닝 기술)

  • Kyungyong Chung
    • Journal of the Institute of Convergence Signal Processing
    • /
    • v.24 no.4
    • /
    • pp.186-191
    • /
    • 2023
  • Data processing techniques play a critical role in decision-making, including handling missing and outlier data, prediction, and recommendation models. This requires a clear explanation of the validity, reliability, and accuracy of all processes and results. In addition, it is necessary to solve data problems through explainable models using decision trees, inference, etc., and proceed with model lightweight by considering various types of learning. The multi-layer mining classification method that applies the sixth principle is a method that discovers multidimensional relationships between variables and attributes that occur frequently in transactions after data preprocessing. This explains how to discover significant relationships using mining on transactions and model the data through regression analysis. It develops scalable models and logistic regression models and proposes mining techniques to generate class labels through data cleansing, relevance analysis, data transformation, and data augmentation to make explanatory decisions.

Pattern Similarity Retrieval of Data Sequences for Video Retrieval System (비디오 검색 시스템을 위한 데이터 시퀀스 패턴 유사성 검색)

  • Lee Seok-Lyong
    • The KIPS Transactions:PartD
    • /
    • v.13D no.3 s.106
    • /
    • pp.347-356
    • /
    • 2006
  • A video stream can be represented by a sequence of data points in a multidimensional space. In this paper, we introduce a trend vector that approximates values of data points in a sequence and represents the moving trend of points in the sequence, and present a pattern similarity matching method for data sequences using the trend vector. A sequence is partitioned into multiple segments, each of which is represented by a trend vector. The query processing is based on the comparison of these vectors instead of scanning data elements of entire sequences. Using the trend vector, our method is designed to filter out irrelevant sequences from a database and to find similar sequences with respect to a query. We have performed an extensive experiment on synthetic sequences as well as video streams. Experimental results show that the precision of our method is up to 2.1 times higher and the processing time is up to 45% reduced, compared with an existing method.

A PIVOT based Query Optimization Technique for Horizontal View Tables in Relational Databases (관계 데이터베이스에서 수평 뷰 테이블에 대한 PIVOT 기반의 질의 최적화 방법)

  • Shin, Sung-Hyun;Moon, Yang-Sae;Kim, Jin-Ho;Kang, Gong-Mi
    • The KIPS Transactions:PartD
    • /
    • v.14D no.2
    • /
    • pp.157-168
    • /
    • 2007
  • For effective analyses in various business applications, OLAP(On-Line Analytical Processing) systems represent the multidimensional data as the horizontal format of tables whose columns are corresponding to values of dimension attributes. Because the traditional RDBMSs have the limitation on the maximum number of attributes in table columns(MS SQLServer and Oracle permit each table to have up to 1,024 columns), horizontal tables cannot be directly stored into relational database systems. In this paper, we propose various efficient optimization strategies in transforming horizontal queries to equivalent vertical queries. To achieve this goral, we first store a horizontal table using an equivalent vertical table, and then develop various query transformation rules for horizontal table queries using the PIVOT operator. In particular, we propose various alternative query transformation rules for the basic relational operators, selection, projection, and join. Here, we note that the transformed queries can be executed in several ways, and their execution times will differ from each other. Thus, we propose various optimization strategies that transform the horizontal queries to the equivalent vertical queries when using the PIVOT operator. Finally, we evaluate these methods through extensive experiments and identify the optimal transformation strategy when using the PIVOT operator.

Development of HDF Browser for the Utilization of EOC Imagery

  • Seo, Hee-Kyung;Ahn, Seok-Beom;Park, Eun-Chul;Hahn, Kwang-Soo;Choi, Joon-Soo;Kim, Choen
    • Korean Journal of Remote Sensing
    • /
    • v.18 no.1
    • /
    • pp.61-69
    • /
    • 2002
  • The purpose of Electro-Optical Camera (EOC), the primary payload of KOMPSAT-1, is to collect high resolution visible imagery of the Earth including Korean Peninsula. EOC images will be distributed to the public or many user groups including government, public corporations, academic or research institutes. KARI will offer the online service to the users through internet. Some application, e.g., generation of Digital Elevation Model (DEM), needs a secondary data such as satellite ephemeris data, attitude data to process the EOC imagery. EOC imagery with these ancillary information will be distributed in a file of Hierarchical Data Format (HDF) file formal. HDF is a physical file format that allows storage of many different types of scientific data including images, multidimensional data arrays, record oriented data, and point data. By the lack of public domain softwares supporting HDF file format, many public users may not access EOC data without difficulty. The purpose of this research is to develop a browsing system of EOC data for the general users not only for scientists who are the main users of HDF. The system is PC-based and huts user-friendly interface.

A Conveyor Algorithm for Complete Consistency of Materialized View in a Self-Maintenance (실체 뷰의 자기관리에서 완전일관성을 위한 컨베이어 알고리듬)

  • Hong, In-Hoon;Kim, Yon-Soo
    • IE interfaces
    • /
    • v.16 no.2
    • /
    • pp.229-239
    • /
    • 2003
  • The On-Line Analytical Processing (OLAP) tools access data from the data warehouse for complex data analysis, such as multidimensional data analysis, and decision support activities. Current research has lead to new developments in all aspects of data warehousing, however, there are still a number of problems that need to be solved for making data warehousing effective. View maintenance, one of them, is to maintain view in response to updates in source data. Keeping the view consistent with updates to the base relations, however, can be expensive, since it may involve querying external sources where the base relations reside. In order to reduce maintenance costs, it is possible to maintain the views using information that is strictly local to the data warehouse. This process is usually referred to as "self-maintenance of views". A number of algorithm have been proposed for self maintenance of views where they keep some additional information in data warehouse in the form of auxiliary views. But those algorithms did not consider a consistency of materialized views using view self-maintenance. The purpose of this paper is to research consistency problem when self-maintenance of views is implemented. The proposed "conveyor algorithm" will resolved a complete consistency of materialized view using self-maintenance with considering network delay. The rationale for conveyor algorithm and performance characteristics are described in detail.

NMR Solvent Peak Suppression by Piecewise Polynomial Truncated Singular Value Decomposition Methods

  • Kim, Dae-Sung;Lee, Hye-Kyoung;Won, Young-Do;Kim, Dai-Gyoung;Lee, Young-Woo;Won, Ho-Shik
    • Bulletin of the Korean Chemical Society
    • /
    • v.24 no.7
    • /
    • pp.967-970
    • /
    • 2003
  • A new modified singular value decomposition method, piecewise polynomial truncated SVD (PPTSVD), which was originally developed to identify discontinuity of the earth's radial density function, has been used for large solvent peak suppression and noise elimination in nuclear magnetic resonance (NMR) signal processing. PPTSVD consists of two algorithms of truncated SVD (TSVD) and L₁ problems. In TSVD, some unwanted large solvent peaks and noise are suppressed with a certain soft threshold value, whereas signal and noise in raw data are resolved and eliminated in L₁ problems. These two algorithms were systematically programmed to produce high quality of NMR spectra, including a better solvent peak suppression with good spectral line shapes and better noise suppression with a higher signal to noise ratio value up to 27% spectral enhancement, which is applicable to multidimensional NMR data processing.

Incremental Maintenance of Horizontal Views Using a PIVOT Operation and a Differential File in Relational DBMSs (관계형 데이터베이스에서 PIVOT 연산과 차등 파일을 이용한 수평 뷰의 점진적인 관리)

  • Shin, Sung-Hyun;Kim, Jin-Ho;Moon, Yang-Sae;Kim, Sang-Wook
    • The KIPS Transactions:PartD
    • /
    • v.16D no.4
    • /
    • pp.463-474
    • /
    • 2009
  • To analyze multidimensional data conveniently and efficiently, OLAP (On-Line Analytical Processing) systems or e-business are widely using views in a horizontal form to represent measurement values over multiple dimensions. These views can be stored as materialized views derived from several sources in order to support accesses to the integrated data. The horizontal views can provide effective accesses to complex queries of OLAP or e-business. However, we have a problem of occurring maintenance of the horizontal views since data sources are distributed over remote sites. We need a method that propagates the changes of source tables to the corresponding horizontal views. In this paper, we address incremental maintenance of horizontal views that makes it possible to reflect the changes of source tables efficiently. We first propose an overall framework that processes queries over horizontal views transformed from source tables in a vertical form. Under the proposed framework, we propagate the change of vertical tables to the corresponding horizontal views. In order to execute this view maintenance process efficiently, we keep every change of vertical tables in a differential file and then modify the horizontal views with the differential file. Because the differential file is represented as a vertical form, its tuples should be converted to those in a horizontal form to apply them to the out-of-date horizontal view. With this mechanism, horizontal views can be efficiently refreshed with the changes in a differential file without accessing source tables. Experimental results show that the proposed method improves average performance by 1.2$\sim$5.0 times over the existing methods.