• Title/Summary/Keyword: multidimensional data processing

Search Result 93, Processing Time 0.028 seconds

Implementation of CNN-based Masking Algorithm for Post Processing of Aerial Image

  • CHOI, Eunsoo;QUAN, Zhixuan;JUNG, Sangwoo
    • Korean Journal of Artificial Intelligence
    • /
    • v.9 no.2
    • /
    • pp.7-14
    • /
    • 2021
  • Purpose: To solve urban problems, empirical research is being actively conducted to implement a smart city based on various ICT technologies, and digital twin technology is needed to effectively implement a smart city. A digital twin is essential for the realization of a smart city. A digital twin is a virtual environment that intuitively visualizes multidimensional data in the real world based on 3D. Digital twin is implemented on the premise of the convergence of GIS and BIM, and in particular, a lot of time is invested in data pre-processing and labeling in the data construction process. In digital twin, data quality is prioritized for consistency with reality, but there is a limit to data inspection with the naked eye. Therefore, in order to improve the required time and quality of digital twin construction, it was attempted to detect a building using Mask R-CNN, a deep learning-based masking algorithm for aerial images. If the results of this study are advanced and used to build digital twin data, it is thought that a high-quality smart city can be realized.

An Index-Based Approach for Subsequence Matching Under Time Warping in Sequence Databases (시퀀스 데이터베이스에서 타임 워핑을 지원하는 효과적인 인덱스 기반 서브시퀀스 매칭)

  • Park, Sang-Hyeon;Kim, Sang-Uk;Jo, Jun-Seo;Lee, Heon-Gil
    • The KIPS Transactions:PartD
    • /
    • v.9D no.2
    • /
    • pp.173-184
    • /
    • 2002
  • This paper discuss an index-based subsequence matching that supports time warping in large sequence databases. Time warping enables finding sequences with similar patterns even when they are of different lengths. In earlier work, Kim et al. suggested an efficient method for whole matching under time warping. This method constructs a multidimensional index on a set of feature vectors, which are invariant to time warping, from data sequences. For filtering at feature space, it also applies a lower-bound function, which consistently underestimates the time warping distance as well as satisfies the triangular inequality. In this paper, we incorporate the prefix-querying approach based on sliding windows into the earlier approach. For indexing, we extract a feature vector from every subsequence inside a sliding window and construct a multidimensional index using a feature vector as indexing attributes. For query processing, we perform a series of index searches using the feature vectors of qualifying query prefixes. Our approach provides effective and scalable subsequence matching even with a large volume of a database. We also prove that our approach does not incur false dismissal. To verify the superiority of our approach, we perform extensive experiments. The results reveal that our approach achieves significant speedup with real-world S&P 500 stock data and with very large synthetic data.

Parallel Multithreaded Processing for Data Set Summarization on Multicore CPUs

  • Ordonez, Carlos;Navas, Mario;Garcia-Alvarado, Carlos
    • Journal of Computing Science and Engineering
    • /
    • v.5 no.2
    • /
    • pp.111-120
    • /
    • 2011
  • Data mining algorithms should exploit new hardware technologies to accelerate computations. Such goal is difficult to achieve in database management system (DBMS) due to its complex internal subsystems and because data mining numeric computations of large data sets are difficult to optimize. This paper explores taking advantage of existing multithreaded capabilities of multicore CPUs as well as caching in RAM memory to efficiently compute summaries of a large data set, a fundamental data mining problem. We introduce parallel algorithms working on multiple threads, which overcome the row aggregation processing bottleneck of accessing secondary storage, while maintaining linear time complexity with respect to data set size. Our proposal is based on a combination of table scans and parallel multithreaded processing among multiple cores in the CPU. We introduce several database-style and hardware-level optimizations: caching row blocks of the input table, managing available RAM memory, interleaving I/O and CPU processing, as well as tuning the number of working threads. We experimentally benchmark our algorithms with large data sets on a DBMS running on a computer with a multicore CPU. We show that our algorithms outperform existing DBMS mechanisms in computing aggregations of multidimensional data summaries, especially as dimensionality grows. Furthermore, we show that local memory allocation (RAM block size) does not have a significant impact when the thread management algorithm distributes the workload among a fixed number of threads. Our proposal is unique in the sense that we do not modify or require access to the DBMS source code, but instead, we extend the DBMS with analytic functionality by developing User-Defined Functions.

Construction of Theme Melody Index by Transforming Melody to Time-series Data for Content-based Music Information Retrieval (내용기반 음악정보 검색을 위한 선율의 시계열 데이터 변환을 이용한 주제선율색인 구성)

  • Ha, Jin-Seok;Ku, Kyong-I;Park, Jae-Hyun;Kim, Yoo-Sung
    • The KIPS Transactions:PartD
    • /
    • v.10D no.3
    • /
    • pp.547-558
    • /
    • 2003
  • From the viewpoint of that music melody has the similar features to time-series data, music melody is transformed to a time-series data with normalization and corrections and the similarity between melodies is defined as the Euclidean distance between the transformed time-series data. Then, based the similarity between melodies of a music object, melodies are clustered and the representative of each cluster is extracted as one of theme melodies for the music. To construct the theme melody index, a theme melody is represented as a point of the multidimensional metric space of M-tree. For retrieval of user's query melody, the query melody is also transformed into a time-series data by the same way of indexing phase. To retrieve the similar melodies to the query melody given by user from the theme melody index the range query search algorithm is used. By the implementation of the prototype system using the proposed theme melody index we show the effectiveness of the proposed methods.

A Hybrid Multi-Level Feature Selection Framework for prediction of Chronic Disease

  • G.S. Raghavendra;Shanthi Mahesh;M.V.P. Chandrasekhara Rao
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.12
    • /
    • pp.101-106
    • /
    • 2023
  • Chronic illnesses are among the most common serious problems affecting human health. Early diagnosis of chronic diseases can assist to avoid or mitigate their consequences, potentially decreasing mortality rates. Using machine learning algorithms to identify risk factors is an exciting strategy. The issue with existing feature selection approaches is that each method provides a distinct set of properties that affect model correctness, and present methods cannot perform well on huge multidimensional datasets. We would like to introduce a novel model that contains a feature selection approach that selects optimal characteristics from big multidimensional data sets to provide reliable predictions of chronic illnesses without sacrificing data uniqueness.[1] To ensure the success of our proposed model, we employed balanced classes by employing hybrid balanced class sampling methods on the original dataset, as well as methods for data pre-processing and data transformation, to provide credible data for the training model. We ran and assessed our model on datasets with binary and multivalued classifications. We have used multiple datasets (Parkinson, arrythmia, breast cancer, kidney, diabetes). Suitable features are selected by using the Hybrid feature model consists of Lassocv, decision tree, random forest, gradient boosting,Adaboost, stochastic gradient descent and done voting of attributes which are common output from these methods.Accuracy of original dataset before applying framework is recorded and evaluated against reduced data set of attributes accuracy. The results are shown separately to provide comparisons. Based on the result analysis, we can conclude that our proposed model produced the highest accuracy on multi valued class datasets than on binary class attributes.[1]

Data Sampling-based Angular Space Partitioning for Parallel Skyline Query Processing (데이터 샘플링을 통한 각 기반 공간 분할 병렬 스카이라인 질의처리 기법)

  • Chung, Jaehwa
    • The Journal of Korean Association of Computer Education
    • /
    • v.18 no.5
    • /
    • pp.63-70
    • /
    • 2015
  • In the environment that the complex conditions need to be satisfied, skyline query have been applied to various field. To processing a skyline query in centralized scheme, several techniques have been suggested and recently map/reduce platform based approaches has been proposed which divides data space into multiple partitions for the vast volume of multidimensional data. However, the performances of these approaches are fluctuated due to the uneven data loading between servers and redundant tasks. Motivated by these issues, this paper suggests a novel technique called MR-DEAP which solves the uneven data loading using the random sampling. The experimental result gains the proposed MR-DEAP outperforms MR-Angular and MR-BNL scheme.

Implementation and performance evaluation of embedded main-memory storage system for real-time retrieval of multidimensional data (다차원 데이타의 실시간 검색을 위한 내장형 주기억장치 자료 저장시스템의 구성 및 성능평가)

  • Kwon, Oh-Su;Jung, Jae-Bo;Hong, Bong-Kweon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2000.10a
    • /
    • pp.109-112
    • /
    • 2000
  • 이동 단말기 관리, 무인 항공 제어 시스템 둥의 시스템에서는 검색 대상의 정보(위치, 여러 가지 상태등)가 시시각각으로 빠르게 변화하므로 현재의 상태를 정확히 파악하기 위하여 많은 양의 자료 검색, 변경 요청이 빈번히 발생한다. 이와 같은 시스템에서의 상태 정보 검색은 자료의 효용성이 사라지기 전에 이루어져야 하므로 디스크 I/O가 많은 디스크 상주형 데이터베이스로는 한계점을 안고 있다. 또한 빠른 검색을 지원할 수 있는 주기억장치 상주형 데이터베이스로는 다량의 데이터를 저장해야 하는 어려움을 안고 있다. 본 논문에서는 위와 같은 실시간 검색 기능과 대용량 자료 저장의 2가지 요구 사항을 만족시키기 위한 내장형 주기억장치 저장 시스템을 개발하였다.

  • PDF

A Storage Scheme of Health Data Stream for Multidimensional Analysis (건강 스트림 데이터의 다차원적 분석을 위한 저장 구조)

  • Shin, Hea-Won;Lim, Yoon-Sun;Kim, Myung
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2005.05a
    • /
    • pp.81-84
    • /
    • 2005
  • 유비쿼터스 의료 기술이 본격화되면서 센서 네트워크를 통해 환자의 건강 관련 데이터 스트림을 수집하여 위험상황을 탐지하고 지속적인 건강 상태를 모니터링할 수 있게 되었다. 그러나 방대한 양의 스트림 데이터로부터 의미 있는 데이터를 효과적으로 찾아내기 위해서는 실시간으로 데이터의 갱신과 집계 연산이 가능해야 하고 데이터의 압축이 효율적으로 처리 될 수 있는 다차원 저장구조가 필요하다. 기존의 다차원 데이터 분석 도구인 OLAP 큐브 저장구조는 실시간 업데이트가 힘들고, 스트림 데이터 저장 구조인 DSMS들은 다차원 데이터 분석이 용이하지 않다. 이에 본 연구에서는 건강 스트림 데이터의 특징과 질의를 분석하고, 이러한 스트림 데이터에 적합한 저장구조의 요건을 제시하였다. 또한 점진적 갱신이 가능하고, 대용량 데이터를 시간 차원으로 압축, 삭제하기 용이하며 실시간에 분석 데이터 구축이 가능한 저장구조를 제안하고 그 효율성을 보였다.

  • PDF

Suggestion of Multidimensional Analysis Technique Using Sensor Data in IoT Environment (사물인터넷 환경에서 센싱 데이터를 활용한 다차원 분석 기법 제안)

  • Kang, Jung-Ku;Park, Seok-Cheon;Kim, Jong-Hyun
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.04a
    • /
    • pp.637-640
    • /
    • 2015
  • 최근 사물인터넷이 화두에 오르며, 사물인터넷에서 발생된 센싱 데이터의 분석에 많은 관심이 모아지고 있다. 사물인터넷에서는 대규모의 센싱 데이터가 폭발적으로 발생하고 있으아, 이렇게 발생된 센싱 데이터의 분석에 대한 연구는 현재 미비한 상태이다. 사물인터넷 환경에서 발생된 센싱 데이터는 외부 데이터와 통합 분석을 통해 가치 있는 데이터로 재생산이 가능한 만큼 사물인터넷에서 발생되는 센싱 데이터의 분석에 대한 연구가 필요하다. 따라서 본 논문에서는 사물인터넷 환경에서 센싱 데이터를 활용한 다차원 분석 기법을 제안 하고자 한다.

Algorithm Generating Item Response Data Based on Multidimensional Item Response Theory (다차원 문항반응이론에 기반한 문항 응답 데이터 생성 알고리즘)

  • Kim, ByoungWook;Lee, WonGyu
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2014.04a
    • /
    • pp.526-528
    • /
    • 2014
  • 본 논문은 다차원 문항반응이론 모델에 기반하여 시뮬레이션을 위한 피험자들의 문항 응답 데이터를 생성하는 알고리즘을 개발하는 것이 목적으로 하였다. 본 알고리즘은 시험지를 구성하고 있는 문항들의 모수를 읽고, 각각의 차원에 대해 피험자들의 능력 수준을 나타내는 정규 분포 확률 변수를 생성한다. 본 알고리즘은 다차원 문항반응이론 모델에 기반하여 피험자들이 각 문항에 대해 정답으로 응답할 확률을 계산한다. 피험자들의 문항 응답을 결정하는 균일 분포 난수와 비교한다. 만약 확률이 난수보다 크면 피험자는 올바른 답을 한 것으로 보고 그렇지 않을 경우 틀리게 답할 것으로 한다. 본 프로그램은 피험자 수, 문항 수를 조절할 수 있다. 본 알고리즘을 통해 교육 측정 분야에서 다차원 문항반응 이론을 이용하여 학습자들의 문항 응답 데이터를 이용한 시뮬레이션 연구에 기여할 수 있을 것으로 기대한다.