• Title/Summary/Keyword: data sets

Search Result 3,763, Processing Time 0.03 seconds

A Real-Time Data Mining for Stream Data Sets (연속발생 데이터를 위한 실시간 데이터 마이닝 기법)

  • Kim Jinhwa;Min Jin Young
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.29 no.4
    • /
    • pp.41-60
    • /
    • 2004
  • A stream data is a data set that is accumulated to the data storage from a data source over time continuously. The size of this data set, in many cases. becomes increasingly large over time. To mine information from this massive data. it takes much resource such as storage, memory and time. These unique characteristics of the stream data make it difficult and expensive to use this large size data accumulated over time. Otherwise. if we use only recent or part of a whole data to mine information or pattern. there can be loss of information. which may be useful. To avoid this problem. we suggest a method that efficiently accumulates information. in the form of rule sets. over time. It takes much smaller storage compared to traditional mining methods. These accumulated rule sets are used as prediction models in the future. Based on theories of ensemble approaches. combination of many prediction models. in the form of systematically merged rule sets in this study. is better than one prediction model in performance. This study uses a customer data set that predicts buying power of customers based on their information. This study tests the performance of the suggested method with the data set alone with general prediction methods and compares performances of them.

Introduction of Japanese Ocean Flux data sets with Use of Remote sensing Observations (J-OFURO)

  • Kubota, Masahisa
    • Proceedings of the KSRS Conference
    • /
    • 1999.11a
    • /
    • pp.231-236
    • /
    • 1999
  • Accurate ocean surface fluxes with high resolution are critical for understanding a mechanism of global climate. However, it is difficult to derive those fluxes by using ocean observation data because the number of ocean observation data is extremely small and the distribution is inhomogeneous. On the other hand. satellite data are characterized by the high density, the high resolution and the homogeneity. Therefore, it can be considered that we obtain accurate ocean surface by using satellite data. Recently we constructed ocean surface data sets mainly using satellite data. The data set is named by Japanese Ocean Flux data sets with Use of Remote sensing Observations (J-OFURO). Here, we introduce J-OFURO. The data set includes shortwave radiation, longwave radiation, latent heat flux, sensible heat flux, and momentum flux etc. Moreover, sea surface dynamic topography data are included in the data set. Radiation data sets covers western Pacific and eastern Indian Ocean because we use a Japanese geostationally satellite (GMS) to estimate radiation fluxes. On the other hand, turbulent heat fluxes are globally estimated. The constructed data sets are used and shows the effectiveness for many scientific studies.

  • PDF

Wavenumber correlation analysis of satellite magnetometer observations

  • Kim, Jeong-Woo;Kim, Won-Kyun;Kim, Hye-Yun
    • Proceedings of the KSEEG Conference
    • /
    • 2000.04b
    • /
    • pp.311-313
    • /
    • 2000
  • Identifying anomaly correlations between data sets is the basis for rationalizing geopotenial interpretation and theory. A procedure between the two or more geopotential data sets. Anomaly features that show direct, inverse, or no correlationsbetween the data may be separated by applying filters in the frequency domains of the data sets. The correlation filter passes or rejects wavenumbers between co-registered data sets based on the correlation coefficient between common wavenumbers as given by the cosine of their phase difference. This study includes as example of Magsat magnetic anomaly profile that illustrates the usefulness of the procedure for extracting correlative features between the sets.

  • PDF

Efficient Continuous Skyline Query Processing Scheme over Large Dynamic Data Sets

  • Li, He;Yoo, Jaesoo
    • ETRI Journal
    • /
    • v.38 no.6
    • /
    • pp.1197-1206
    • /
    • 2016
  • Performing continuous skyline queries of dynamic data sets is now more challenging as the sizes of data sets increase and as they become more volatile due to the increase in dynamic updates. Although previous work proposed support for such queries, their efficiency was restricted to small data sets or uniformly distributed data sets. In a production database with many concurrent queries, the execution of continuous skyline queries impacts query performance due to update requirements to acquire exclusive locks, possibly blocking other query threads. Thus, the computational costs increase. In order to minimize computational requirements, we propose a method based on a multi-layer grid structure. First, relational data object, elements of an initial data set, are processed to obtain the corresponding multi-layer grid structure and the skyline influence regions over the data. Then, the dynamic data are processed only when they are identified within the skyline influence regions. Therefore, a large amount of computation can be pruned by adopting the proposed multi-layer grid structure. Using a variety of datasets, the performance evaluation confirms the efficiency of the proposed method.

A Fast Algorithm for Mining Association Rules in Web Log Data (상품간 연관 규칙의 효율적 탐색 방법에 관한 연구 : 인터넷 쇼핑몰을 중심으로)

  • 오은정;오상봉
    • Proceedings of the Korea Society for Industrial Systems Conference
    • /
    • 2003.11a
    • /
    • pp.621-626
    • /
    • 2003
  • Mining association rules in web log files can be divided into two steps: 1) discovering frequent item sets in web data; 2) extracting association rules from the frequent item sets found in the previous step. This paper suggests an algorithm for finding frequent item sets efficiently The essence of the proposed algorithm is to transform transaction data files into matrix format. Our experimental results show that the suggested algorithm outperforms the Apriori algorithm, which is widely used to discover frequent item sets, in terms of scan frequency and execution time.

  • PDF

Metadata Analysis of Open Government Data by Formal Concept Analysis (형식 개념 분석을 통한 공공데이터의 메타데이터 분석)

  • Kim, Haklae
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.1
    • /
    • pp.305-313
    • /
    • 2018
  • Public open data is a database or electronic file produced by a public agency or government. The government is opening public data through the open data portals and individual agency websites. However, it is a reality that there is a limit to search and utilize desired public data from the perspective of data users. In particular, it takes a great deal of effort and time to understand the characteristics of data sets and to combine different data sets. This study suggests the possibility of interlinking between data sets by analyzing the common relationship of item names held by public data. The data sets are collected from the open data portal, and item names included in the data sets are extracted. The extracted item names consist of formal context and formal concept through formal concept analysis. The format concept has a list of data sets and a set of item name as extent and intent, respectively, and analyzes the common items of intent end to determine the possibility of data connection. The results derived from the formal concept analysis can be effectively applied to the semantic connection of the public data, and can be applied to data standard and quality improvement for public data release.

GIS-based Spatial Integration and Statistical Analysis using Multiple Geoscience Data Sets : A Case Study for Mineral Potential Mapping (다중 지구과학자료를 이용한 GIS 기반 공간통합과 통계량 분석 : 광물 부존 예상도 작성을 위한 사례 연구)

  • 이기원;박노욱;권병두;지광훈
    • Korean Journal of Remote Sensing
    • /
    • v.15 no.2
    • /
    • pp.91-105
    • /
    • 1999
  • Spatial data integration using multiple geo-based data sets has been regarded as one of the primary GIS application issues. As for this issue, several integration schemes have been developed as the perspectives of mathematical geology or geo-mathematics. However, research-based approaches for statistical/quantitative assessments between integrated layer and input layers are not fully considered yet. Related to this niche point, in this study, spatial data integration using multiple geoscientific data sets by known integration algorithms was primarily performed. For spatial integration by using raster-based GIS functionality, geological, geochemical, geophysical data sets, DEM-driven data sets and remotely sensed imagery data sets from the Ogdong area were utilized for geological thematic mapping related by mineral potential mapping. In addition, statistical/quantitative information extraction with respective to relationships among used data sets and/or between each data set and integrated layer was carried out, with the scope of multiple data fusion and schematic statistical assessment methodology. As for the spatial integration scheme, certainty factor (CF) estimation and principal component analysis (PCA) were applied. However, this study was not aimed at direct comparison of both methodologies; whereas, for the statistical/quantitative assessment between integrated layer and input layers, some statistical methodologies based on contingency table were focused. Especially, for the bias reduction, jackknife technique was also applied in PCA-based spatial integration. Through the statistic analyses with respect to the integration information in this case study, new information for relationships of integrated layer and input layers was extracted. In addition, influence effects of input data sets with respect to integrated layer were assessed. This kind of approach provides a decision-making information in the viewpoint of GIS and is also exploratory data analysis in conjunction with GIS and geoscientific application, especially handing spatial integration or data fusion with complex variable data sets.

A Measure of Agreement for Multivariate Interval Observations by Different Sets of Raters

  • Um, Yong-Hwan
    • Journal of the Korean Data and Information Science Society
    • /
    • v.15 no.4
    • /
    • pp.957-963
    • /
    • 2004
  • A new agreement measure for multivariate interval data by different sets of raters is proposed. The proposed approach builds on Um's multivariate extension of Cohen's kappa. The proposed measure is compared with corresponding earlier measures based on Berry and Mielke's approach and Janson and Olsson approach, respectively. Application of the proposed measure is exemplified using hypothetical data set.

  • PDF

Parameter Estimation and Prediction for NHPP Software Reliability Model and Time Series Regression in Software Failure Data

  • Song, Kwang-Yoon;Chang, In-Hong
    • Journal of Integrative Natural Science
    • /
    • v.7 no.1
    • /
    • pp.67-73
    • /
    • 2014
  • We consider the mean value function for NHPP software reliability model and time series regression model in software failure data. We estimate parameters for the proposed models from two data sets. The values of SSE and MSE is presented from two data sets. We compare the predicted number of faults with the actual two data sets using the mean value function and regression curve.

A Knowledge Discovery Framework for Spatiotemporal Data Mining

  • Lee, Jun-Wook;Lee, Yong-Joon
    • Journal of Information Processing Systems
    • /
    • v.2 no.2
    • /
    • pp.124-129
    • /
    • 2006
  • With the explosive increase in the generation and utilization of spatiotemporal data sets, many research efforts have been focused on the efficient handling of the large volume of spatiotemporal sets. With the remarkable growth of ubiquitous computing technology, mining from the huge volume of spatiotemporal data sets is regarded as a core technology which can provide real world applications with intelligence. In this paper, we propose a 3-tier knowledge discovery framework for spatiotemporal data mining. This framework provides a foundation model not only to define the problem of spatiotemporal knowledge discovery but also to represent new knowledge and its relationships. Using the proposed knowledge discovery framework, we can easily formalize spatiotemporal data mining problems. The representation model is very useful in modeling the basic elements and the relationships between the objects in spatiotemporal data sets, information and knowledge.