• 응용통계연구
• /
• 제29권6호
• /
• pp.1061-1075
• /
• 2016

본 논문에서는 고차원 대용량 자료의 시각화에서 발생할 수 있는 문제점들을 살펴보고 이에 대하여 개발된 방법들에 대하여 논의하였다. 고차원 자료의 경우 2차원 공간상에 표현하기 위하여 중요 변수를 선택해야하며 다양한 시각적 표현 속성과 다면화 방법을 이용하여 좀 더 많은 변수들을 표현할 수 있었다. 또한 관심있는 뷰를 보이는 낮은 차원을 찾는 사영추정방법을 이용할 수 있다. 대용량 자료에서는 점들이 겹쳐지는 문제점을 흩트림과 알파 블렌딩 등을 이용하여 해결할 수 있었다. 또한 고차원 대용량 자료의 탐색을 위하여 개발된 R 패키지인 tabplot과 scagnostics, 그리고 대화형 웹 그래프를 위한 다양한 형태의 R 패키지들을 살펴보았다.

• 한국멀티미디어학회논문지
• /
• 제12권10호
• /
• pp.1457-1465
• /
• 2009

Massively Multiplayer Online Role-Playing Game (MMORPG) 제작에서 최종 데이터 입력 수단으로 쓰이는 스프레드시트 기반 개발 환경은 대용량의 데이터의 수식 처리, 분석에 있어 장점이 있으나 데이터 삽입 삭제, 검색, 연계성 관리에 어려움이 있다. 본 논문에서 대용량 데이터 관리를 위한 스프레드시트 환경 기반 툴 모델을 제안함으로써 실무에서 발생할 수 있는 다양한 데이터 관리 문제들을 입력 단계에서 해결하고자 한다. 본 논문에서 제안된 툴 모델은 MMORPG 데이터 입력을 위해 다수의 데이터 입력자들이 있는 환경에서 데이터 관리 위험을 최소화시키며 동시에 데이터 입력과 관리 작업 효율을 높이는데 효용성을 가지고 있다.

• 대한산업공학회지
• /
• 제32권2호
• /
• pp.133-140
• /
• 2006

Large signal type data sets are difficult to classify, especially if the data sets are non-stationary. In this paper, large signal type and non-stationary data sets are wavelet transformed so that distinct features of the data are extracted in wavelet domain rather than time domain. For the classification of the data, a few wavelet coefficients representing class properties are employed for statistical classification methods : Linear Discriminant Analysis, Quadratic Discriminant Analysis, Neural Network etc. The application of our wavelet-based feature selection method to a mass spectrometry data set for ovarian cancer diagnosis resulted in 100% classification accuracy.

• 한국컴퓨터정보학회논문지
• /
• 제23권11호
• /
• pp.59-66
• /
• 2018

Data Distribution Service(DDS) provides the communications service programmers need to distribute time-critical data between embedded and/or enterprise devices or nodes. In this paper, I propose efficient methods for transmitting and receiving messages of various characteristics in real-time using DDS middleware. For high-frequency characteristic data, I describe several DDS packet types and various default and extended DDS QoS policies. In particular, the batching method is probably the best solution when considering several performance aspects. For large-capacity characteristic data. I will show a method using extended DDS QoS policies, a segmentation and reassembly method, and transmitting and receiving a large-capacity data with low priority method considering network conditions. Finally, I simulate and compare the result of performance for each methods. This results will help determine efficient methods for transmitting and receiving messages of various characteristics using DDS middleware.

• Communications for Statistical Applications and Methods
• /
• 제31권2호
• /
• pp.191-202
• /
• 2024

The goal of this paper is to show how multivariate regression analysis with high-dimensional responses is facilitated by the response dimension reduction. Multivariate regression, characterized by multi-dimensional response variables, is increasingly prevalent across diverse fields such as repeated measures, longitudinal studies, and functional data analysis. One of the key challenges in analyzing such data is managing the response dimensions, which can complicate the analysis due to an exponential increase in the number of parameters. Although response dimension reduction methods are developed, there is no practically useful illustration for various types of data such as so-called large p-small n data. This paper aims to fill this gap by showcasing how response dimension reduction can enhance the analysis of high-dimensional response data, thereby providing significant assistance to statistical practitioners and contributing to advancements in multiple scientific domains.

• JSTS:Journal of Semiconductor Technology and Science
• /
• 제15권2호
• /
• pp.184-193
• /
• 2015

This paper proposes a cost-efficient and automatic method for large data acquisition from a test chip without expensive equipment to characterize random process variation in an integrated circuit. Our method requires only a test chip, a personal computer, a cheap digital-to-analog converter, a controller and multimeters, and thus large volume measurement can be performed on an office desk at low cost. To demonstrate the proposed method, we designed a test chip with a current model logic driver and an array of 128 current mirrors that mimic the random process variation of the driver's tail current mirror. Using our method, we characterized the random process variation of the driver's voltage due to the random process variation on the driver's tail current mirror from large volume measurement data. The statistical characteristics of the driver's output voltage calculated from the measured data are compared with Monte Carlo simulation. The difference between the measured and the simulated averages and standard deviations are less than 20% showing that we can easily characterize the random process variation at low cost by using our cost-efficient automatic large data acquisition method.

• 한국산학기술학회논문지
• /
• 제10권11호
• /
• pp.3207-3212
• /
• 2009

반도체 산업이 발전함에 따라 생산 효율을 높이기 위해 무인 자동 생산 공정이 요구되고 있다. 이러한 무인자동화 생산 관리 시스템은 생산성 향상을 위해 생산 공정에서 발생하는 대량의 실시간 데이터 분석 및 관리를 필요로 한다. 따라서 실시간으로 발생하는 대용량 데이터를 저장하기 위한 저장 관리 시스템이 요구된다. 기존의 저장 관리 시스템으로 오라클, MY-SQL, MS-SQL 등의 디스크 기반 DBMS가 있다. 하지만 기존의 디스크 기반 DBMS는 반도체 장비로부터 실시간으로 발생하는 대용량 데이터 처리에 한계가 있다. 본 논문에서는 대용량 데이터를 저비용으로 실시간 저장하기 위해 블록 단위 삽입 트랜잭션을 이용한 압축-합병 저장 기법을 제안한다. 제안 기법은 블록 단위 트랜잭션을 이용하여 실시간 데이터를 빠르게 저장하며 데이터를 압축하고 압축된 데이터를 합병하여 저장하기 때문에 보다 적은 디스크 공간을 사용하여 저장할 수 있다. 따라서 반도체 공정에서 빠르게 발생하는 대용량 데이터를 기존 DBMS보다 빠르게 저장이 가능하고 저장 공간 비용을 감소시킨다.

• 천문학회보
• /
• 제43권1호
• /
• pp.63.1-63.1
• /
• 2018

The modern large-scale surveys and state-of-the-art cosmological simulations produce various kinds of big data composed of millions and billions of galaxies. Inevitably, we need to adopt modern Big Data platforms to properly handle such large-scale data sets. In my talk, I will briefly introduce the de facto standard of modern Big Data platform, Apache Spark, and present some examples to demonstrate how Apache Spark can be utilized for solving data-driven astronomical problems.

• Journal of the Korean Data and Information Science Society
• /
• 제27권2호
• /
• pp.549-557
• /
• 2016

In this paper we propose multiclassification method for large data sets by ensembling least squares support vector machines (LS-SVM) with principal components instead of raw input vector. We use the revised one-vs-all method for multiclassification, which is one of voting scheme based on combining several binary classifications. The revised one-vs-all method is performed by using the hat matrix of LS-SVM ensemble, which is obtained by ensembling LS-SVMs trained using each random sample from the whole large training data. The leave-one-out cross validation (CV) function is used for the optimal values of hyper-parameters which affect the performance of multiclass LS-SVM ensemble. We present the generalized cross validation function to reduce computational burden of leave-one-out CV functions. Experimental results from real data sets are then obtained to illustrate the performance of the proposed multiclass LS-SVM ensemble.

• 대한산업공학회지
• /
• 제36권1호
• /
• pp.13-21
• /
• 2010

The prediction of contract cancellation of customers is essential in insurance companies but it is a difficult problem because the customer database is large and the target or cancelled customers are a small proportion of the database. This paper proposes a new data mining approach to the binary classification by handling a large-scale unbalanced data. Over-sampling, clustering, regularized logistic regression and boosting are also incorporated in the proposed approach. The proposed approach was applied to a real data set in the area of insurance and the results were compared with some other classification techniques.