• Title/Summary/Keyword: Data Dimension

Search Result 1,954, Processing Time 0.03 seconds

DR-LSTM: Dimension reduction based deep learning approach to predict stock price

  • Ah-ram Lee;Jae Youn Ahn;Ji Eun Choi;Kyongwon Kim
    • Communications for Statistical Applications and Methods
    • /
    • v.31 no.2
    • /
    • pp.213-234
    • /
    • 2024
  • In recent decades, increasing research attention has been directed toward predicting the price of stocks in financial markets using deep learning methods. For instance, recurrent neural network (RNN) is known to be competitive for datasets with time-series data. Long short term memory (LSTM) further improves RNN by providing an alternative approach to the gradient loss problem. LSTM has its own advantage in predictive accuracy by retaining memory for a longer time. In this paper, we combine both supervised and unsupervised dimension reduction methods with LSTM to enhance the forecasting performance and refer to this as a dimension reduction based LSTM (DR-LSTM) approach. For a supervised dimension reduction method, we use methods such as sliced inverse regression (SIR), sparse SIR, and kernel SIR. Furthermore, principal component analysis (PCA), sparse PCA, and kernel PCA are used as unsupervised dimension reduction methods. Using datasets of real stock market index (S&P 500, STOXX Europe 600, and KOSPI), we present a comparative study on predictive accuracy between six DR-LSTM methods and time series modeling.

Image Enhancement for Two-dimension bar code PDF417

  • Park, Ji-Hue;Woo, Hong-Chae
    • Proceedings of the Korea Society of Information Technology Applications Conference
    • /
    • 2005.11a
    • /
    • pp.69-72
    • /
    • 2005
  • As life style becomes to be complicated, lots of support technologies were developed. The bar code technology is one of them. It was renovating approach to goods industry. However, data storage ability in one dimension bar code came in limit because of industry growth. Two-dimension bar code was proposed to overcome one-dimension bar code. PDF417 bar code most commonly used in standard two-dimension bar codes is well defined at data decoding and error correction area. More works could be done in bar code image acquisition process. Applying various image enhancement algorithms, the recognition rate of PDF417 bar code is improved.

  • PDF

Fused inverse regression with multi-dimensional responses

  • Cho, Youyoung;Han, Hyoseon;Yoo, Jae Keun
    • Communications for Statistical Applications and Methods
    • /
    • v.28 no.3
    • /
    • pp.267-279
    • /
    • 2021
  • A regression with multi-dimensional responses is quite common nowadays in the so-called big data era. In such regression, to relieve the curse of dimension due to high-dimension of responses, the dimension reduction of predictors is essential in analysis. Sufficient dimension reduction provides effective tools for the reduction, but there are few sufficient dimension reduction methodologies for multivariate regression. To fill this gap, we newly propose two fused slice-based inverse regression methods. The proposed approaches are robust to the numbers of clusters or slices and improve the estimation results over existing methods by fusing many kernel matrices. Numerical studies are presented and are compared with existing methods. Real data analysis confirms practical usefulness of the proposed methods.

Incremental Linear Discriminant Analysis for Streaming Data Using the Minimum Squared Error Solution (스트리밍 데이터에 대한 최소제곱오차해를 통한 점층적 선형 판별 분석 기법)

  • Lee, Gyeong-Hoon;Park, Cheong Hee
    • Journal of KIISE
    • /
    • v.45 no.1
    • /
    • pp.69-75
    • /
    • 2018
  • In the streaming data where data samples arrive sequentially in time, it is difficult to apply the dimension reduction method based on batch learning. Therefore an incremental dimension reduction method for the application to streaming data has been studied. In this paper, we propose an incremental linear discriminant analysis method using the least squared error solution. Instead of computing scatter matrices directly, the proposed method incrementally updates the projective direction for dimension reduction by using the information of a new incoming sample. The experimental results demonstrate that the proposed method is more efficient compared with previously proposed incremental dimension reduction methods.

A Study on the Changing Dimension Management Methodology With Semantic Layer Data Management and Integrated Data Model (의미론적 계층 데이터 관리와 통합 데이터 모델을 통한 Changing Dimension 관리에 관한 연구)

  • Park Kyong-Seok;Kim Chan-Ho;Song Hye-Eun;You Yong-Bok
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2004.11a
    • /
    • pp.101-104
    • /
    • 2004
  • Business Intelligence 나 DSS 구축과 운영을 위한 근간은 기업의 통합 데이터 인프라로서의 Data Warehouse 구축이 중심을 이룬다. Data Warehouse 는 통합적, 시계열적, 비휘발적, 주제중심적 Data로 구성된다. 이러한 특성이 이론적으로 정교함에도 불구하고 현실적인 프로젝트를 진행함에 있어서 많은 어려움을 발생시킨다. 이러한 문제의 가장 핵심적인 원인이라면 운영시스템의 변화에 따른 운영상의 리스크와 함께 Subject Area 의 요소적 변경에서 그 원인을 찾을 수 있다. 초기에 Data Warehouse 가 아무리 Business User 의 요구사항을 제대로 충족시킬 수 있다 하더라도 시간의 경과에 따라 운영시스템의 변화와 Subject Area 의 요소적 변경은 피할 수 없는 사실인데 이러한 환경에 유연하게 대처할 수 있는 Data Warehouse 가 구축되지 못한다면 결국 Data Warehouse 프로젝트는 현업의 Business 적 문제와는 거리가 먼 고비용을 투자하고 아무런 수익적 가치도 내지 못하는 거추장스러운 시스템에 지나지 않을 것이다. 본 논문에서는 Dimension 관리의 핵심이라고 할 수 있는Changing Dimension 관리 기법과 함께 EDW(Enterprise Data Warehouse)방식의 아키텍처를 중심으로 한 통합데이터모델과 함께 OLAP 메타데이터에 기반한 복합적이면서도 현실적인 Data Warehouse 설계를 제시하고자 한다

  • PDF

A Semi-supervised Dimension Reduction Method Using Ensemble Approach (앙상블 접근법을 이용한 반감독 차원 감소 방법)

  • Park, Cheong-Hee
    • The KIPS Transactions:PartD
    • /
    • v.19D no.2
    • /
    • pp.147-150
    • /
    • 2012
  • While LDA is a supervised dimension reduction method which finds projective directions to maximize separability between classes, the performance of LDA is severely degraded when the number of labeled data is small. Recently semi-supervised dimension reduction methods have been proposed which utilize abundant unlabeled data and overcome the shortage of labeled data. However, matrix computation usually used in statistical dimension reduction methods becomes hindrance to make the utilization of a large number of unlabeled data difficult, and moreover too much information from unlabeled data may not so helpful compared to the increase of its processing time. In order to solve these problems, we propose an ensemble approach for semi-supervised dimension reduction. Extensive experimental results in text classification demonstrates the effectiveness of the proposed method.

Detecting Anomalies, Sabotage, and Malicious Acts in a Cyber-physical System Using Fractal Dimension Based on Higuchi's Algorithm

  • Marwan Albahar
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.4
    • /
    • pp.69-78
    • /
    • 2023
  • With the global rise of digital data, the uncontrolled quantity of data is susceptible to cyber warfare or cyber attacks. Therefore, it is necessary to improve cyber security systems. This research studies the behavior of malicious acts and uses Higuchi Fractal Dimension (HFD), which is a non-linear mathematical method to examine the intricacy of the behavior of these malicious acts and anomalies within the cyber physical system. The HFD algorithm was tested successfully using synthetic time series network data and validated on real-time network data, producing accurate results. It was found that the highest fractal dimension value was computed from the DoS attack time series data. Furthermore, the difference in the HFD values between the DoS attack data and the normal traffic data was the highest. The malicious network data and the non-malicious network data were successfully classified using the Receiver Operating Characteristics (ROC) method in conjunction with a scaling stationary index that helps to boost the ROC technique in classifying normal and malicious traffic. Hence, the suggested methodology may be utilized to rapidly detect the existence of abnormalities in traffic with the aim of further using other methods of cyber-attack detection.

Quantitative assessment of offshore wind speed variability using fractal analysis

  • Shu, Z.R.;Chan, P.W.;Li, Q.S.;He, Y.C.;Yan, B.W.
    • Wind and Structures
    • /
    • v.31 no.4
    • /
    • pp.363-371
    • /
    • 2020
  • Proper understanding of offshore wind speed variability is of essential importance in practice, which provides useful information to a wide range of coastal and marine activities. In this paper, long-term wind speed data recorded at various offshore stations are analyzed in the framework of fractal dimension analysis. Fractal analysis is a well-established data analysis tool, which is particularly suitable to determine the complexity in time series from a quantitative point of view. The fractal dimension is estimated using the conventional box-counting method. The results suggest that the wind speed data are generally fractals, which are likely to exhibit a persistent nature. The mean fractal dimension varies from 1.31 at an offshore weather station to 1.43 at an urban station, which is mainly associated with surface roughness condition. Monthly variability of fractal dimension at offshore stations is well-defined, which often possess larger values during hotter months and lower values during winter. This is partly attributed to the effect of thermal instability. In addition, with an increase in measurement interval, the mean and minimum fractal dimension decrease, whereas the maximum and coefficient of variation increase in parallel.

Nonlinear Correlation Dimension Analysis of EEG and HRV (뇌파의 상관차원과 HRV의 상관분석)

  • Kim, Jung-Gyun;Park, Young-Bae;Park, Young-Jae;Kim, Min-Yong
    • The Journal of the Society of Korean Medicine Diagnostics
    • /
    • v.11 no.2
    • /
    • pp.84-95
    • /
    • 2007
  • Background and Purpose: We have studied the trends of EEG signals in the voluntary breathing condition by applying the fractal analysis. According to chaos theory, irregularity of EEG signals can result from low dimensional deterministic chaos. A principal parameter to quantify the degree of Chaotic nonlinear dynamics is correlation dimension. The aim of this study was to analyze correlation between the correlation dimension of EEG and HRV(heart rate variability). We have studied the trends of EEG signals in the voluntary breathing condition by applying the fractal analysis. Methods: EEG raw data were measured by moving windows during 15 minutes. Then, the correlation dimension(D2) was calculated by each 40-seconds-segment in 15 minutes data, totally 36 segments. 8 channels EEG study on the Fp, F, T, P was carried out in 30 subjects. Results and Conclusion: Correlation analysis of HRV was calculated with deterministic non-linear data and stochastic non-linear data. 1. Ch1(Fp1), Ch4(F3), Ch4(F4) is positive correlated with In LF. 2. Ch1(Fp1), Ch3(F3) is positive correlated with In TF.

  • PDF

The Pattern Recognition System Using the Fractal Dimension of Chaos Theory

  • Shon, Young-Woo
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.15 no.2
    • /
    • pp.121-125
    • /
    • 2015
  • In this paper, we propose a method that extracts features from character patterns using the fractal dimension of chaos theory. The input character pattern image is converted into time-series data. Then, using the modified Henon system suggested in this paper, it determines the last features of the character pattern image after calculating the box-counting dimension, natural measure, information bit, and information (fractal) dimension. Finally, character pattern recognition is performed by statistically finding each information bit that shows the minimum difference compared with a normalized character pattern database.