Industrial Engineering and Management Systems

Korean Institute of Industrial Engineers (대한산업공학회)
권호 표지
DOI QR코드

DOI QR Code

Effect of Dimension Reduction on Prediction Performance of Multivariate Nonlinear Time Series

  • Jeong, Jun-Yong (Department of Industrial and Management Engineering, Pohang University of Science and Technology) ;
  • Kim, Jun-Seong (Department of Industrial and Management Engineering, Pohang University of Science and Technology) ;
  • Jun, Chi-Hyuck (Department of Industrial and Management Engineering, Pohang University of Science and Technology)
  • Received : 2015.09.04
  • Accepted : 2015.09.18
  • Published : 2015.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

The dynamic system approach in time series has been used in many real problems. Based on Taken's embedding theorem, we can build the predictive function where input is the time delay coordinates vector which consists of the lagged values of the observed series and output is the future values of the observed series. Although the time delay coordinates vector from multivariate time series brings more information than the one from univariate time series, it can exhibit statistical redundancy which disturbs the performance of the prediction function. We apply dimension reduction techniques to solve this problem and analyze the effect of this approach for prediction. Our experiment uses delayed Lorenz series; least squares support vector regression approximates the predictive function. The result shows that linearly preserving projection improves the prediction performance.

Keywords

References

  1. Adeli, H., Ghosh-Dastidar, S., and Dadmehr, N. (2008), A spatio-temporal wavelet-chaos methodology for EEG-based diagnosis of Alzheimer's disease, Neuroscience Letters, 444(2), 190-194. https://doi.org/10.1016/j.neulet.2008.08.008
  2. Barnard, J. P., Aldrich, C., and Gerber, M. (2001), Embedding of multidimensional time-dependent observations, Physical Review E, 64(4), 046201.
  3. Belkin, M. and Niyogi, P. (2002), Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering, In: T. G. Dietterich, S. Becker, and Z. Ghahramani. (eds), Advance in Neural Information Processing Systems, MIT Press, 585-591.
  4. Cao, L., Mees, A., and Judd, K. (1998), Dynamics from multivariate time series, Physica D: Nonlinear Phenomena, 121(1), 75-88. https://doi.org/10.1016/S0167-2789(98)00151-1
  5. Chen, D. and Han, W. (2013), Prediction of multivariate chaotic time series via radial basis function neural network, Complexity, 18(4), 55-66. https://doi.org/10.1002/cplx.21441
  6. Das, A. and Das, P. (2007), Chaotic analysis of the foreign exchange rates, Applied Mathematics and Computation, 185(1), 388-396. https://doi.org/10.1016/j.amc.2006.06.106
  7. Dhanya, C. and Kumar, D. N. (2011), Multivariate nonlinear ensemble prediction of daily chaotic rainfall with climate inputs, Journal of Hydrology, 403(3), 292-306. https://doi.org/10.1016/j.jhydrol.2011.04.009
  8. Dudul, S. V. (2005), Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network, Applied Soft Computing, 5(4), 333-355. https://doi.org/10.1016/j.asoc.2004.07.005
  9. Fraser, A. M. and Swinney, H. L. (1986), Independent coordinates for strange attractors from mutual information, Physical Review A, 33(2), 1134. https://doi.org/10.1103/PhysRevA.33.1134
  10. Gholipour, A., Araabi, B. N., and Lucas, C. (2006), Predicting chaotic time series using neural and neurofuzzy models: a comparative study, Neural Processing Letters, 24(3), 217-239. https://doi.org/10.1007/s11063-006-9021-x
  11. Grassberger, P. and Procaccia, I. (2004), Measuring the strangeness of strange attractors, In: B. R. Hunt, J. A. Kennedy, T.-Y. Li and H. E. Nusse. (eds.), The Theory of Chaotic Attractors, Springer, 170-189.
  12. Han, M. and Wang, Y. (2009), Analysis and modeling of multivariate chaotic time series based on neural network, Expert Systems with Applications, 36(2), 1280-1290. https://doi.org/10.1016/j.eswa.2007.11.057
  13. Harding, A. K., Shinbrot, T., and Cordes, J. M. (1990), A chaotic attractor in timing noise from the VELA pulsar?, The Astrophysical Journal, 353, 588-596. https://doi.org/10.1086/168648
  14. He, X. and Niyogi, P. (2004), Locality preserving projections, In: S. Thrun, L. K. Saul and B. Scholkopf. (eds), In Advances in Neural Information Processing Systems 16, The MIT Press, Cambridge, USA, MA, 153-160.
  15. He, X., Cai, D., Yan, S., and Zhang, H.-J. (2005), Neighborhood preserving embedding, Proceedings of the Tenth IEEE International Conference on the Computer Vision (ICCV), 2, 1208-1213.
  16. Hotelling, H. (1933), Analysis of a complex of statistical variables into principal components, Journal of Educational Psychology, 24(6), 417- 441. https://doi.org/10.1037/h0071325
  17. Kennel, M. B., Brown, R., and Abarbanel, H. D. (1992), Determining embedding dimension for phase-space reconstruction using a geometrical construction, Physical Review A, 45(6), 3403-3411. https://doi.org/10.1103/PhysRevA.45.3403
  18. Lorenz, E. N. (1963), Deterministic nonperiodic flow, Journal of the Atmospheric Sciences, 20(2), 130-141. https://doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2
  19. Lorenz, E. N. (1995), The essence of chaos, University of Washington Press.
  20. Mackey, M. C. and Glass, L. (1977), Oscillation and chaos in physiological control systems, Science, 197(4300), 287-289. https://doi.org/10.1126/science.267326
  21. Mei-Ying, Y. and Xiao-Dong, W. (2004), Chaotic time series prediction using least squares support vector machines, Chinese Physics, 13(4), 454-458. https://doi.org/10.1088/1009-1963/13/4/007
  22. Monahan, A. H. (2000), Nonlinear principal component analysis by neural networks: Theory and application to the Lorenz system, Journal of Climate, 13(4), 821-835. https://doi.org/10.1175/1520-0442(2000)013<0821:NPCABN>2.0.CO;2
  23. Mukherjee, S., Osuna, E., and Girosi, F. (1997), Nonlinear prediction of chaotic time series using support vector machines, Proceedings of the 1997 IEEE Workshop of the Neural Networks for Signal Processing, 511-520.
  24. Roweis, S. T. and Saul, L. K. (2000), Nonlinear dimensionality reduction by locally linear embedding, Science, 290(5500), 2323-2326. https://doi.org/10.1126/science.290.5500.2323
  25. Shang, P., Li, X., and Kamae, S. (2005), Chaotic analysis of traffic time series, Chaos, Solitons and Fractals, 25(1), 121-128. https://doi.org/10.1016/j.chaos.2004.09.104
  26. Su, L.-y. (2010), Prediction of multivariate chaotic time series with local polynomial fitting, Computers and Mathematics with Applications, 59(2), 737-744. https://doi.org/10.1016/j.camwa.2009.10.019
  27. Suykens, J. A., De Brabanter, J., Lukas, L., and Vandewalle, J. (2002), Weighted least squares support vector machines: robustness and sparse approximation, Neurocomputing, 48(1), 85-105. https://doi.org/10.1016/S0925-2312(01)00644-0
  28. Takens, F. (1981), Detecting strange attractors in turbulence. In: D. A. Rand and L. S. Young (eds.), Dynamical Systems and Turbulence, Warwick 1980, Springer, Berlin, German, 366-381.
  29. Tenenbaum, J. B., De Silva, V., and Langford, J. C. (2000), A global geometric framework for nonlinear dimensionality reduction, Science, 290(5500), 2319-2323. https://doi.org/10.1126/science.290.5500.2319
  30. Torgerson, W. S. (1952), Multidimensional scaling: I. Theory and method, Psychometrika, 17(4), 401-419. https://doi.org/10.1007/BF02288916
  31. Van der Maaten, L. (2007), An introduction to dimensionality reduction using matlab, Report, 1201(07-07), 62.
  32. Vapnik, V. (2013), The nature of statistical learning theory, Springer Science and Business Media.
  33. Zhang, T., Yang, J., Zhao, D., and Ge, X. (2007), Linear local tangent space alignment and application to face recognition, Neurocomputing, 70(7), 1547-1553. https://doi.org/10.1016/j.neucom.2006.11.007
  34. Zhang, Z.-Y. and Zha, H.-Y. (2004), Principal manifolds and nonlinear dimensionality reduction via tangent space alignment, Journal of Shanghai University (English Edition), 8(4), 406-424. https://doi.org/10.1007/s11741-004-0051-1
  35. Zhi-Yong, Y., Guang, Y., and Cun-Bing, D. (2011), Timedelay feedback control in a delayed dynamical chaos system and its applications, Chinese Physics B, 20(1), 010207. https://doi.org/10.1088/1674-1056/20/1/010207

Cited by

  1. Data fusion combined with echo state network for multivariate time series prediction in complex electromechanical system pp.1807-0302, 2018, https://doi.org/10.1007/s40314-018-0669-4