International Journal of Contents

The Korea Contents Association (한국콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Combining Dynamic Time Warping and Single Hidden Layer Feedforward Neural Networks for Temporal Sign Language Recognition

  • Received : 2011.01.28
  • Accepted : 2011.03.18
  • Published : 2011.03.28
Download PDF
⟨ Previous Next ⟩

Abstract

Temporal Sign Language Recognition (TSLR) from hand motion is an active area of gesture recognition research in facilitating efficient communication with deaf people. TSLR systems consist of two stages: a motion sensing step which extracts useful features from signers' motion and a classification process which classifies these features as a performed sign. This work focuses on two of the research problems, namely unknown time varying signal of sign languages in feature extraction stage and computing complexity and time consumption in classification stage due to a very large sign sequences database. In this paper, we propose a combination of Dynamic Time Warping (DTW) and application of the Single hidden Layer Feedforward Neural networks (SLFNs) trained by Extreme Learning Machine (ELM) to cope the limitations. DTW has several advantages over other approaches in that it can align the length of the time series data to a same prior size, while ELM is a useful technique for classifying these warped features. Our experiment demonstrates the efficiency of the proposed method with the recognition accuracy up to 98.67%. The proposed approach can be generalized to more detailed measurements so as to recognize hand gestures, body motion and facial expression.

Keywords

References

  1. G. B. Huang, Q. Y Zhu and C. K. Siew, “Extreme learning machine: Theory and application,” Nero computing, vol. 70, May. 2006, pp. 489 – 501. https://doi.org/10.1016/j.neucom.2005.12.126
  2. H.T. Huynh, Y. Won, and J.J. Kim, “An improvement of extreme learning machine for compact single hidden layer feedforward neural networks,” International journal of neural systems, vol. 18, no. 5, 2008, pp. 433- 441. https://doi.org/10.1142/S0129065708001695
  3. H.T. Huynh and Y. Won, “Small number of hidden units for ELM with two-stage linear model,” IEICE Trans. On Information and Systems, vol. E91.D, Issue. 4, 2008, pp. 1042- 1049. https://doi.org/10.1093/ietisy/e91-d.4.1042
  4. X.Weng, Junyi Shen, “Classification of multivariate time series using two dimensional singular value decomposition,” Knowledge-Based Systems, vol. 21, Issue 7, 2008, pp. 535-539. https://doi.org/10.1016/j.knosys.2008.03.014
  5. G. B. Huang, “Learning capability and storage capacity of two hidden layer feedforward networks,” IEEE Transactions on Neural Networks, vol. 14, no. 2, 2003, pp. 274- 281. https://doi.org/10.1109/TNN.2003.809401
  6. J. F. Lichtenauer, Emile A. Hendriks, and Marcel J. T. Reinders, “Sign Language Recognition by combining Statiscal DTW and Independent Classification,” IEEE transactions on pattern analysis and machine intelligence, vol.30, no. 11, Nov. 2008, pp. 2040- 2046. https://doi.org/10.1109/TPAMI.2008.123
  7. P. Vamplew, “Recognition of sign language gestures using neural network,” Proc. 1st Euro. Conf. Disability Virtual Reality & Assoc. Tech, Maidenhead, UK, 1996, pp. 27- 33.
  8. E. Jung Holden, Geoffray G. Roy, Robyn Owens, “Hand movement Classification an adaptive fuzzy expert system,” Intl. J. Expert Systems, 1996, pp. 465- 480.
  9. E. Wilson, & G. Anspach, “Neural networks for sign language translation,” SPIE: Applications of Artificial Neural networks, pp. 589- 599.
  10. D. Gavrila and L. Davis, “Towards 3D Model Based Tracking and Recognition of Human Movement: A Multi View Approach,” Proc. IEEE Int’l Workshop Face and Gesture Recognition, June 1995, pp. 272- 277.
  11. A. Corradini, “Dynamic Time Warping for OffLine Recognition of a Small Gesture Vocabulary,” Proc. IEEE ICCV Workshop Recognition, Analysis, and Tracking of Faces and Gestures in RealTime Systems (RATFGRTS '01), July 2001, pp. 82- 89. https://doi.org/10.1109/RATFG.2001.938914
  12. S. Yang and R. Sarkar, “Gesture Recognition Using Hidden Markov Models from Fragmented Observations,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '06), 2006, pp. 766- 773. https://doi.org/10.1109/CVPR.2006.126
  13. N. Morgan and H. Bourlard, “Continuous Speech Recognition Using Multilayer Perceptrons with Hidden Markov Models,” Proc. Int'l Conf. Acoustics, Speech and Signal Processing (ICASSP '90), 1990, pp. 413- 416. https://doi.org/10.1109/ICASSP.1990.115720
  14. Y. Matsuura, H. Miyazawa, and T. Skinner, “Word Recognition Using a Neural Network and a Phonetically Based DTW,” Proc. IEEE Int'l Workshop Neural Networks for Signal Processing (NNSP '94), Sep. 1994, pp. 329- 334. https://doi.org/10.1109/NNSP.1994.366034
  15. A. Corradini and H. Gross, “Camera Based Gesture Recognition for Robot Control,” Proc. Int'l Joint Conf. Neural Networks (IJCNN '00), July 200, pp. 133- 138.
  16. J. Ye, H. Yao, and F. Jiang, “Based on HMM and SVM Multilayer Architecture Classifier for Chinese Sign Language Recognition with Large Vocabulary,” Proc. Third Int’l Conf. Image and Graphics (ICIG ’04), Dec. 2004, pp. 377- 380.
  17. O. Aran and L. Akarun, “Recognizing Two Handed Gestures with Generative, Discriminative and Ensemble Methods via Fisher Kernels,” Proc. Int’l Workshop Multimedia Content Representation, Classification and Security (MCRCS ’06), Sep. 2006, pp. 159- 166.
  18. C. Bahlmann, B. Haasdonk, and H. Burkhardt, “Online Handwriting Recognition with Support Vector Machines- A Kernel Approach,” Proc. Eighth Int’l Workshop Frontiers in Handwriting Recognition (IWFHR ’02), 2002, pp. 49- 54.
  19. B. Strle, Martin Mozina, Ivan Bratko, “Qualitative approximation to Dynamic Time Warping similarity between time series data,” 23rd Annual Workshop on Qualitative Reasoning, QR– LJUBLJANA SLOVENIA, June 2009, pp. 22- 24.
  20. S.H. Kim, Hyung Jeong Yang, Kam Swee Ng, "Temporal Sign Language Analysis Based on DTW and Incremental Model," MICC 2009, Dec. 2009, pp. 14- 16.
  21. E. Keogh, “Exact Indexing of Dynamic Time Warping”, Very Large Database, 2002, pp. 406- 417.
  22. H.T. Huynh and Y. Won, “Hematocrit Estimation from compact single hidden layer feedforward neural networks trained by evolutionary algorithm,” IEEE Congress on Evolutionary Computation, 2008, pp. 2962- 2966.
  23. G.B, Q. Y. Zhu, and C. K. Siew, “Extreme learning machine: A new learning scheme of feedforward neural networks,” Proc. of International Joint Conference on Neural Networks (IJCNN2004), Jun. 2004, pp.489- 501.
  24. T. Starner, Visual Recognition of American Sign Language Using Hidden Markov Models, master’s thesis, Massachusetts Inst. of Technology, Media Arts and Sciences, Jan. 1995.
  25. http://archive.ics.uci.edu/ml/datasets.html