Study on Heart Rate Variability and PSD Analysis of PPG Data for Emotion Recognition

감정 인식을 위한 PPG 데이터의 심박변이도 및 PSD 분석

  • Choi, Jin-young (Department of Computer Science and Engineering, Chungnam National University) ;
  • Kim, Hyung-shin (Department of Computer Science and Engineering, Chungnam National University)
  • 최진영 (충남대학교 컴퓨터공학과) ;
  • 김형신 (충남대학교 컴퓨터공학과)
  • Received : 2017.11.12
  • Accepted : 2018.01.29
  • Published : 2018.01.31


In this paper, we propose a method of recognizing emotions using PPG sensor which measures blood flow according to emotion. From the existing PPG signal, we use a method of determining positive emotions and negative emotions in the frequency domain through PSD (Power Spectrum Density). Based on James R. Russell's two-dimensional prototype model, we classify emotions as joy, sadness, irritability, and calmness and examine their association with the magnitude of energy in the frequency domain. It is significant that this study used the same PPG sensor used in wearable devices to measure the top four kinds of emotions in the frequency domain through image experiments. Through the questionnaire, the accuracy, the immersion level according to the individual, the emotional change, and the biofeedback for the image were collected. The proposed method is expected to be various development such as commercial application service using PPG and mobile application prediction service by merging with context information of existing smart phone.


Emotion recognition;PPG(Photoplethysmography);Heart rate variability;PSD(Power Spectrum Density)


Supported by : 한국연구재단


  1. D. Yi and C. H. Kah, "Emotions and continued usage of mobile applications," in Industrial Management and Data Systems, Vol. 115, Issue 5, pp. 833-852, 1980.
  2. A. L. Jorge, O. Venet, M. Oscar, F. Mads, B. Jakob, F.-J. Maria and K. V. Lars, "Tell me your apps and I will tell your mood: Correlation of apps usage with Bipolar Disorder State," in Proceedings of the 7th International Conference on Pervasive Technologies Related to Assistive Environments, No. 19, 2014.
  3. T. Yan, C. David, G. Deepak, K. Aman and L. Jie, "Fast App Launching for Mobile Devices Using Predictive User Context," in Proceedings of the 10th international conference on Mobile systems, applications, and services, pp. 13-126, 2012.
  4. Veljko Pejovic, et al., "Interrupt Me: Designing Intelligent Prompting Mechanisms for Pervasive Applications," Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp.897-908, Sep 2014.
  5. Jung Chan-Min, et al., "Personalized Application Recommendation System using Emotion Recognition on Smartphone," Korea Institute of Communication Sciences, pp. 236-237, Feb 2012.
  6. CAI Jing, et al., "The Research on Emotion Recognition from ECG Signal," International Conference on Information Technology and Computer Science, Vol. 1, pp. 497-500, Jul 2009.
  7. Bagha, Sangeeta, and Laxmi Shaw. "A real time analysis of PPG signal for measurement of SpO2 and pulse rate." International journal of computer applications, Vol. 36, No. 11, pp. 45-50, Dec 2011.
  8. M. Bolanos, Student Member, IEEE, H. Nazeran, Senior Member, IEEE, and E. Haltiwanger, "Comparison of Heart Rate Variability Signal Features Derived from Electrocardiography and Photoplethysmography in Healthy Individuals." Engineering in Medicine and Biology Society, No. 1, pp. 4289-4294, Sept 2006.
  9. J. Russell, "Pancultural aspects of the human conceptual organization of emotions," Journal of Personality and Social Psychology, Vol. 45(6), pp. 1281-1288, Dec 1983.
  10. Lombardi, Federico, Malliani, Alberto, "Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology," European Heart Journal, Vol. 17, pp. 354-381, March 1996.
  11. T. D. M. Trinh and G. P. Daniel, "Where and What Using smartphones to predict next locations and applications in daily life," in Pervasive and Mobile Computing, Vol. 12, pp. 79-91, June 2014.
  12. S. Akane and P. W. Rosalind, "Stress Recognition using Wearable Sensors and Mobile Phones," in Humanize Association Conference on Affective Computing and Intelligent Interaction, pp. 671-676, Sep 2013
  13. P. Martin, D. Tilman, P. S. Jose and O. Nuria, "When Attention is not Scarce - Detecting Boredom from Mobile Phone Usage," in Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 825-836, Sep 2015.
  14. W. J. Clancey, Situated Cognition:On Human Knowledge and Computer Representations, Cambridge University, 1997.
  15. K. Rachuri, M. Musolesi, C. Mascolo, P. Rentfrow, C. Longworth, A. Aucinas, "EmotionSense: a mobile phones based adaptive platform for experimental social psychology research," in Proceedings of the 12th ACM international conference on Ubiquitous computing, pp. 281-290, Sep 2010.
  16. L. Robert, L. Yunxin, L. D. Nicholas and Z. Lin, "MoodSchope: Building a Mood Sensor from Smartphone Usage Patterns," in Proceedings of the 11th annual international conference on Mobile systems, applications, and services, pp. 389-402, Sep 2013.
  17. H.W. Kim, W.J. Choi, S.Y. Ihm, Y.H. Park, "A Study on Utilizing a Vital Sign for Emotional Objectification," The Journal of Digital Contents Society, Vol. 18, No. 3, pp. 485-497, June 2017.
  18. S. Kay and S. Marple, "Spectrum analysis-A modern Perspective," in Proceedings of the IEEE, Vol. 69, No. 11, pp. 1380-1419, Nov 1981.
  19. W. J. G., Design of Pulse Oximeter, in Institute of physics Publishing, 1997.
  20. S.H. Hong, M.J. Bae, "Sound of Improving One's Ability to Concentrate," in Proceedings of the Institute of Electronics and Information Engineers, pp. 671-672, Nov 2007.