• Title/Summary/Keyword: Arousal-Valence

Search Result 77, Processing Time 0.022 seconds

Multimodal Emotional State Estimation Model for Implementation of Intelligent Exhibition Services (지능형 전시 서비스 구현을 위한 멀티모달 감정 상태 추정 모형)

  • Lee, Kichun;Choi, So Yun;Kim, Jae Kyeong;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.20 no.1
    • /
    • pp.1-14
    • /
    • 2014
  • Both researchers and practitioners are showing an increased interested in interactive exhibition services. Interactive exhibition services are designed to directly respond to visitor responses in real time, so as to fully engage visitors' interest and enhance their satisfaction. In order to install an effective interactive exhibition service, it is essential to adopt intelligent technologies that enable accurate estimation of a visitor's emotional state from responses to exhibited stimulus. Studies undertaken so far have attempted to estimate the human emotional state, most of them doing so by gauging either facial expressions or audio responses. However, the most recent research suggests that, a multimodal approach that uses people's multiple responses simultaneously may lead to better estimation. Given this context, we propose a new multimodal emotional state estimation model that uses various responses including facial expressions, gestures, and movements measured by the Microsoft Kinect Sensor. In order to effectively handle a large amount of sensory data, we propose to use stratified sampling-based MRA (multiple regression analysis) as our estimation method. To validate the usefulness of the proposed model, we collected 602,599 responses and emotional state data with 274 variables from 15 people. When we applied our model to the data set, we found that our model estimated the levels of valence and arousal in the 10~15% error range. Since our proposed model is simple and stable, we expect that it will be applied not only in intelligent exhibition services, but also in other areas such as e-learning and personalized advertising.

Representation of Facial Expressions of Different Ages: A Multidimensional Scaling Study (다양한 연령의 얼굴 정서 표상: 다차원척도법 연구)

  • Kim, Jongwan
    • Science of Emotion and Sensibility
    • /
    • v.24 no.3
    • /
    • pp.71-80
    • /
    • 2021
  • Previous studies using facial expressions have revealed valence and arousal as two core dimensions of affective space. However, it remains unknown if the two dimensional structure is consistent across ages. This study investigated affective dimensions using six facial expressions (angry, disgusted, fearful, happy, neutral, and sad) at three ages (young, middle-aged, and old). Several studies previously required participants to directly rate subjective similarity between facial expression pairs. In this study, we collected indirect measures by asking participants to decide if a pair of two stimuli conveyed the same emotions. Multidimensional scaling showed that "angry-disgusted" and "sad-disgusted" pairs are similar at all three ages. In addition, "angry-sad," "angry-neutral," "neutral-sad," and "disgusted-fearful" pairs were similar at old age. When two faces in a pair reflect the same emotion, "sad" was the most inaccurate in old age, suggesting that the ability to recognize "sad" decreases with old age. This study suggested that the general two-core dimension structure is robust across all age groups with the exception of specific emotions.

Consistency between Individuals of Affective Responses for Multiple Modalities based on Behavioral and Physiological Data (행동 및 생리측정기반 개인 간 다중 감각정서 반응일치성)

  • Junhyuk Jang;Jongwan Kim
    • Science of Emotion and Sensibility
    • /
    • v.26 no.1
    • /
    • pp.43-54
    • /
    • 2023
  • In this study, we assessed how participants represent various sensory stimuli experiences through behavioral ratings and physiological measurements. Utilizing intersubject correlation (ISC) analysis, we evaluated whether individuals' affective responses of dominance, arousal, and valence differed when stimuli of three modality conditions (auditory, visual, and haptic) were presented. ISC analyses were used to measure the similarities between one participant's responses and those of the others. To calculate the intersubject correlation, we divided the entire dataset into one subject and all other subject datasets and then correlated the two for all possible stimulus pair combinations. The results revealed that for dominance, ISCs of the visual modality condition were greater than the auditory modality condition, whereas, for arousal, the auditory condition was greater than the visual modality. Last, negative valence conditions had the greater consistency of the participants' reactions than positive conditions in each of the sensory modalities. When comparing modalities, greater ISCs were observed in haptic modality conditions than in visual and auditory modality conditions, regardless of the affective categories. We discussed three core affective representations of multiple modalities and proposed ISC analysis as a tool for examining differences in individuals' affective representations.

Audio and Video Bimodal Emotion Recognition in Social Networks Based on Improved AlexNet Network and Attention Mechanism

  • Liu, Min;Tang, Jun
    • Journal of Information Processing Systems
    • /
    • v.17 no.4
    • /
    • pp.754-771
    • /
    • 2021
  • In the task of continuous dimension emotion recognition, the parts that highlight the emotional expression are not the same in each mode, and the influences of different modes on the emotional state is also different. Therefore, this paper studies the fusion of the two most important modes in emotional recognition (voice and visual expression), and proposes a two-mode dual-modal emotion recognition method combined with the attention mechanism of the improved AlexNet network. After a simple preprocessing of the audio signal and the video signal, respectively, the first step is to use the prior knowledge to realize the extraction of audio characteristics. Then, facial expression features are extracted by the improved AlexNet network. Finally, the multimodal attention mechanism is used to fuse facial expression features and audio features, and the improved loss function is used to optimize the modal missing problem, so as to improve the robustness of the model and the performance of emotion recognition. The experimental results show that the concordance coefficient of the proposed model in the two dimensions of arousal and valence (concordance correlation coefficient) were 0.729 and 0.718, respectively, which are superior to several comparative algorithms.

Effects of LED on Emotion-Like Feedback of a Single-Eyed Spherical Robot

  • Onchi, Eiji;Cornet, Natanya;Lee, SeungHee
    • Science of Emotion and Sensibility
    • /
    • v.24 no.3
    • /
    • pp.115-124
    • /
    • 2021
  • Non-verbal communication is important in human interaction. It provides a layer of information that complements the message being transmitted. This type of information is not limited to human speakers. In human-robot communication, increasing the animacy of the robotic agent-by using non-verbal cues-can aid the expression of abstract concepts such as emotions. Considering the physical limitations of artificial agents, robots can use light and movement to express equivalent emotional feedback. This study analyzes the effects of LED and motion animation of a spherical robot on the emotion being expressed by the robot. A within-subjects experiment was conducted at the University of Tsukuba where participants were asked to rate 28 video samples of a robot interacting with a person. The robot displayed different motions with and without light animations. The results indicated that adding LED animations changes the emotional impression of the robot for valence, arousal, and dominance dimensions. Furthermore, people associated various situations according to the robot's behavior. These stimuli can be used to modulate the intensity of the emotion being expressed and enhance the interaction experience. This paper facilitates the possibility of designing more affective robots in the future, using simple feedback.

On the Predictive Model for Emotion Intensity Improving the Efficacy of Emotionally Supportive Chat (챗봇의 효과적 정서적 지지를 위한 한국어 대화 감정 강도 예측 모델 개발)

  • Sae-Lim Jeong;You-Jin Roh;Eun-Seok Oh;A-Yeon Kim;Hye-Jin Hong;Jee Hang Lee
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2023.11a
    • /
    • pp.656-659
    • /
    • 2023
  • 정서적 지원 대화를 위한 챗봇 개발 시, 사용자의 챗봇에 대한 사용성 및 대화 적절성을 높이기 위해서는 사용자 감정에 적합한 지원 콘텐츠를 제공하는 것이 중요하다. 이를 위해, 본 논문은 사용자 입력 텍스트의 감정 강도 예측 모델을 제안하고, 사용자 발화 맞춤형 정서적 지원 대화에 적용하고자 한다. 먼저 입력된 한국어 문장에서 키워드를 추출한 뒤, 이를 각성도 (arousal)과 긍정부 정도(valence) 공간에 투영하여 키워드가 내포하는 각성도-긍정부정도에 가장 근접한 감정을 예측하였다. 뿐만 아니라, 입력된 전체 문장에 대한 감정 강도를 추가로 예측하여, 핵심 감정 강도 - 문맥상 감정강도를 모두 추출하였다. 이러한 통섭적 감정 강도 지수들은 사용자 감정에 따른 최적 지원 전략 선택 및 최적 대화 콘텐츠 생성에 공헌할 것으로 기대한다.

Application of Support Vector Regression for Improving the Performance of the Emotion Prediction Model (감정예측모형의 성과개선을 위한 Support Vector Regression 응용)

  • Kim, Seongjin;Ryoo, Eunchung;Jung, Min Kyu;Kim, Jae Kyeong;Ahn, Hyunchul
    • Journal of Intelligence and Information Systems
    • /
    • v.18 no.3
    • /
    • pp.185-202
    • /
    • 2012
  • .Since the value of information has been realized in the information society, the usage and collection of information has become important. A facial expression that contains thousands of information as an artistic painting can be described in thousands of words. Followed by the idea, there has recently been a number of attempts to provide customers and companies with an intelligent service, which enables the perception of human emotions through one's facial expressions. For example, MIT Media Lab, the leading organization in this research area, has developed the human emotion prediction model, and has applied their studies to the commercial business. In the academic area, a number of the conventional methods such as Multiple Regression Analysis (MRA) or Artificial Neural Networks (ANN) have been applied to predict human emotion in prior studies. However, MRA is generally criticized because of its low prediction accuracy. This is inevitable since MRA can only explain the linear relationship between the dependent variables and the independent variable. To mitigate the limitations of MRA, some studies like Jung and Kim (2012) have used ANN as the alternative, and they reported that ANN generated more accurate prediction than the statistical methods like MRA. However, it has also been criticized due to over fitting and the difficulty of the network design (e.g. setting the number of the layers and the number of the nodes in the hidden layers). Under this background, we propose a novel model using Support Vector Regression (SVR) in order to increase the prediction accuracy. SVR is an extensive version of Support Vector Machine (SVM) designated to solve the regression problems. The model produced by SVR only depends on a subset of the training data, because the cost function for building the model ignores any training data that is close (within a threshold ${\varepsilon}$) to the model prediction. Using SVR, we tried to build a model that can measure the level of arousal and valence from the facial features. To validate the usefulness of the proposed model, we collected the data of facial reactions when providing appropriate visual stimulating contents, and extracted the features from the data. Next, the steps of the preprocessing were taken to choose statistically significant variables. In total, 297 cases were used for the experiment. As the comparative models, we also applied MRA and ANN to the same data set. For SVR, we adopted '${\varepsilon}$-insensitive loss function', and 'grid search' technique to find the optimal values of the parameters like C, d, ${\sigma}^2$, and ${\varepsilon}$. In the case of ANN, we adopted a standard three-layer backpropagation network, which has a single hidden layer. The learning rate and momentum rate of ANN were set to 10%, and we used sigmoid function as the transfer function of hidden and output nodes. We performed the experiments repeatedly by varying the number of nodes in the hidden layer to n/2, n, 3n/2, and 2n, where n is the number of the input variables. The stopping condition for ANN was set to 50,000 learning events. And, we used MAE (Mean Absolute Error) as the measure for performance comparison. From the experiment, we found that SVR achieved the highest prediction accuracy for the hold-out data set compared to MRA and ANN. Regardless of the target variables (the level of arousal, or the level of positive / negative valence), SVR showed the best performance for the hold-out data set. ANN also outperformed MRA, however, it showed the considerably lower prediction accuracy than SVR for both target variables. The findings of our research are expected to be useful to the researchers or practitioners who are willing to build the models for recognizing human emotions.

Reproducibility of physiological patterns in disgust visual stimulation design

  • Lee, Kyung-Hwa;E. Sokhadze;Jang, Eun-Hye;Yang, Gyung-Hye;Sohn, Jin-Hun
    • Proceedings of the Korean Society for Emotion and Sensibility Conference
    • /
    • 2000.11a
    • /
    • pp.73-80
    • /
    • 2000
  • The paper is addressed to the topic of physiological response-specificity in disgust induced by visual stimulation. The purpose of this study was to evaluate reproducibility of physiological reactivity pattern during disgust elicited by the International Affective Pictures System (IAPS) in 2 experiments. Twenty-nine subjects participated in the first experiment with 3 visual stimulation sessions with disgust-eliciting slides (3 slides in each 1 min long session). In the second experiment disgust-eliciting slides from the IAPS were presented to 42 subjects in 2 sessions (one slide for 1 min). Spectral power of frontal EEG, skin conductance (SCL, SCR and NS.SCR), heart rate(HR), heart period variability(HPV) and respiration rate were recorded. Visual stimulation evoked 1:.n deceleration, higher power of high frequency component of HPV, increased SCL and NS.SCR frequency, frontal slow alpha blocking and moderate increase in fast beta power in most of the sessions in both experiments. However in the second experiment the EEG pattern associated with disgust showed inconsistent shifts in fast alpha and slow beta bands, but was marked by higher power of theta activity. Our data in both experiments emphasizes presence of disgust-specific profiles of autonomic and at the less extent EEG responses in visual stimulation context. Discussed are potential behavioral mechanisms leading to observed physiological manifestations in disgust elicited by visual stimulation. The results support the consideration that disgust is an withdrawal type negative valence emotion associated with relatively low autonomic arousal (low HR, low amplitude SCRs with relatively high NS.SCR frequency) and moderate EEG activation signs. Obtained data showed more consistent reproducibility of disgust-specific autonomic rather than EEG response patterns during visual stimulation design.

  • PDF

A Classification and Selection Method of Emotion Based on Classifying Emotion Terms by Users (사용자의 정서 단어 분류에 기반한 정서 분류와 선택 방법)

  • Rhee, Shin-Young;Ham, Jun-Seok;Ko, Il-Ju
    • Science of Emotion and Sensibility
    • /
    • v.15 no.1
    • /
    • pp.97-104
    • /
    • 2012
  • Recently, a big text data has been produced by users, an opinion mining to analyze information and opinion about users is becoming a hot issue. Of the opinion mining, especially a sentiment analysis is a study for analysing emotions such as a positive, negative, happiness, sadness, and so on analysing personal opinions or emotions for commercial products, social issues and opinions of politician. To analyze the sentiment analysis, previous studies used a mapping method setting up a distribution of emotions using two dimensions composed of a valence and arousal. But previous studies set up a distribution of emotions arbitrarily. In order to solve the problem, we composed a distribution of 12 emotions through carrying out a survey using Korean emotion words list. Also, certain emotional states on two dimension overlapping multiple emotions, we proposed a selection method with Roulette wheel method using a selection probability. The proposed method shows to classify a text into emotion extracting emotion terms from a text.

  • PDF

Comparison Between Core Affect Dimensional Structures of Different Ages using Representational Similarity Analysis (표상 유사성 분석을 이용한 연령별 얼굴 정서 차원 비교)

  • Jongwan Kim
    • Science of Emotion and Sensibility
    • /
    • v.26 no.1
    • /
    • pp.33-42
    • /
    • 2023
  • Previous emotion studies employing facial expressions have focused on the differences between age groups for each of the emotion categories. Instead, Kim (2021) has compared representations of facial expressions in the lower-dimensional emotion space. However, he reported descriptive comparisons without statistical significance testing. This research used representational similarity analysis (Kriegeskorte et al., 2008) to directly compare empirical datasets from young, middle-aged, and old groups and conceptual models. In addition, individual differences multidimensional scaling (Carroll & Chang, 1970) was conducted to explore individual weights on the emotional dimensions for each age group. The results revealed that the old group was the least similar to the other age groups in the empirical datasets and the valence model. In addition, the arousal dimension was the least weighted for the old group compared to the other groups. This study directly tested the differences between the three age groups in terms of empirical datasets, conceptual models, and weights on the emotion dimensions.