• Journal of Korean Society of Industrial and Systems Engineering
• /
• v.37 no.1
• /
• pp.1-8
• /
• 2014

This paper proposes an emotion classifier from EEG signals based on Bayes' theorem and a machine learning using a perceptron convergence algorithm. The emotions are represented on the valence and arousal dimensions. The fast Fourier transform spectrum analysis is used to extract features from the EEG signals. To verify the proposed method, we use an open database for emotion analysis using physiological signal (DEAP) and compare it with C-SVC which is one of the support vector machines. An emotion is defined as two-level class and three-level class in both valence and arousal dimensions. For the two-level class case, the accuracy of the valence and arousal estimation is 67% and 66%, respectively. For the three-level class case, the accuracy is 53% and 51%, respectively. Compared with the best case of the C-SVC, the proposed classifier gave 4% and 8% more accurate estimations of valence and arousal for the two-level class. In estimation of three-level class, the proposed method showed a similar performance to the best case of the C-SVC.

• Science of Emotion and Sensibility
• /
• v.5 no.1
• /
• pp.17-24
• /
• 2002

This research was the first step to develop Expert System for Evaluation of Human Sensibility, where human sensibility can be inferred from objective physiological signals. The study aim was to develop an algorithm in which human arousal level can be judged using measured physiological signals. Fuzzy theory was applied for mathematical handling of the ambiguity related to evaluation of human sensibility, and the degree of belonging to a certain sensibility dimension was quantified by membership function through which the sensibility evaluation was able to be done. Determining membership function was achieved using results from a physiological signal database of arousal/relaxation that was generated from imagination. To induce one final result (arousal level) based on measuring the results of more than 2 physiological signals and the membership function of each physiological signal, Dempster-Shafer's Rule of Combination in Evidence was applied, through which the final arousal level was inferred.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2020.11a
• /
• pp.79-82
• /
• 2020

얼굴 영상으로부터 사람의 감정을 예측하는 연구는 최근 딥러닝의 발전과 함께 주목받고 있다. 본 연구에서 우리는 연속적인 변수를 사용하여 감정을 표현하는 dimensional model에 기반하여 얼굴 영상으로부터 감정 상태를 나타내는 지표인 valance/arousal(V/A)을 예측하는 딥러닝 네트워크를 제안한다. 그러나 V/A 예측 모델의 학습에 사용되는 기존의 데이터셋들은 데이터 불균형(data imbalance) 문제를 가진다. 이를 해소하기 위해, 우리는 오토인코더 구조를 가지는 얼굴 영상 생성 네트워크를 학습하고, 이로부터 얻어지는 균일한 분포의 데이터로부터 V/A 예측 네트워크를 학습한다. 실험을 통해 우리는 제안하는 얼굴 생성 오토인코더가 in-the-wild 환경의 데이터셋으로부터 임의의 valence, arousal에 대응하는 얼굴 영상을 성공적으로 생생함을 보인다. 그리고, 이를 통해 학습된 V/A 예측 네트워크가 기존의 under-sampling, over-sampling 방영들과 비교하여 더 높은 인식 성능을 달성함을 보인다. 마지막으로 기존의 방법들과 제안하는 V/A 예측 네트워크의 성능을 정량적으로 비교한다.

• Proceedings of the Korean Society for Emotion and Sensibility Conference
• /
• 2002.05a
• /
• pp.86-89
• /
• 2002

본 연구는 객관적인 생리 신호로부터 인간의 감성을 추론할 수 있는 감성 평가 전문가 시스템을 개발하기 위한 첫 번째 단계로, 측정된 생리 신호를 이용하여 인간의 긴장도를 판단하는 알고리즘의 개발이 목표이다. 감성 평가에 관련된 애매함을 수리적으로 취급하기 위해 퍼지 이론을 적용하여 임의의 감성 영역에 속하는 정도를 소속 함수로 정량화 함으로써 감성 평가를 가능하게 하고자 하였다 소속 함수의 결정은 상상을 통해 유발된 긴장/이완의 생리 신호 데이터 베이스 결과를 사용하였다. 그리고 두 가지 이상의 생리 신호 측정 결과와 각 생리 신호의 소속 함수로부터 하나의 최종 결과 (긴장도)를 유추하기 위해서 Dempster-Shafer 증거합 법칙을 적용하였고, 이를 통해 최종적인 긴장도를 도출할 수 있도록 하였다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.8 no.6
• /
• pp.2056-2069
• /
• 2014

In order to implement interactive and personalized Web services properly, it is necessary to understand the tangible and intangible responses of the users and to recognize their emotional states. Recently, some studies have attempted to build emotional state estimation models based on facial expressions. Most of these studies have applied multiple regression analysis (MRA), artificial neural network (ANN), and support vector regression (SVR) as the prediction algorithm, but the prediction accuracies have been relatively low. In order to improve the prediction performance of the emotion prediction model, we propose a novel SVR model that is optimized using a genetic algorithm (GA). Our proposed algorithm-GASVR-is designed to optimize the kernel parameters and the feature subsets of SVRs in order to predict the levels of two aspects-valence and arousal-of the emotions of the users. In order to validate the usefulness of GASVR, we collected a real-world data set of facial responses and emotional states via a survey. We applied GASVR and other algorithms including MRA, ANN, and conventional SVR to the data set. Finally, we found that GASVR outperformed all of the comparative algorithms in the prediction of the valence and arousal levels.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.24 no.3
• /
• pp.316-322
• /
• 2014

A movie recommendation system is proposed to learn a preference model of a viewer by using multimodal features of a video content and their evoked implicit responses of the viewer in synchronized manner. In this system, facial expression, body posture, and physiological signals are measured to estimate the affective states of the viewer, in accordance with the stimuli consisting of low-level and affective features from video, audio, and text streams. Experimental results show that it is possible to predict arousal response, which is measured by electrodermal activity, of a viewer from auditory and text features in a video stimuli, for estimating interestingness on the video.

• Journal of Intelligence and Information Systems
• /
• v.20 no.1
• /
• pp.1-14
• /
• 2014

Both researchers and practitioners are showing an increased interested in interactive exhibition services. Interactive exhibition services are designed to directly respond to visitor responses in real time, so as to fully engage visitors' interest and enhance their satisfaction. In order to install an effective interactive exhibition service, it is essential to adopt intelligent technologies that enable accurate estimation of a visitor's emotional state from responses to exhibited stimulus. Studies undertaken so far have attempted to estimate the human emotional state, most of them doing so by gauging either facial expressions or audio responses. However, the most recent research suggests that, a multimodal approach that uses people's multiple responses simultaneously may lead to better estimation. Given this context, we propose a new multimodal emotional state estimation model that uses various responses including facial expressions, gestures, and movements measured by the Microsoft Kinect Sensor. In order to effectively handle a large amount of sensory data, we propose to use stratified sampling-based MRA (multiple regression analysis) as our estimation method. To validate the usefulness of the proposed model, we collected 602,599 responses and emotional state data with 274 variables from 15 people. When we applied our model to the data set, we found that our model estimated the levels of valence and arousal in the 10~15% error range. Since our proposed model is simple and stable, we expect that it will be applied not only in intelligent exhibition services, but also in other areas such as e-learning and personalized advertising.

• Journal of Distribution Research
• /
• v.17 no.2
• /
• pp.63-92
• /
• 2012

This study investigated consumer intention to use a location-based mobile shopping service (LBMSS) that integrates cognitive and affective responses. Information relevancy was integrated into pleasure-arousal-dominance (PAD) emotional state model in the present study as a conceptual framework. The results of an online survey of 335 mobile phone users in the U.S. indicated the positive effects of arousal and information relevancy on pleasure. In addition, there was a significant relationship between pleasure and intention to use a LBMSS. However, the relationship between dominance and pleasure was not statistically significant. The results of the present study provides insight to retailers and marketers as to what factors they need to consider to implement location-based mobile shopping services to improve their business performance. Extended Abstract : Location aware technology has expanded the marketer's reach by reducing space and time between a consumer's receipt of advertising and purchase, offering real-time information and coupons to consumers in purchasing situations (Dickenger and Kleijnen, 2008; Malhotra and Malhotra, 2009). LBMSS increases the relevancy of SMS marketing by linking advertisements to a user's location (Bamba and Barnes, 2007; Malhotra and Malhotra, 2009). This study investigated consumer intention to use a location-based mobile shopping service (LBMSS) that integrates cognitive and affective response. The purpose of the study was to examine the relationship among information relevancy and affective variables and their effects on intention to use LBMSS. Thus, information relevancy was integrated into pleasure-arousal-dominance (PAD) model and generated the following hypotheses. Hypothesis 1. There will be a positive influence of arousal concerning LBMSS on pleasure in regard to LBMSS. Hypothesis 2. There will be a positive influence of dominance in LBMSS on pleasure in regard to LBMSS. Hypothesis 3. There will be a positive influence of information relevancy on pleasure in regard to LBMSS. Hypothesis 4. There will be a positive influence of pleasure about LBMSS on intention to use LBMSS. E-mail invitations were sent out to a randomly selected sample of three thousand consumers who are older than 18 years old and mobile phone owners, acquired from an independent marketing research company. An online survey technique was employed utilizing Dillman's (2000) online survey method and follow-ups. A total of 335 valid responses were used for the data analysis in the present study. Before the respondents answer any of the questions, they were told to read a document describing LBMSS. The document included definitions and examples of LBMSS provided by various service providers. After that, they were exposed to a scenario describing the participant as taking a saturday shopping trip to a mall and then receiving a short message from the mall. The short message included new product information and coupons for same day use at participating stores. They then completed a questionnaire containing various questions. To assess arousal, dominance, and pleasure, we adapted and modified scales used in the previous studies in the context of location-based mobile shopping service, each of the five items from Mehrabian and Russell (1974). A total of 15 items were measured on a seven-point bipolar scale. To measure information relevancy, four items were borrowed from Mason et al. (1995). Intention to use LBMSS was captured using two items developed by Blackwell, and Miniard (1995) and one items developed by the authors. Data analyses were conducted using SPSS 19.0 and LISREL 8.72. A total of usable 335 data were obtained after deleting the incomplete responses, which results in a response rate of 11.20%. A little over half of the respondents were male (53.9%) and approximately 60% of respondents were married (57.4%). The mean age of the sample was 29.44 years with a range from 19 to 60 years. In terms of the ethnicity there were European Americans (54.5%), Hispanic American (5.3%), African-American (3.6%), and Asian American (2.9%), respectively. The respondents were highly educated; close to 62.5% of participants in the study reported holding a college degree or its equivalent and 14.5% of the participants had graduate degree. The sample represents all income categories: less than $24,999 (10.8%), $25,000-$49,999 (28.34%), $50,000-$74,999 (13.8%), and $75,000 or more (10.23%). The respondents of the study indicated that they were employed in many occupations. Responses came from all 42 states in the U.S. To identify the dimensions of research constructs, Exploratory Factor Analysis (EFA) using a varimax rotation was conducted. As indicated in table 1, these dimensions: arousal, dominance, relevancy, pleasure, and intention to use, suggested by the EFA, explained 82.29% of the total variance with factor loadings ranged from .74 to .89. As a next step, CFA was conducted to validate the dimensions that were identified from the exploratory factor analysis and to further refine the scale. Table 1 exhibits the results of measurement model analysis and revealed a chi-square of 202.13 with degree-of-freedom of 89 (p =.002), GFI of .93, AGFI = .89, CFI of .99, NFI of .98, which indicates of the evidence of a good model fit to the data (Bagozzi and Yi, 1998; Hair et al., 1998). As table 1 shows, reliability was estimated with Cronbach's alpha and composite reliability (CR) for all multi-item scales. All the values met evidence of satisfactory reliability in multi-item measure for alpha (>.91) and CR (>.80). In addition, we tested the convergent validity of the measure using average variance extracted (AVE) by following recommendations from Fornell and Larcker (1981). The AVE values for the model constructs ranged from .74 through .85, which are higher than the threshold suggested by Fornell and Larcker (1981). To examine discriminant validity of the measure, we again followed the recommendations from Fornell and Larcker (1981). The shared variances between constructs were smaller than the AVE of the research constructs and confirm discriminant validity of the measure. The causal model testing was conducted using LISREL 8.72 with a maximum-likelihood estimation method. Table 2 shows the results of the hypotheses testing. The results for the conceptual model revealed good overall fit for the proposed model. Chi-square was 342.00 (df = 92, p =.000), NFI was .97, NNFI was .97, GFI was .89, AGFI was .83, and RMSEA was .08. All paths in the proposed model received significant statistical support except H2. The paths from arousal to pleasure (H1: ${\ss}$=.70; t = 11.44), from information relevancy to intention to use (H3 ${\ss}$ =.12; t = 2.36), from information relevancy to pleasure (H4 ${\ss}$ =.15; t = 2.86), and pleasure to intention to use (H5: ${\ss}$=.54; t = 9.05) were significant. However, the path from dominance to pleasure was not supported. This study investigated consumer intention to use a location-based mobile shopping service (LBMSS) that integrates cognitive and affective responses. Information relevancy was integrated into pleasure-arousal-dominance (PAD) emotional state model as a conceptual framework. The results of the present study support previous studies indicating that emotional responses as well as cognitive responses have a strong impact on accepting new technology. The findings of this study suggest potential marketing strategies to mobile service developers and retailers who are considering the implementation of LBMSS. It would be rewarding to develop location-based mobile services that integrate information relevancy and which cause positive emotional responses.

• Phonetics and Speech Sciences
• /
• v.12 no.3
• /
• pp.25-32
• /
• 2020

An analysis of emotional English utterances with 7 emotions (calm, happy, sad, angry, fearful, disgust, surprised) was conducted using the measurement of prosodic distance between 672 emotional and 48 neutral utterances. Applying the technique proposed in the automatic evaluation model of English pronunciation to the present study on emotional utterances, Euclidean distance measurement of 3 prosodic elements such as F0, intensity and duration extracted from emotional and neutral utterances was utilized. This paper, furthermore, extended the analytical methods to include Euclidean distance normalization, z-score and z-score normalization resulting in 4 groups of measurement schemes (sqrF0, sqrINT, sqrDUR; norsqrF0, norsqrINT, norsqrDUR; sqrzF0, sqrzINT, sqrzDUR; norsqrzF0, norsqrzINT, norsqrzDUR). All of the results from perceptual analysis and acoustical analysis of emotional utteances consistently indicated the greater effectiveness of norsqrF0, norsqrINT and norsqrDUR, among 4 groups of measurement schemes, which normalized the Euclidean measurement. The greatest acoustical change of prosodic information influenced by emotion was shown in the values of F0 followed by duration and intensity in descending order according to the effect size based on the estimation of distance between emotional utterances and neutral counterparts. Tukey Post Hoc test revealed 4 homogeneous subsets (calm