• The Journal of Korea Robotics Society
• /
• v.17 no.3
• /
• pp.264-272
• /
• 2022

Human-robot cooperative tasks are increasingly required in our daily life with the development of robotics and artificial intelligence technology. Interactive reinforcement learning strategies suggest that robots learn task by receiving feedback from an experienced human trainer during a training process. However, most of the previous studies on Interactive reinforcement learning have required an extra feedback input device such as a mouse or keyboard in addition to robot itself, and the scenario where a robot can interactively learn a task with human have been also limited to virtual environment. To solve these limitations, this paper studies training strategies of robot that learn table balancing tasks interactively using deep reinforcement learning with human's facial expression feedback. In the proposed system, the robot learns a cooperative table balancing task using Deep Q-Network (DQN), which is a deep reinforcement learning technique, with human facial emotion expression feedback. As a result of the experiment, the proposed system achieved a high optimal policy convergence rate of up to 83.3% in training and successful assumption rate of up to 91.6% in testing, showing improved performance compared to the model without human facial expression feedback.

• International Journal of Control, Automation, and Systems
• /
• v.1 no.2
• /
• pp.236-242
• /
• 2003

This paper presents a cooperation strategy between a human operator and autonomous robots for an interactive robot soccer game, The interactive robot soccer game has been developed to allow humans to join into the game dynamically and reinforce entertainment characteristics. In order to make these games more interesting, a cooperation strategy between humans and autonomous robots on a team is very important. Strategies can be pre-programmed or learned by robots themselves with learning or evolving algorithms. Since the robot soccer system is hard to model and its environment changes dynamically, it is very difficult to pre-program cooperation strategies between robot agents. Q-learning - one of the most representative reinforcement learning methods - is shown to be effective for solving problems dynamically without explicit knowledge of the system. Therefore, in our research, a Q-learning based learning method has been utilized. Prior to utilizing Q-teaming, state variables describing the game situation and actions' sets of robots have been defined. After the learning process, the human operator could play the game more easily. To evaluate the usefulness of the proposed strategy, some simulations and games have been carried out.

• Journal of Institute of Control, Robotics and Systems
• /
• v.12 no.7
• /
• pp.666-670
• /
• 2006

This paper presents the novel feature selection method for Emotion Recognition, which may include a lot of original features. Specially, the emotion recognition in this paper treated speech signal with emotion. The feature selection has some benefits on the pattern recognition performance and 'the curse of dimension'. Thus, We implemented a simulator called 'IFS' and those result was applied to a emotion recognition system(ERS), which was also implemented for this research. Our novel feature selection method was basically affected by Reinforcement Learning and since it needs responses from human user, it is called 'Interactive feature Selection'. From performing the IFS, we could get 3 best features and applied to ERS. Comparing those results with randomly selected feature set, The 3 best features were better than the randomly selected feature set.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.6 no.4
• /
• pp.282-287
• /
• 2006

This paper presents an original feature selection method for Emotion Recognition which includes many original elements. Feature selection has some merits regarding pattern recognition performance. Thus, we developed a method called thee 'Interactive Feature Selection' and the results (selected features) of the IFS were applied to an emotion recognition system (ERS), which was also implemented in this research. The innovative feature selection method was based on a Reinforcement Learning Algorithm and since it required responses from human users, it was denoted an 'Interactive Feature Selection'. By performing an IFS, we were able to obtain three top features and apply them to the ERS. Comparing those results from a random selection and Sequential Forward Selection (SFS) and Genetic Algorithm Feature Selection (GAFS), we verified that the top three features were better than the randomly selected feature set.

• ETRI Journal
• /
• v.38 no.6
• /
• pp.1229-1239
• /
• 2016

For efficient interaction between humans and robots, robots should be able to understand the meaning and intention of human behaviors as well as recognize them. This paper proposes an interactive human intention reading method in which a robot develops its own knowledge about the human intention for an object. A robot needs to understand different human behavior structures for different objects. To this end, this paper proposes a hierarchical behavior knowledge network that consists of behavior nodes and directional edges between them. In addition, a human intention reading algorithm that incorporates reinforcement learning is proposed to interactively learn the hierarchical behavior knowledge networks based on context information and human feedback through human behaviors. The effectiveness of the proposed method is demonstrated through play-based experiments between a human and a virtual teddy bear robot with two virtual objects. Experiments with multiple participants are also conducted.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.16 no.6
• /
• pp.647-652
• /
• 2006

This paper presents the novel feature selection method for Emotion Recognition, which may include a lot of original features. Specially, the emotion recognition in this paper treated speech signal with emotion. The feature selection has some benefits on the pattern recognition performance and 'the curse of dimension'. Thus, We implemented a simulator called 'IFS' and those result was applied to a emotion recognition system(ERS), which was also implemented for this research. Our novel feature selection method was basically affected by Reinforcement Learning and since it needs responses from human user, it is called 'Interactive Feature Selection'. From performing the IFS, we could get 3 best features and applied to ERS. Comparing those results with randomly selected feature set, The 3 best features were better than the randomly selected feature set.

• Proceedings of the KIEE Conference
• /
• 2005.10b
• /
• pp.174-176
• /
• 2005

This treatise studies composing process that renew information mastered by interactive experience between environment and system via network among individuals. In the previous study map information regarding free space is learned by using of reinforced learning algorithm, which enable each individual to construct optimal action policy. Based on those action policy each individuals can obtain optimal path. Moreover decision process to distinguish best optimal path by comparing those in the network composed of each individuals is added. Also information about the finally chosen path is being updated. A self renewing method of each system information by sharing the each individual data via network is proposed Data enrichment by shilling the information of many maps not in the single map is tried Numerical simulation is conducted to confirm the propose concept. In order to prove its suitability experiment using micro-mouse by integrating and comparing the information between individuals is carried out in various types of map to reveal successful result.

• Steel and Composite Structures
• /
• v.49 no.1
• /
• pp.109-123
• /
• 2023

Headed studs welded to steel beams and embedded within the concrete of deck slabs are vital components of modern composite floor systems, where safety and economy depend on the accurate predictions of the stud shear resistance. The multitude of existing deck profiles and the complex behavior of studs in deck slab ribs makes developing accurate and reliable mechanical or empirical design models challenging. The paper addresses this issue by presenting a machine learning (ML) model developed from the natural gradient boosting (NGBoost) algorithm capable of producing probabilistic predictions and a database of 464 push-out tests, which is considerably larger than the databases used for developing existing design models. The proposed model outperforms models based on other ML algorithms and existing descriptive equations, including those in EC4 and AISC 360, while offering probabilistic predictions unavailable from other models and producing higher shear resistances for many cases. The present study also showed that the stud shear resistance is insensitive to the concrete elastic modulus, stud welding type, location of slab reinforcement, and other parameters considered important by existing models. The NGBoost model was interpreted by evaluating the feature importance and dependence determined with the SHapley Additive exPlanations (SHAP) method. The model was calibrated via reliability analyses in accordance with the Eurocodes to ensure that its predictions meet the required reliability level and facilitate its use in design. An interactive open-source web application was created and deployed to the cloud to allow for convenient and rapid stud shear resistance predictions with the developed model.

• Journal of Institute of Control, Robotics and Systems
• /
• v.13 no.9
• /
• pp.832-837
• /
• 2007

Electroencephalographic(EEG) is used to record activities of human brain in the area of psychology for many years. As technology develope, neural basis of functional areas of emotion processing is revealed gradually. So we measure fundamental areas of human brain that controls emotion of human by using EEG. Hands gestures such as shaking and head gesture such as nodding are often used as human body languages for communication with each other, and their recognition is important that it is a useful communication medium between human and computers. Research methods about gesture recognition are used of computer vision. Many researchers study Emotion Recognition method which uses one of EEG signals and Gestures in the existing research. In this paper, we use together EEG signals and Gestures for Emotion Recognition of human. And we select the driver emotion as a specific target. The experimental result shows that using of both EEG signals and gestures gets high recognition rates better than using EEG signals or gestures. Both EEG signals and gestures use Interactive Feature Selection(IFS) for the feature selection whose method is based on a reinforcement learning.

• Journal of KIISE:Computer Systems and Theory
• /
• v.33 no.4
• /
• pp.231-241
• /
• 2006

Creating controllable, responsive avatars is an important problem in computer games and virtual environments. Recently, large collections of motion capture data have been exploited for increased realism in avatar animation and control. Large motion sets have the advantage of accommodating a broad variety of natural human motion. However, when a motion set is large, the time required to identify an appropriate sequence of motions is the bottleneck for achieving interactive avatar control. In this paper, we present a novel method for training avatar behaviors from unlabelled motion data in order to animate and control avatars at minimal runtime cost. Based on machine learning technique, called Q-teaming, our training method allows the avatar to learn how to act in any given situation through trial-and-error interactions with a dynamic environment. We demonstrate the effectiveness of our approach through examples that include avatars interacting with each other and with the user.