• 로봇학회논문지
• /
• 제17권3호
• /
• pp.264-272
• /
• 2022

Human-robot cooperative tasks are increasingly required in our daily life with the development of robotics and artificial intelligence technology. Interactive reinforcement learning strategies suggest that robots learn task by receiving feedback from an experienced human trainer during a training process. However, most of the previous studies on Interactive reinforcement learning have required an extra feedback input device such as a mouse or keyboard in addition to robot itself, and the scenario where a robot can interactively learn a task with human have been also limited to virtual environment. To solve these limitations, this paper studies training strategies of robot that learn table balancing tasks interactively using deep reinforcement learning with human's facial expression feedback. In the proposed system, the robot learns a cooperative table balancing task using Deep Q-Network (DQN), which is a deep reinforcement learning technique, with human facial emotion expression feedback. As a result of the experiment, the proposed system achieved a high optimal policy convergence rate of up to 83.3% in training and successful assumption rate of up to 91.6% in testing, showing improved performance compared to the model without human facial expression feedback.

• International Journal of Control, Automation, and Systems
• /
• 제1권2호
• /
• pp.236-242
• /
• 2003

This paper presents a cooperation strategy between a human operator and autonomous robots for an interactive robot soccer game, The interactive robot soccer game has been developed to allow humans to join into the game dynamically and reinforce entertainment characteristics. In order to make these games more interesting, a cooperation strategy between humans and autonomous robots on a team is very important. Strategies can be pre-programmed or learned by robots themselves with learning or evolving algorithms. Since the robot soccer system is hard to model and its environment changes dynamically, it is very difficult to pre-program cooperation strategies between robot agents. Q-learning - one of the most representative reinforcement learning methods - is shown to be effective for solving problems dynamically without explicit knowledge of the system. Therefore, in our research, a Q-learning based learning method has been utilized. Prior to utilizing Q-teaming, state variables describing the game situation and actions' sets of robots have been defined. After the learning process, the human operator could play the game more easily. To evaluate the usefulness of the proposed strategy, some simulations and games have been carried out.

• 제어로봇시스템학회논문지
• /
• 제12권7호
• /
• pp.666-670
• /
• 2006

This paper presents the novel feature selection method for Emotion Recognition, which may include a lot of original features. Specially, the emotion recognition in this paper treated speech signal with emotion. The feature selection has some benefits on the pattern recognition performance and 'the curse of dimension'. Thus, We implemented a simulator called 'IFS' and those result was applied to a emotion recognition system(ERS), which was also implemented for this research. Our novel feature selection method was basically affected by Reinforcement Learning and since it needs responses from human user, it is called 'Interactive feature Selection'. From performing the IFS, we could get 3 best features and applied to ERS. Comparing those results with randomly selected feature set, The 3 best features were better than the randomly selected feature set.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• 제6권4호
• /
• pp.282-287
• /
• 2006

This paper presents an original feature selection method for Emotion Recognition which includes many original elements. Feature selection has some merits regarding pattern recognition performance. Thus, we developed a method called thee 'Interactive Feature Selection' and the results (selected features) of the IFS were applied to an emotion recognition system (ERS), which was also implemented in this research. The innovative feature selection method was based on a Reinforcement Learning Algorithm and since it required responses from human users, it was denoted an 'Interactive Feature Selection'. By performing an IFS, we were able to obtain three top features and apply them to the ERS. Comparing those results from a random selection and Sequential Forward Selection (SFS) and Genetic Algorithm Feature Selection (GAFS), we verified that the top three features were better than the randomly selected feature set.

• ETRI Journal
• /
• 제38권6호
• /
• pp.1229-1239
• /
• 2016

For efficient interaction between humans and robots, robots should be able to understand the meaning and intention of human behaviors as well as recognize them. This paper proposes an interactive human intention reading method in which a robot develops its own knowledge about the human intention for an object. A robot needs to understand different human behavior structures for different objects. To this end, this paper proposes a hierarchical behavior knowledge network that consists of behavior nodes and directional edges between them. In addition, a human intention reading algorithm that incorporates reinforcement learning is proposed to interactively learn the hierarchical behavior knowledge networks based on context information and human feedback through human behaviors. The effectiveness of the proposed method is demonstrated through play-based experiments between a human and a virtual teddy bear robot with two virtual objects. Experiments with multiple participants are also conducted.

• 한국지능시스템학회논문지
• /
• 제16권6호
• /
• pp.647-652
• /
• 2006

본 논문은 일반적으로 많은 특징들을 갖고 있는 패턴 분류 문제인 감정 인식을 위한 새로운 특징 선택 방법을 제안한다. '특징 선택'은 패턴 인식 성능의 향상에 기여하고 '차원의 저주'문제에도 좋은 해결책으로 많이 사용되는 방법이다. 그래서, 본 논문에서는 강화학습의 개념을 사용한 상호 작용에 의한 특징 선택 방법인 IFS(Interactiv Feature Selection)를 고안하였고 이 알고리즘을 사용하여 선택된 특징들을 감정 인식 시스템에 적용하여 성능이 향상됨을 확인하였다. 또한 기존의 특징 선택 방법과의 비교를 통하여 본 알고리즘의 우수성을 확인하였다.

• 대한전기학회:학술대회논문집
• /
• 대한전기학회 2005년도 학술대회 논문집 정보 및 제어부문
• /
• pp.174-176
• /
• 2005

This treatise studies composing process that renew information mastered by interactive experience between environment and system via network among individuals. In the previous study map information regarding free space is learned by using of reinforced learning algorithm, which enable each individual to construct optimal action policy. Based on those action policy each individuals can obtain optimal path. Moreover decision process to distinguish best optimal path by comparing those in the network composed of each individuals is added. Also information about the finally chosen path is being updated. A self renewing method of each system information by sharing the each individual data via network is proposed Data enrichment by shilling the information of many maps not in the single map is tried Numerical simulation is conducted to confirm the propose concept. In order to prove its suitability experiment using micro-mouse by integrating and comparing the information between individuals is carried out in various types of map to reveal successful result.

• Steel and Composite Structures
• /
• 제49권1호
• /
• pp.109-123
• /
• 2023

Headed studs welded to steel beams and embedded within the concrete of deck slabs are vital components of modern composite floor systems, where safety and economy depend on the accurate predictions of the stud shear resistance. The multitude of existing deck profiles and the complex behavior of studs in deck slab ribs makes developing accurate and reliable mechanical or empirical design models challenging. The paper addresses this issue by presenting a machine learning (ML) model developed from the natural gradient boosting (NGBoost) algorithm capable of producing probabilistic predictions and a database of 464 push-out tests, which is considerably larger than the databases used for developing existing design models. The proposed model outperforms models based on other ML algorithms and existing descriptive equations, including those in EC4 and AISC 360, while offering probabilistic predictions unavailable from other models and producing higher shear resistances for many cases. The present study also showed that the stud shear resistance is insensitive to the concrete elastic modulus, stud welding type, location of slab reinforcement, and other parameters considered important by existing models. The NGBoost model was interpreted by evaluating the feature importance and dependence determined with the SHapley Additive exPlanations (SHAP) method. The model was calibrated via reliability analyses in accordance with the Eurocodes to ensure that its predictions meet the required reliability level and facilitate its use in design. An interactive open-source web application was created and deployed to the cloud to allow for convenient and rapid stud shear resistance predictions with the developed model.

• 제어로봇시스템학회논문지
• /
• 제13권9호
• /
• pp.832-837
• /
• 2007

Electroencephalographic(EEG) is used to record activities of human brain in the area of psychology for many years. As technology develope, neural basis of functional areas of emotion processing is revealed gradually. So we measure fundamental areas of human brain that controls emotion of human by using EEG. Hands gestures such as shaking and head gesture such as nodding are often used as human body languages for communication with each other, and their recognition is important that it is a useful communication medium between human and computers. Research methods about gesture recognition are used of computer vision. Many researchers study Emotion Recognition method which uses one of EEG signals and Gestures in the existing research. In this paper, we use together EEG signals and Gestures for Emotion Recognition of human. And we select the driver emotion as a specific target. The experimental result shows that using of both EEG signals and gestures gets high recognition rates better than using EEG signals or gestures. Both EEG signals and gestures use Interactive Feature Selection(IFS) for the feature selection whose method is based on a reinforcement learning.

• 한국정보과학회논문지:시스템및이론
• /
• 제33권4호
• /
• pp.231-241
• /
• 2006

제어 가능하고 상황에 따라 반응하는 아바타의 제작은 컴퓨터 게임 및 가상현실 분야에서 중요한 연구 주제이다. 최근에는 아바타 애니메이션과 제어의 사실성을 높이기 위해 대규모 동작 캡처 데이타가 활용되고 있다. 방대한 양의 동작 데이타는 넓은 범위의 자연스러운 인간 동작을 수용할 수 있다는 장점을 갖는다. 하지만 동작 데이타가 많아지면 적절한 동작을 찾는데 필요한 계산량이 크게 증가하여 대화형 아바타 제어에 있어 병목으로 작용한다. 이 논문에서 우리는 레이블링(labeling)이 되어있지 않은 모션 데이타로부터 아바타의 행동을 학습시키는 새로운 방법을 제안한다. 이 방법을 사용하면 최소의 실시간 비용으로 아바타를 애니메이션하고 제어하는 것이 가능하다. 본 논문에서 제시하는 알고리즘은 Q-러닝이라는 기계 학습 기법에 기초하여 아바타가 동적인 환경과의 상호작용에 따른 시행착오를 통해 주어진 상황에 어떻게 반응할지 학습하도록 한다. 이 접근 방식의 유효성은 아바타가 서로 간에, 그리고 사용자에 대해 상호작용하는 예를 보임으로써 증명한다.