• Title/Summary/Keyword: multi-agent learning

Search Result 121, Processing Time 0.032 seconds

Energy-efficient intrusion detection system for secure acoustic communication in under water sensor networks

  • N. Nithiyanandam;C. Mahesh;S.P. Raja;S. Jeyapriyanga;T. Selva Banu Priya
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.6
    • /
    • pp.1706-1727
    • /
    • 2023
  • Under Water Sensor Networks (UWSN) has gained attraction among various communities for its potential applications like acoustic monitoring, 3D mapping, tsunami detection, oil spill monitoring, and target tracking. Unlike terrestrial sensor networks, it performs an acoustic mode of communication to carry out collaborative tasks. Typically, surface sink nodes are deployed for aggregating acoustic phenomena collected from the underwater sensors through the multi-hop path. In this context, UWSN is constrained by factors such as lower bandwidth, high propagation delay, and limited battery power. Also, the vulnerabilities to compromise the aquatic environment are in growing numbers. The paper proposes an Energy-Efficient standalone Intrusion Detection System (EEIDS) to entail the acoustic environment against malicious attacks and improve the network lifetime. In EEIDS, attributes such as node ID, residual energy, and depth value are verified for forwarding the data packets in a secured path and stabilizing the nodes' energy levels. Initially, for each node, three agents are modeled to perform the assigned responsibilities. For instance, ID agent verifies the node's authentication of the node, EN agent checks for the residual energy of the node, and D agent substantiates the depth value of each node. Next, the classification of normal and malevolent nodes is performed by determining the score for each node. Furthermore, the proposed system utilizes the sheep-flock heredity algorithm to validate the input attributes using the optimized probability values stored in the training dataset. This assists in finding out the best-fit motes in the UWSN. Significantly, the proposed system detects and isolates the malicious nodes with tampered credentials and nodes with lower residual energy in minimal time. The parameters such as the time taken for malicious node detection, network lifetime, energy consumption, and delivery ratio are investigated using simulation tools. Comparison results show that the proposed EEIDS outperforms the existing acoustic security systems.

Simple Pyramid RAM-Based Neural Network Architecture for Localization of Swarm Robots

  • Nurmaini, Siti;Zarkasi, Ahmad
    • Journal of Information Processing Systems
    • /
    • v.11 no.3
    • /
    • pp.370-388
    • /
    • 2015
  • The localization of multi-agents, such as people, animals, or robots, is a requirement to accomplish several tasks. Especially in the case of multi-robotic applications, localization is the process for determining the positions of robots and targets in an unknown environment. Many sensors like GPS, lasers, and cameras are utilized in the localization process. However, these sensors produce a large amount of computational resources to process complex algorithms, because the process requires environmental mapping. Currently, combination multi-robots or swarm robots and sensor networks, as mobile sensor nodes have been widely available in indoor and outdoor environments. They allow for a type of efficient global localization that demands a relatively low amount of computational resources and for the independence of specific environmental features. However, the inherent instability in the wireless signal does not allow for it to be directly used for very accurate position estimations and making difficulty associated with conducting the localization processes of swarm robotics system. Furthermore, these swarm systems are usually highly decentralized, which makes it hard to synthesize and access global maps, it can be decrease its flexibility. In this paper, a simple pyramid RAM-based Neural Network architecture is proposed to improve the localization process of mobile sensor nodes in indoor environments. Our approach uses the capabilities of learning and generalization to reduce the effect of incorrect information and increases the accuracy of the agent's position. The results show that by using simple pyramid RAM-base Neural Network approach, produces low computational resources, a fast response for processing every changing in environmental situation and mobile sensor nodes have the ability to finish several tasks especially in localization processes in real time.

Proximal Policy Optimization Reinforcement Learning based Optimal Path Planning Study of Surion Agent against Enemy Air Defense Threats (근접 정책 최적화 기반의 적 대공 방어 위협하 수리온 에이전트의 최적 기동경로 도출 연구)

  • Jae-Hwan Kim;Jong-Hwan Kim
    • Journal of the Korea Society for Simulation
    • /
    • v.33 no.2
    • /
    • pp.37-44
    • /
    • 2024
  • The Korean Helicopter Development Program has successfully introduced the Surion helicopter, a versatile multi-domain operational aircraft that replaces the aging UH-1 and 500MD helicopters. Specifically designed for maneuverability, the Surion plays a crucial role in low-altitude tactical maneuvers for personnel transportation and specific missions, emphasizing the helicopter's survivability. Despite the significance of its low-altitude tactical maneuver capability, there is a notable gap in research focusing on multi-mission tactical maneuvers that consider the risk factors associated with deploying the Surion in the presence of enemy air defenses. This study addresses this gap by exploring a method to enhance the Surion's low-altitude maneuvering paths, incorporating information about enemy air defenses. Leveraging the Proximal Policy Optimization (PPO) algorithm, a reinforcement learning-based approach, the research aims to optimize the helicopter's path planning. Visualized experiments were conducted using a Surion model implemented in the Unity environment and ML-Agents library. The proposed method resulted in a rapid and stable policy convergence for generating optimal maneuvering paths for the Surion. The experiments, based on two key criteria, "operation time" and "minimum damage," revealed distinct optimal paths. This divergence suggests the potential for effective tactical maneuvers in low-altitude situations, considering the risk factors associated with enemy air defenses. Importantly, the Surion's capability for remote control in all directions enhances its adaptability in complex operational environments.

Cooperative Surveillance and Boundary Tracking with Multiple Quadrotor UAVs (복수 쿼드로터 무인기를 이용한 협업 감시 및 경계선 추종)

  • Lee, Hyeon Beom;Moon, Sung Won;Kim, Woo Jin;Kim, Hyoun Jin
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.19 no.5
    • /
    • pp.423-428
    • /
    • 2013
  • This paper investigates a boundary tracking problem using multiple quadrotor UAVs to detect and track the boundary of physical events. We set the boundary estimation problem as a classification problem of the region in which the physical events occur, and employ SVL (Support Vector Learning). We also demonstrate a velocity vector field which is globally attractive to a desired closed path with circulation at the desired speed and a virtual phase for stabilizing the collective configuration of the multiple quadrotors. Experimental results with multiple quadrotors show that this study provides good performance of the collective boundary tracking.

Developing artificial football agents based upon multi-agent techniques in the AI world cup (AI World Cup 환경을 이용한 멀티 에이전트 기반 지능형 가상 축구 에이전트 구현)

  • Lee, Eunhoo;Seong, Hyeon-ah;Jung, Minji;Lee, Hye-in;Joung, Jinoo;Lee, Eui Chul;Lee, Jee Hang
    • Annual Conference of KIPS
    • /
    • 2021.11a
    • /
    • pp.819-822
    • /
    • 2021
  • AI World Cup 환경은 다수 가상 에이전트들이 팀을 이뤄서 서로 상호작용하며 대전이 가능한 가상 축구 환경이다. 본 논문에서는 AI World Cup 환경에서 멀티 에이전트기반 학습/추론 기술을 사용하여 다양한 전략과 전술을 구사하는 가상 축구 에이전트 구현과 시뮬레이션 결과를 소개한다. 먼저, 역할을 바탕으로 협동하여 상대방과 대전할 수 있는 논리 기반 추론형 멀티 에이전트 기술이 적용된 Dynamic planning 축구 에이전트 9 세트를 구현하였다. 이후, 강화학습 에이전트 기반, 단일 에이전트를 조합한 Independent Q-Learning 방식의 학습형 축구 에이전트를 구현한 후, 이를 멀티 에이전트 강화학습으로 확장하여 역할 기반 전략 학습이 가능한 가상 축구 에이전트를 구현하고 시뮬레이션 하였다. 구현된 가상 축구 에이전트들 간 대전을 통해 승률을 확인하고, 전략의 우수성을 분석하였다. 시뮬레이션 예제는 다음에서 확인할 수 있다 (https://github.com/I-hate-Soccer/Simulation).

Development of Interactive Content Services through an Intelligent IoT Mirror System (지능형 IoT 미러 시스템을 활용한 인터랙티브 콘텐츠 서비스 구현)

  • Jung, Wonseok;Seo, Jeongwook
    • Journal of Advanced Navigation Technology
    • /
    • v.22 no.5
    • /
    • pp.472-477
    • /
    • 2018
  • In this paper, we develop interactive content services for preventing depression of users through an intelligent Internet of Things(IoT) mirror system. For interactive content services, an IoT mirror device measures attention and meditation data from an EEG headset device and also measures facial expression data such as "sad", "angery", "disgust", "neutral", " happy", and "surprise" classified by a multi-layer perceptron algorithm through an webcam. Then, it sends the measured data to an oneM2M-compliant IoT server. Based on the collected data in the IoT server, a machine learning model is built to classify three levels of depression (RED, YELLOW, and GREEN) given by a proposed merge labeling method. It was verified that the k-nearest neighbor (k-NN) model could achieve about 93% of accuracy by experimental results. In addition, according to the classified level, a social network service agent sent a corresponding alert message to the family, friends and social workers. Thus, we were able to provide an interactive content service between users and caregivers.

Improvements of pursuit performance using episodic parameter optimization in probabilistic games (에피소드 매개변수 최적화를 이용한 확률게임에서의 추적정책 성능 향상)

  • Kwak, Dong-Jun;Kim, H.-Jin
    • Journal of the Korean Society for Aeronautical & Space Sciences
    • /
    • v.40 no.3
    • /
    • pp.215-221
    • /
    • 2012
  • In this paper, we introduce an optimization method to improve pursuit performance of a pursuer in a pursuit-evasion game (PEG). Pursuers build a probability map and employ a hybrid pursuit policy which combines the merits of local-max and global-max pursuit policies to search and capture evaders as soon as possible in a 2-dimensional space. We propose an episodic parameter optimization (EPO) algorithm to learn good values for the weighting parameters of a hybrid pursuit policy. The EPO algorithm is performed while many episodes of the PEG are run repeatedly and the reward of each episode is accumulated using reinforcement learning, and the candidate weighting parameter is selected in a way that maximizes the total averaged reward by using the golden section search method. We found the best pursuit policy in various situations which are the different number of evaders and the different size of spaces and analyzed results.

Collision Avoidance Path Control of Multi-AGV Using Multi-Agent Reinforcement Learning (다중 에이전트 강화학습을 이용한 다중 AGV의 충돌 회피 경로 제어)

  • Choi, Ho-Bin;Kim, Ju-Bong;Han, Youn-Hee;Oh, Se-Won;Kim, Kwi-Hoon
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.11 no.9
    • /
    • pp.281-288
    • /
    • 2022
  • AGVs are often used in industrial applications to transport heavy materials around a large industrial building, such as factories or warehouses. In particular, in fulfillment centers their usefulness is maximized for automation. To increase productivity in warehouses such as fulfillment centers, sophisticated path planning of AGVs is required. We propose a scheme that can be applied to QMIX, a popular cooperative MARL algorithm. The performance was measured with three metrics in several fulfillment center layouts, and the results are presented through comparison with the performance of the existing QMIX. Additionally, we visualize the transport paths of trained AGVs for a visible analysis of the behavior patterns of the AGVs as heat maps.

Multi-task Deep Neural Network Model for T1CE Image Synthesis and Tumor Region Segmentation in Glioblastoma Patients (교모세포종 환자의 T1CE 영상 생성 및 암 영역분할을 위한 멀티 태스크 심층신경망 모델)

  • Kim, Eunjin;Park, Hyunjin
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.05a
    • /
    • pp.474-476
    • /
    • 2021
  • Glioblastoma is the most common brain malignancies arising from glial cells. Early diagnosis and treatment plan establishment are important, and cancer is diagnosed mainly through T1CE imaging through injection of a contrast agent. However, the risk of injection of gadolinium-based contrast agents is increasing recently. Region segmentation that marks cancer regions in medical images plays a key role in CAD systems, and deep neural network models for synthesizing new images are also being studied. In this study, we propose a model that simultaneously learns the generation of T1CE images and segmentation of cancer regions. The performance of the proposed model is evaluated using similarity measurements including mean square error and peak signal-to-noise ratio, and shows average result values of 21 and 39 dB.

  • PDF

A Study on Teaching-Learning and Evaluation Methods of Environmental Studies in the Middle School (중학교 "환경" 교과의 교수.학습 및 평가 방법 연구)

  • 남상준
    • Hwankyungkyoyuk
    • /
    • v.7 no.1
    • /
    • pp.1-17
    • /
    • 1994
  • This study was performed to determine appropriate teaching-learning and evaluation methods for Environmental Studies. To promote the relevance of our study to the needs of the schools and concerned educational communities of environmental education, we reviewed related literature, conducted questionnaire surveys, interviewed related teachers and administrator, held meetings with experts, and field-tested our findings. For selecting and developing teaching-learning methods of Environmental Studies, findings of educational research in general are considered. moreover, principles of environmental education, general aim of environmental education, orientations of environmental education, and developmental stages of middle school students in educational psychology were attended. In addition, relevance to the purpose of the Environmental Studies curriculum, appropriateness for value inquiry as well as knowledge inquiry, small group centered class organization, social interaction centered teaching-learning process, regional environmental situation, significance of personal environment, evaluation methods of Environmental Studies, multi- and inter-disciplinary contents of the Environmental Studies textbook, suitability to the evaluation methods of Environmental Studies, and emphasis on the social interaction in teaching-learning process were regarded. It was learned the Environmental Studies can be taught most effectively in via of holding discussion sessions, conducting actual investigation, doing experiment-practice, doing games and plate, role-playing and carrying out simulation activities, and doing inquiry. These teaching-learning methods were field-tested and proved appropriate methods for the subject. For selecting and developing evaluation method of Environmental Studies, such principles and characteristics of Environmental Studies as objective domains stated in the Environmental Studies curriculum, diversity of teaching-learning organization, were appreciated. We categorized nine evaluation methods: the teacher may conduct questionnaire surveys, testings, interviews, non-participatory observations; they may evaluate student's experiment-practice performances, reports preparation ability, ability to establish a research project, the teacher may ask the students to conduct a self-evaluation, or reciprocal evaluation. To maximize the effect of these methods, we further developed an application system. It considered three variables, that is, evaluates, evaluation objectives domains, and evaluation agent, and showed how to choose the most appropriate methods and, when necessary, how to combine uses of different methods depending on these variables. A sample evaluation instrument made on the basis of this application system was developed and tested in the classes. The system proved effective. Pilot applications of the teaching-learning methods and evaluation method were made simultaneously; and the results and their implications are as follows. Discussion program was applied in a lesson dealing with the problems of waste disposal, in which students showed active participation and creative thinking. The evaluation method used in this lesson was a multiple-choice written test for knowledge and skills. It was shown that this evaluation method and device are effective in helping students' revision of the lesson and in stimulating their creative interpretations and responces. Pupils showed great interests in the actual investigation program, and this programme was proved to be effective in enhancing students' participation. However, it was also turned out that there must be pre-arranged plans for the objects, contents and procedures of survey if this program is to effective. In this lesson, non-participatory observation methods were used with a focus on the attitudes of students. A scaled reported in general description rather than in grade. Experiment-practice programme was adopted in a lesson for purifying contaminated water and in this lesson, instruction objectives were properly established, the teaching-learning process was clearly specified and students were highly motivated. On the other hand, however, it was difficult to control the class when some groups of students require more times to complete their experiment, and sometimes different results. As regards to evaluation, performance observation test were used for assessing skills and attitudes. If teachers use well-prepared Likert scale, evaluation of all groups within a reasonablely short period of time will be possible. The most effective and successful programme in therms of students' participation and enjoyment, was the 'ah-nah-bah-dah-market' program, which is kind of game of the flea market. For better organized program of this kind, however, are essential, In this program, students appraise their own attitudes and behavior by responding to a written questionnaire. In addition, students were asked to record any anecdotes relating to self-appraisal of changes on one's own attitudes and behaviours. Even after the lesson, students keep recording those changes on letters to herself. Role-playing and simulation game programme was applied to a case of 'NIMBY', in which students should decide where to located a refuse dumping ground. For this kind of programme to e successful, concepts and words used in the script should be appropriate for students' intellectual levels, and students should by adequately introduced into the objective and the procedures of the lessons. Written questionnaire was used to assess individual students' attitudes after the lesson, but in order to acquire information on the changes of students' attitudes and skills, pre-test may have to be made. Doing inquiry programme, in which advantages in which students actually investigated the environmental influence of the areas where school os located, had advantages in developing students' ability to study the environmental problems and to present the results of their studies. For this programme to be more efficient, areas of investigation should be clearly divided and alloted to each group so that repetition or overlap in areas of study and presentation be avoided, and complementary wok between groups bee enhanced. In this programme, teacher assessed students' knowledge and attitudes on the basis of reports prepared by each group. However, there were found some difficults in assessing students' attitudes and behaviours solely on the grounds of written report. Perhaps, using a scaled checklist assessing students' attitudes while their presentation could help to relieve the difficulties.

  • PDF