• ETRI Journal
• /
• v.43 no.5
• /
• pp.787-798
• /
• 2021

This paper applies learning automata to improve the performance of a Q-learning based random access channel (QL-RACH) scheme in a cellular machine-to-machine (M2M) communication system. A prioritized learning automata QL-RACH (PLA-QL-RACH) access scheme is proposed. The scheme employs a prioritized learning automata technique to improve the throughput performance by minimizing the level of interaction and collision of M2M devices with human-to-human devices sharing the RACH of a cellular system. In addition, this scheme eliminates the excessive punishment suffered by the M2M devices by controlling the administration of a penalty. Simulation results show that the proposed PLA-QL-RACH scheme improves the RACH throughput by approximately 82% and reduces access delay by 79% with faster learning convergence when compared with QL-RACH.

• Journal of Electrical Engineering and Technology
• /
• v.12 no.2
• /
• pp.890-903
• /
• 2017

The problem of determining a smooth and collision-free path with maximum possible speed for a Mobile Robot (MR) which is chasing a moving target in a dynamic environment is addressed in this paper. Genetic Network Programming with Reinforcement Learning (GNP-RL) has several important features over other evolutionary algorithms such as it combines offline and online learning on the one hand, and it combines diversified and intensified search on the other hand, but it was used in solving the problem of MR navigation in static environment only. This paper presents GNP-RL based on predicting collision positions as a first attempt to apply it for MR navigation in dynamic environment. The combination between features of the proposed collision prediction and that of GNP-RL provides safe navigation (effective obstacle avoidance) in dynamic environment, smooth movement, and reducing the obstacle avoidance latency time. Simulation in dynamic environment is used to evaluate the performance of collision prediction based GNP-RL compared with that of two state-of-the art navigation approaches, namely, Q-Learning (QL) and Artificial Potential Field (APF). The simulation results show that the proposed GNP-RL outperforms both QL and APF in terms of smooth movement and safer navigation. In addition, it outperforms APF in terms of preserving maximum possible speed during obstacle avoidance.

• Journal of Internet of Things and Convergence
• /
• v.10 no.4
• /
• pp.1-8
• /
• 2024

As technology advances, Internet of Things (IoT) technology is rapidly evolving as well. Various protocols, including Message Queuing Telemetry Transport (MQTT), are being used in IoT technology. MQTT, a lightweight messaging protocol, is considered a de-facto standard in the IoT field due to its efficiency in transmitting data even in environments with limited bandwidth and power. In this paper, we propose a method to improve the message transmission method in MQTT 5.0, specifically focusing on the shared subscription feature. The widely used round-robin method in shared subscriptions has the drawback of not considering the current state of the clients. To address this limitation, we propose a method to select the optimal transmission method by considering the current state. We model this problem based on Markov decision process (MDP) and utilize Q-Learning to select the optimal transmission method. Through simulation results, we compare our proposed method with existing methods in various environments and conduct performance analysis. We confirm that our proposed method outperforms existing methods in terms of performance and conclude by suggesting future research directions.