Journal of Internet of Things and Convergence (사물인터넷융복합논문지)

The Korea Internet of Things Society (한국사물인터넷학회)
권호 표지
DOI QR코드

DOI QR Code

Retained Message Delivery Scheme utilizing Reinforcement Learning in MQTT-based IoT Networks

MQTT 기반 IoT 네트워크에서 강화학습을 활용한 Retained 메시지 전송 방법

  • Yeunwoong Kyung (Division of Information & Communication Engineering, Kongju National University ) ;
  • Tae-Kook Kim (School of Computer and Artificial Intelligence Engineering, Pukyong National University ) ;
  • Youngjun Kim (School of Computer Science and Engineering, Kyungnam University)
  • 경연웅 (국립공주대학교 정보통신공학과 ) ;
  • 김태국 (국립부경대학교 컴퓨터공학부) ;
  • 김영준 (경남대학교 컴퓨터공학부 )
  • Received : 2024.02.24
  • Accepted : 2024.03.22
  • Published : 2024.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In the MQTT protocol, if the retained flag of a message published by a publisher is set, the message is stored in the broker as a retained message. When a new subscriber performs a subscribe, the broker immediately sends the retained message. This allows the new subscriber to perform updates on the current state via the retained message without waiting for new messages from the publisher. However, sending retained messages can become a traffic overhead if new messages are frequently published by the publisher. This situation could be considered an overhead when new subscribers frequently subscribe. Therefore, in this paper, we propose a retained message delivery scheme by considering the characteristics of the published messages. We model the delivery and waiting actions to new subscribers from the perspective of the broker using reinforcement learning, and determine the optimal policy through Q learning algorithm. Through performance analysis, we confirm that the proposed method shows improved performance compared to existing methods.

MQTT 프로토콜에서 Publisher로부터 발행되는 메시지의 retained flag가 세팅되어 있으면 해당 메시지는 Broker에 Retained 메시지로 저장되고, 새로운 Subscriber가 subscribe를 수행할 때 Broker는 Retained 메시지를 바로 전송하게 된다. 이를 통해 새로운 Subscriber는 Publisher의 새로운 메시지 발행을 기다리지 않고 Retained 메시지를 통해 현재 상태에 대한 업데이트를 수행할 수 있다. 하지만 Publisher로부터 새로운 메시지가 자주 발행되는 경우에는 retained 메시지를 보내는 것이 트래픽의 오버헤드가 될 수 있고, 해당 상황은 새로운 Subscriber들의 subscribe가 자주 수행되는 경우 더욱 큰 오버헤드로 고려될 수 있다. 그러므로 본 연구에서는 이러한 문제를 해결하기 위해 발행되는 메시지의 특성을 고려하여 Broker의 Retained 메시지 전송 방법을 제안하고자 한다. 본 연구에서는 Broker 입장에서 새로운 Subscriber로의 전송 및 대기 액션을 고려하여 강화학습을 기반으로 모델링하였고, Q learning 알고리즘을 통해 최적의 전송 방법을 결정하였다. 성능 분석을 통해 제안하는 방법이 기존 방법 대비 개선된 성능을 보이는 것을 확인하였다.

Keywords

Acknowledgement

This research was supported by "Regional innovation Strategy (RIS)" through the National Research Foundation of Korea(NRF) funded by the Ministry of Education(MOE)(2021RIS-003)

References

  1. D.W.Lee, K.Cho and S.H.Lee, "Analysis on Smart Factory in IoT Environment," Journal of The Korea Internet of Things Society, Vol.5, No.2, pp.1-5, 2019.  https://doi.org/10.20465/KIOTS.2019.5.2.001
  2. C.Choi., "A Study On IoT Data Consistency in IoT Environment," Journal of The Korea Internet of Things Society, Vol.8, No.5, pp.127-132, 2022. 
  3. F.Buccafurri et al., "MQTT-I: Achieving end-to-end data flow integrity in MQTT," IEEE Trans. on Dependable and Secure Computing, pp.1-18, 2024. 
  4. H.Shi et al., "Construction of industrial internet of things based on MQTT and OPC UA protocols," in Proc. ICAICA, pp.1263-1267, 2020. 
  5. Y.Kyung et al., "Priority-aware actuation update scheme in heterogeneous industrial networks," Sensors, Vol.24, 2024. 
  6. K.Govindan et al., "End-to-end service assurance in IoT MQTT-SN," in Proc. IEEE CCNC, 2015. 
  7. "Information technology-message queuing telemetry transport (mqtt)," International Organization for Standardization, iso.org, Standard, 2016. 
  8. X.Wu et al., "Improvements of MQTT retain message storage mechanism," in Proc. IMCEC, 2018. 
  9. Y.Kyung et al., "Opportunistic offloading scheme for content delivery service using electro-mobility networks," IET Intelligent Transport System, 2022. 
  10. X.Wang, C.Chen, J.He, S.Zhu and X.Guan, "Aoi-aware control and communication co-design for industrial iot systems," IEEE Internet of Things Journal, vol. 8, no. 10, pp. 8464-8473, 2021.  https://doi.org/10.1109/JIOT.2020.3046742
  11. Y.Sun, E.Uysal-Biyikoglu, R.D.Yates, C.E.Koksal and N.B.Shroff, "Update or wait: How to keep your data fresh," IEEE Transactions on Information Theory, vol. 63, no. 11, pp. 7492-7508, 2017.  https://doi.org/10.1109/TIT.2017.2735804
  12. F.Palmese, A.E.C.Redondi and M.Cesana, "Adaptive quality of service control for mqtt-sn," Sensors, vol. 22, no. 22, 2022. 
  13. L.Hu, Z.Chen, Y.Dong, Y.Jia, L.Liang and M.Wang, "Status update in iot networks: Age-of-information violation probability and optimal update rate," IEEE Internet of Things Journal, vol. 8, no. 14, pp. 11329-11344, 2021.  https://doi.org/10.1109/JIOT.2021.3051722
  14. Y.Chiang, H.Lin and Y.Ji, "Information cofreshness-aware grant assignment and transmission scheduling for internet of things," IEEE Internet of Things Journal, vol. 8, no. 19, pp. 14435-14446, 2021.  https://doi.org/10.1109/JIOT.2021.3052007
  15. R.O.Aburukba, M.AliKarrar, T.Landolsi, and K.El-Fakih, "Scheduling internet of things requests to minimize latency in hybrid fog-cloud computing," Future Generation Computer Systems, vol. 111, pp. 539-551, 2020 https://doi.org/10.1016/j.future.2019.09.039