KIPS Transactions on Computer and Communication Systems (정보처리학회논문지:컴퓨터 및 통신 시스템)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Hybrid Offloading Technique Based on Auction Theory and Reinforcement Learning in MEC Industrial IoT Environment

MEC 산업용 IoT 환경에서 경매 이론과 강화 학습 기반의 하이브리드 오프로딩 기법

  • Received : 2023.03.02
  • Accepted : 2023.05.06
  • Published : 2023.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

Industrial Internet of Things (IIoT) is an important factor in increasing production efficiency in industrial sectors, along with data collection, exchange and analysis through large-scale connectivity. However, as traffic increases explosively due to the recent spread of IIoT, an allocation method that can efficiently process traffic is required. In this thesis, I propose a two-stage task offloading decision method to increase successful task throughput in an IIoT environment. In addition, I consider a hybrid offloading system that can offload compute-intensive tasks to a mobile edge computing server via a cellular link or to a nearby IIoT device via a Device to Device (D2D) link. The first stage is to design an incentive mechanism to prevent devices participating in task offloading from acting selfishly and giving difficulties in improving task throughput. Among the mechanism design, McAfee's mechanism is used to control the selfish behavior of the devices that process the task and to increase the overall system throughput. After that, in stage 2, I propose a multi-armed bandit (MAB)-based task offloading decision method in a non-stationary environment by considering the irregular movement of the IIoT device. Experimental results show that the proposed method can obtain better performance in terms of overall system throughput, communication failure rate and regret compared to other existing methods.

산업용 IoT는 대규모 연결을 통해 데이터 수집, 교환, 분석과 함께 산업 분야의 생산 효율성 개선에 중요한 요소이다. 그러나 최근 산업용 IoT의 확산으로 인해 트래픽이 폭발적으로 증가함에 따라 트래픽을 효율적으로 처리해줄 할당 기법이 필요하다. 본 논문에서는 산업용 IoT 환경에서 성공적인 태스크 처리율을 높이기 위한 2단계 태스크 오프로딩 결정 기법을 제안한다. 또한, 컴퓨팅 집약적인 태스크를 셀룰러 링크를 통해 이동 엣지 컴퓨팅(Mobile Edge Computing: MEC) 서버로 오프로드 하거나 D2D(Device to Device) 링크를 통해 근처의 산업용 IoT 장치로 오프로드 할 수 있는 하이브리드 오프로딩(Hybrid-offloading) 시스템을 고려한다. 먼저 1단계는 태스크 오프로딩에 참여하는 기기들이 이기적으로 행동하여 태스크 처리율 향상에 어려움을 주는 것을 방지하기 위해 인센티브 메커니즘을 설계한다. 메커니즘 디자인 중 McAfee's 메커니즘을 사용하여 태스크를 처리해주는 기기들의 이기적인 행동을 제어하고 전체 시스템 처리율을 높일 수 있도록 한다. 그 후 2단계에서는 산업용 IoT 장치의 불규칙한 움직임을 고려하여 비정상성(Non-stationary) 환경에서 멀티 암드 밴딧(Multi-Armed Bandit: MAB) 기반 태스크 오프로딩 결정 기법을 제안한다. 실험 결과로 제안된 기법이 기존의 다른 기법에 비해 전체 시스템 처리율, 통신 실패율, 후회 측면에서 더 나은 성능을 달성할 수 있음을 보인다.

Keywords

Acknowledgement

본 연구는 과학기술정보통신부 및 정보통신기획평가원의 대학ICT연구센터 지원사업의 연구결과로 수행되었음(IITP-2023-2018-0-01799).

References

  1. N. Zhu, X. Xu, S. Han, and S. Lv, "Sleep-scheduling and joint computation-communication resource allocation in MEC networks for 5G IIoT," 2021 IEEE Wireless Communications and Networking Conference (WCNC), pp.1-7, 2021.
  2. X. Hou, Z. Ren, K. Yang, C. Chen, H. Zhang, and Y. Xiao, "IIoT-MEC: A novel mobile edge computing framework for 5G-enabled IIoT," 2019 IEEE Wireless Communications and Networking Conference (WCNC), pp.1-7, 2019.
  3. S. Koo and Y. Lim, "Optimal task offloading decision in IIoT enviornments using reinforcement learning," 2021 IEEE 3rd Eurasia Conference on IOT, Communication and Engineering (ECICE), pp.86-89, 2021.
  4. Z. Zhou, H. Liao, B. Gu, S. Mumtaz, and J. Rodriguez, "Resource sharing and task offloading in IoT fog computing: A contract-learning approach," in IEEE Transactions on Emerging Topics in Computational Intelligence, Vol.4, No.3, pp.227-240, 2020. https://doi.org/10.1109/TETCI.2019.2902869
  5. N. Fan, X. Wang, D. Wang, Y. Lan, and J. Hou, "A collaborative task offloading scheme in D2D-assisted fog computing networks," 2020 IEEE Wireless Communications and Networking Conference (WCNC), pp.1-6, 2020.
  6. J. X. Dai et al., "Task Co-offloading for D2D-assisted mobile edge computing in industrial internet of Things," in IEEE Transactions on Industrial Informatics, 2022.
  7. X. Wu, T. Li, S. Yuan, Y. Lu, and Y. Wang, "A hybrid mode task offloading algorithm for edge computing," 2021 IEEE 5th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp.81-85, 2021.
  8. L. Sun, G. Xue, and R. Yu, "TAFS: A truthful auction for IoT application offloading in fog computing networks," in IEEE Internet of Things Journal, Vol.10, No.4, pp.3252- 3263, 2022.
  9. W. Sun, J. Liu, Y. Yue, and H. Zhang, "Double auction-based resource allocation for mobile edge computing in industrial internet of things," in IEEE Transactions on Industrial Informatics, Vol.14, No.10, pp.4692-4701, 2018. https://doi.org/10.1109/TII.2018.2855746
  10. W. Lu, S. Zhang, J. Xu, D. Yang, and L. Xu, "Truthful multi-resource transaction mechanism for P2P task offloading based on edge computing," in IEEE Transactions on Vehicular Technology, Vol.70, No.6, pp.6122-6135, 2021. https://doi.org/10.1109/TVT.2021.3079258
  11. R. P. McAfee, "A dominant strategy double auction," Journal of Economic Theory, Vol.56, No.2, pp.434-450, 1992. https://doi.org/10.1016/0022-0531(92)90091-U
  12. V. Raj and S. Kalyani, "Taming non-stationary bandits: A Bayesian approach," arXiv preprint arXiv:1707.09727, 2017.
  13. M. Mehrabi, D. You, V. Latzko, H. Salah, M. Reisslein, and F. H. P. Fitzek, "Device-enhanced MEC: Multi-access edge computing (MEC) aided by end device computation and caching: A survey," in IEEE Access, Vol.7, pp.166079- 166108, 2019. https://doi.org/10.1109/ACCESS.2019.2953172
  14. N. Nasaruddin, E. D. Meutia, and R. Adriman, "Outage probability and power efficiency of quantize-and-forward relay in multi-hop D2D networks," 2020 2nd International Conference on Broadband Communications, Wireless Sensors and Powering (BCWSP), pp.183-188, 2020.
  15. D. J., Russo et al., "A tutorial on thompson sampling," Foundations and Trends® in Machine Learning, Vol.11, No.1, pp.1-96, 2018. https://doi.org/10.1561/2200000070
  16. M. A. Qureshi and C. Tekin, "Fast learning for dynamic resource allocation in AI-enabled radio networks," in IEEE Transactions on Cognitive Communications and Networking, Vol.6, No.1, pp.95-110, 2020.  https://doi.org/10.1109/TCCN.2019.2953607