Advances in Computational Design

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

CSI-based human activity recognition via lightweight compact convolutional transformers

  • Fahd Saad Abuhoureyah (Centre for Telecommunication Research and Innovation (CeTRI) Fakulti Teknologi dan Kejuruteraan Elektronik dan. Komputer (FTKEK), Universiti Teknikal Malaysia Melaka (UTeM)) ;
  • Yan Chiew Wong (Centre for Telecommunication Research and Innovation (CeTRI) Fakulti Teknologi dan Kejuruteraan Elektronik dan. Komputer (FTKEK), Universiti Teknikal Malaysia Melaka (UTeM)) ;
  • Malik Hasan Al-Taweel (Centre for Telecommunication Research and Innovation (CeTRI) Fakulti Teknologi dan Kejuruteraan Elektronik dan. Komputer (FTKEK), Universiti Teknikal Malaysia Melaka (UTeM)) ;
  • Nihad Ibrahim Abdullah (Computer Science, Sulaimani Polytechnic University)
  • Received : 2024.03.13
  • Accepted : 2024.09.11
  • Published : 2024.07.25
⟨ Previous Next ⟩

Abstract

WiFi sensing integration enables non-intrusive and is utilized in applications like Human Activity Recognition (HAR) to leverage Multiple Input Multiple Output (MIMO) systems and Channel State Information (CSI) data for accurate signal monitoring in different fields, such as smart environments. The complexity of extracting relevant features from CSI data poses computational bottlenecks, hindering real-time recognition and limiting deployment on resource-constrained devices. The existing methods sacrifice accuracy for computational efficiency or vice versa, compromising the reliability of activity recognition within pervasive environments. The lightweight Compact Convolutional Transformer (CCT) algorithm proposed in this work offers a solution by streamlining the process of leveraging CSI data for activity recognition in such complex data. By leveraging the strengths of both CNNs and transformer models, the CCT algorithm achieves state-of-the-art accuracy on various benchmarks, emphasizing its excellence over traditional algorithms. The model matches convolutional networks' computational efficiency with transformers' modeling capabilities. The evaluation process of the proposed model utilizes self-collected dataset for CSI WiFi signals with few daily activities. The results demonstrate the improvement achieved by using CCT in real-time activity recognition, as well as the ability to operate on devices and networks with limited computational resources.

Keywords

Acknowledgement

The authors acknowledge the technical and financial support by the Ministry of Higher Education, Malaysia, under the research grant no. FRGS/1/2024/ICT02/UTEM/02/3 and Universiti Teknikal Malaysia Melaka (UTeM)

References

  1. Abuhoureyah, F.S., Wong, Y.C., Sadhiqin, A. and Mohd, B. (2024), "WiFi-based human activity recognition through wall using deep learning", Eng. Appl. Artif. Intell., 127(PA), 107171. https://doi.org/10.1016/j.engappai.2023.107171.
  2. Abuhoureyah, F., Yan Chiew, W., Bin Mohd Isira, A.S. and Al-Andoli, M. (2023), "Free device location independent WiFi-based localisation using received signal strength indicator and channel state information", IET Wireless Sensor Syst., 13(5), 163-177. https://doi.org/10.1049/wss2.12065
  3. Ahmed, H.F.T., Ahmad, H., Narasingamurthi, K., Harkat, H. and Phang, S.K. (2020), "DF-WiSLR: Device-Free Wi-Fi-based sign language recognition", Pervasive Mobile Comput., 69, 101289. https://doi.org/10.1016/j.pmcj.2020.101289.
  4. Abuhoureyah, F., Sim, K.S. and Wong, Y.C. (2024), "Multi-user human activity recognition through adaptive location-independent WiFi signal characteristics", IEEE Access, 12, 112008-112024. https://doi.org/10.1109/ACCESS.2024.3438871.
  5. Ahmed Ouameur, M., Caza-Szoka, M. and Massicotte, D. (2020), "Machine learning enabled tools and methods for indoor localization using low power wireless network", Internet Thing., 12, 100300. https://doi.org/10.1016/j.iot.2020.100300.
  6. Akhtar, Z.U.A. and Wang, H. (2020), "Wifi-based driver's activity monitoring with efficient computation of radio-image features", Sensors, 20(5). https://doi.org/10.3390/s20051381
  7. Darabkh, K.A., Al-Akhras, M., Zomot, J.N. and Atiquzzaman, M. (2022), "RPL routing protocol over IoT: A comprehensive survey, recent advances, insights, bibliometric analysis, recommendations, and future directions", J. Netw. Comput. Appl., 207(2021), 103476. https://doi.org/10.1016/j.jnca.2022.103476.
  8. Dierickx, P., Van Damme, A., Dupuis, N. and Delaby, O. (2023), "Comparison between CNN, ViT and CCT for channel frequency response interpretation and application to G.Fast", IEEE Access, 11, 24039-24052. https://doi.org/10.1109/ACCESS.2023.3247877.
  9. Ding, J. and Wang, Y. (2019), "WiFi CSI-based human activity recognition using deep recurrent neural network", IEEE Access, 7, 174257-174269, https://doi.org/10.1109/ACCESS.2019.2956952
  10. Ding, X., Jiang, T., Zhong, Y., Huang, Y. and Li, Z. (2021), "Wi-Fi-based location-independent human activity recognition via meta learning", Sensors, 21(8). https://doi.org/10.3390/s21082654.
  11. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J. and Houlsby, N. (2021), "An image is worth 16X16 words: Transformers for image recognition at scale", Proceedings of the ICLR 2021 9th International Conference on Learning Representations.
  12. Fard Moshiri, P., Shahbazian, R., Nabati, M. and Ghorashi, S.A. (2021), "A CSI-based human activity recognition using deep learning", Sensors, 21(21), 1-19. https://doi.org/10.3390/s21217225.
  13. Abuhoureyah, F., Yan Chiew, W. and Zitouni, M.S. (2024), "WIFI based human activity recognition using multi-head adaptive attention mechanism", J. Intell. Fuzzy Syst., Preprint, 1-16.
  14. Guo, L., Wang, L., Liu, J., Zhou, W. and Lu, B. (2018), "HuAc: Human activity recognition using crowdsourced WiFi signals and skeleton data", Wireless Commun. Mobile Comput., 2018, 1-15. https://doi.org/10.1155/2018/6163475.
  15. Jiang, D., Li, M. and Xu, C. (2020), "Wigan: A wifi based gesture recognition system with gans", Sensors, 20(17), 1-19. https://doi.org/10.3390/s20174757.
  16. Lee, D.J., Lee, J.Y., Shon, H., Yi, E., Park, Y.H., Cho, S.S. and Kim, J. (2023), "Lightweight monocular depth estimation via token-sharing transformer", Proceedings of the IEEE International Conference on Robotics and Automation, 2023-May(Icra), 4895-4901. https://doi.org/10.1109/ICRA48891.2023.10160566.
  17. Li, T., Shi, C., Li, P. and Chen, P. (2021), "A novel gesture recognition system based on CSI extracted from a smartphone with nexmon firmware", Sensor, 21(1), 1-19. https://doi.org/10.3390/s21010222.
  18. Li, Y., Xie, X., Fu, H., Luo, X. and Guo, Y. (2023), "A compact transformer for adaptive style transfer", Proceedings of the IEEE International Conference on Multimedia and Expo, 2023-July, 2687-2692. https://doi.org/10.1109/ICME55011.2023.00457.
  19. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S. and Guo, B. (2021), "Swin transformer: Hierarchical vision transformer using shifted windows", Proceedings of the IEEE/CVF International Conference on Computer Vision, 10012. https://doi.org/10.48550/arXiv.2103.14030.
  20. Lu, Z., Xie, H., Liu, C. and Zhang, Y. (2022), "Bridging the gap between vision transformers and convolutional neural networks on small datasets", Adv. Neural Inform. Proc. Syst., 35(NeurIPS), 1-15. https://doi.org/10.48550/arXiv.2210.05958.
  21. Luo, Z., Cheng, X. and Yang, Y. (2022), "Computational electromechanical approach for stability/instability of smart system actuated with piezoelectric NEMS", Adv. Comput. Des., 7(3), 21. https://doi.org/10.12989/acd.2022.7.3.211
  22. Ma, Y., Zhou, G, and Wang, S. (2019), "WiFi sensing with channel state information: A survey", ACM Comput. Surveys, 46(1), 1-32. https://doi.org/10.1186/1687-6180-2011-10
  23. Muaaz, M., Chelli, A., Gerdes, M.W. and Patzold, M. (2022), "Wi-Sense: A passive human activity recognition system using Wi-Fi and convolutional neural network and its integration in health information systems", Annal. Telecommun., 77(3-4), 163-175. https://doi.org/10.1007/s12243-021-00865-9
  24. Palanivelu, R. and Srinivasan, P.S.S. (2020), "Safety and security measurement in industrial environment based on smart IOT technology based augmented data recognizing scheme", Comput. Commun., 150, 777-787. https://doi.org/10.1016/j.comcom.2019.12.013.
  25. Pan, X., Jiang, T., Li, X., Ding, X., Wang, Y. and Li, Y. (2019), "Dynamic hand gesture detection and recognition with WiFi Signal Based on 1D-CNN", Proceedings of the 2019 IEEE International Conference on Communications Workshops, ICC Workshops 2019, 0-5. https://doi.org/10.1109/ICCW.2019.8756690.
  26. Saw, C.Y. and Wong, Y.C. (2023), "Neuromorphic computing with hybrid CNN-Stochastic Reservoir for time series WiFi based human activity recognition", Comput. Electr. Eng., 111(PA), 108917. https://doi.org/10.1016/j.compeleceng.2023.108917
  27. Schafer, J., Barrsiwal, B.R., Kokhkharova, M., Adil, H. and Liebehenschel, J. (2021), "Human activity recognition using csi information with nexmon", Appl. Sci., 11(19). https://doi.org/10.3390/app11198860.
  28. Sharma, L., Chao, C., Wu, S.L. and Li, M.C. (2021), "High accuracy wifi-based human activity classification system with time-frequency diagram cnn method for different places", Sensors, 21(11). https://doi.org/10.3390/s21113797.
  29. Showmik, I.A., Sanam, T.F. and Imtiaz, H. (2023), "Human activity recognition from Wi-Fi CSI data using principal component-based wavelet CNN", Digital Signal Proc. Rev. J., 138, 104056. https://doi.org/10.1016/j.dsp.2023.104056.
  30. Tiku, S., Pasricha, S., Notaros, B. and Han, Q. (2020), "A Hidden Markov Model based smartphone heterogeneity resilient portable indoor localization framework", J. Syst. Arch., 108(8), 101806. https://doi.org/10.1016/j.sysarc.2020.101806
  31. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N. and Polosukhin, I. (2017), "Attention is all you need", Adv. Neural Inform. Proc. Syst., 30.
  32. Wang, D., Yang, J., Cui, W., Xie, L. and Sun, S. (2022), "AirFi: Empowering WiFi-based passive human gesture recognition to unseen environment via domain generalization", IEEE T. Mobile Comput., 23(2), 1156-1168. https://doi.org/10.1109/TMC.2022.3230665
  33. Yang, J., Chen, X., Wang, D., Zou, H., Lu, C.X., Sun, S. and Xie, L. (2022a), "Deep learning and its applications to WiFi human sensing: A benchmark and a tutorial", arXiv preprint, arXiv:2207.07859.
  34. Yang, J., Chen, X., Wang, D., Zou, H., Lu, C.X., Sun, S. and Xie, L. (2022b), "SenseFi: A library and benchmark on deep-learning- empowered WiFi human sensing", Patterns, 4(3), 100703. https://doi.org/10.1016/j.patter.2023.100703
  35. Yang, J., Zou, H. and Xie, L. (2022), "SecureSense: Defending adversarial attack for secure device-free human activity recognition", IEEE T. Mobile Comput., 1(11). https://doi.org/10.1109/TMC.2022.3226742
  36. Yang, Z., Zhou, Z. and Liu, Y. (2013), "From RSSI to CSI: Indoor localization via channel response", ACM Comput. Surveys, 46(2), 1-32. https://doi.org/10.1145/2543581.2543592
  37. Zhang, Y., Yin, Y., Wang, Y., Ai, J. and Wu, D. (2023), "CSI-based location-independent Human Activity Recognition with parallel convolutional networks", Comput. Commun., 197(2022), 87-95. https://doi.org/10.1016/j.comcom.2022.10.027
  38. Zhang, Y., Zheng, Y., Qian, K., Zhang, G., Liu, Y., Wu, C. and Yang, Z. (2021), "Widar3.0: Zero-effort cross-domain gesture recognition with Wi-Fi", IEEE T. Pattern Anal. Machine Intell., 8828(c). https://doi.org/10.1109/TPAMI.2021.3105387
  39. Zhou, C., Yang, L., Liao, H., Liang, B. and Ye, X. (2021), "Ankle foot motion recognition based on wireless wearable sEMG and acceleration sensors for smart AFO", Sensors Actuat. A Phys., 331, 113025. https://doi.org/10.1016/j.sna.2021.113025
  40. Zhou, R., Hou, H., Gong, Z., Chen, Z., Tang, K. and Zhou, B. (2021), "Adaptive device-free localization in dynamic neural networks", IEEE Sensors J., 21(1), 548-559. https://doi.org/10.1109/JSEN.2020.3014641