International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Privacy-Preserving Deep Learning using Collaborative Learning of Neural Network Model

  • Hye-Kyeong Ko (Department of Computer Engineering, Sungkyul University)
  • Received : 2023.03.22
  • Accepted : 2023.03.27
  • Published : 2023.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

The goal of deep learning is to extract complex features from multidimensional data use the features to create models that connect input and output. Deep learning is a process of learning nonlinear features and functions from complex data, and the user data that is employed to train deep learning models has become the focus of privacy concerns. Companies that collect user's sensitive personal information, such as users' images and voices, own this data for indefinite period of times. Users cannot delete their personal information, and they cannot limit the purposes for which the data is used. The study has designed a deep learning method that employs privacy protection technology that uses distributed collaborative learning so that multiple participants can use neural network models collaboratively without sharing the input datasets. To prevent direct leaks of personal information, participants are not shown the training datasets during the model training process, unlike traditional deep learning so that the personal information in the data can be protected. The study used a method that can selectively share subsets via an optimization algorithm that is based on modified distributed stochastic gradient descent, and the result showed that it was possible to learn with improved learning accuracy while protecting personal information.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT).(NO.NRF-2021R1A2C1012827) in (2023).

References

  1. A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, A. Coates, et al, "Deep speech: Scaling up End-to-End Speech Recognition," arXiv:1412.5557, 2014. DOI: https://doi.org/10.48550/asXiv.1412.5567
  2. Seokwon, Lee, "Development of a Method for ACF Bonding Based on Machine Vision," The Journal of the Convergence of Culture Technology (JCCT), Vol. 4, No. 3, pp. 209-212, 2018. DOI: https://doi.org/10.17703/JCCT.2018.4.3.209
  3. H. Lee and J. Choi, "Implementation of Smart Ventilation Control System using IoT and Machine Learning," The Journal of the Institute of Internet, Broadcasting and Communication (JIIBC), Vol. 20, No. 2, 2020. DOI: https://doi.org/10.7236/JIIBC.2020.20.2.283
  4. D. Shultz, "When your Voice Betrays You," Science, Vol. 347, No. 6221, 2015. DOI: https://doi.org/10.1126/science.347.6221.494
  5. Yo-Seop, Lee, "Analysis on Machine Learning-as-a-Service," International Journal of Advanced Culture Technology (IJACT), Vol. 6, No. 4, pp. 303-308, 2018. \ \ DOI: https://doi.org/10.17703/IJACT2018.6.4.303
  6. J. Bos, K. Lauter, and M. Naehrig, "Private Predictive Analysis on Encrypted Medical Data," Informatics, Vol. 50, pp. 234-243, 2014. DOI: https://doi.org/10.1016/j.jbi.2014.04.003
  7. M. Pathak, S. Rane, W. Sun, and B. Raj, "Privacy Preserving Probabilistic Inference with Hidden Markov Models," In Proc. of 2011 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1-3, 2011. DOI: https://doi.org/10.1109/ICASSP.2011.5947696
  8. P. Xie, M. Bilenko, T. Finley, R. Gilad-Bachrach, K. Lauter, and M. Naehrig, "Crypto-nets: Neural Networks over Encrypted Data," arXiv:1412.6181, 2014. DOI: https://doi.org/10.48550/arXiv.1412.6181
  9. M. Liang, Z. Li, T. Chen, and J. Zeng, "Integrative Data Analysis of Multi-Platform Cancer Data with a Multimodal Deep Learning Approach," In Proc. of IEEE/ACM Transactions on Computational Biology and Bioinformatics, Vol. 12, No. 4, pp. 928-937, 2015. DOI: https://doi.org/10.1109/TCBB.2014.2377729.
  10. M. Pathak, S. Rane, W. Sun, and B. Raj, "Multiparty Different Privacy via Aggregation of Locally Trained Classifiers," In Proc. of International Conference on Neural Information Processing Systems, pp. 1876-1884, 2010.
  11. Y, LeCun, L, Bottou, Y. Bengio, and P. Haffner, "Gradient-based Learning Applied to Document Recognition," IEEE, Vol. 86, No. 11, pp. 2278-2324, 1998. DOI: https://doi.org/10.1109/5.726791.
  12. https://github.com/torch/nn.