Korean Journal of Mathematics

The Kangwon-Kyungki Mathematical Society (강원경기수학회)
권호 표지
DOI QR코드

DOI QR Code

GRADIENTS IN A DEEP NEURAL NETWORK AND THEIR PYTHON IMPLEMENTATIONS

  • Park, Young Ho (Department of Mathematics, Kangwon National University)
  • Received : 2021.12.05
  • Accepted : 2022.03.01
  • Published : 2022.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

This is an expository article about the gradients in deep neural network. It is hard to find a place where gradients in a deep neural network are dealt in details in a systematic and mathematical way. We review and compute the gradients and Jacobians to derive formulas for gradients which appear in the backpropagation and implement them in vectorized forms in Python.

Keywords

References

  1. A. Geron, Hands-on Machine Learning with Scikit-Learn & TensorFlow (핸즈온 머신러닝), Hanbit Media, 2018
  2. Andrew Ng, Note on neural network and deep learning, https://github.com/ashishpatel26/Andrew-NG-Notes, accessed February 23, 2022
  3. Y.H. Park, Verifying gradient formulas by PyTorch, https://deepmath.kangwon.ac.kr/-yhpark/verify gradients.pdf, accessed February 23, 2022
  4. T. Rashid, Make your own neural network (신경망 첫걸음), Hanbit Media, 2017
  5. J. Stewart, Calculus, Books-Hill, 2021