Multi-gradient learning algorithm for multilayer neural networks

다층 신경망을 위한 Multi-gradient 학습 알고리즘

Recently, a new learning algorithm for multilayer neural networks has been proposed 〔1〕. In the new learning algorithm, each output neuron is considered as a function of weights and the weights are adjusted so that the output neurons produce desired outputs. And the adjustment is accomplished by taking gradients. However, the gradient computation was performed numerically, resulting in a long computation time. In this paper, we derive the all necessary equations so that the gradient computation is performed analytically, resulting in a much faster learning time comparable to the backpropagation. Since the weight adjustments are accomplished by summing the gradients of the output neurons, we will call the new learning algorithm “multi-gradient.” Experiments show that the multi-gradient consistently outperforms the backpropagation.