• Journal of Practical Engineering Education
• /
• v.14 no.2
• /
• pp.305-312
• /
• 2022

In this paper, we focused on training to create and optimize a basic data prediction model. And we proposed a gradient descent training method of machine learning that is widely used to optimize data prediction models. It visually shows the entire operation process of gradient descent used in the process of optimizing parameter values required for data prediction models by applying the differential method and teaches the effective use of mathematical differentiation in machine learning. In order to visually explain the entire operation process of gradient descent, we implement gradient descent SW in a spreadsheet. In this paper, first, a two-variable gradient descent training method is presented, and the accuracy of the two-variable data prediction model is verified by comparison with the error least squares method. Second, a three-variable gradient descent training method is presented and the accuracy of a three-variable data prediction model is verified. Afterwards, the direction of the optimization practice for gradient descent was presented, and the educational effect of the proposed gradient descent method was analyzed through the results of satisfaction with education for non-majors.

• Journal of applied mathematics & informatics
• /
• v.20 no.1_2
• /
• pp.239-254
• /
• 2006

An efficient descent method for unconstrained optimization problems is line search method in which the step size is required to choose at each iteration after a descent direction is determined. There are many ways to choose the step sizes, such as the exact line search, Armijo line search, Goldstein line search, and Wolfe line search, etc. In this paper we propose a new inexact line search for a general descent method and establish some global convergence properties. This new line search has many advantages comparing with other similar inexact line searches. Moreover, we analyze the global convergence and local convergence rate of some special descent methods with the new line search. Preliminary numerical results show that the new line search is available and efficient in practical computation.

• Communications of Mathematical Education
• /
• v.37 no.3
• /
• pp.467-482
• /
• 2023

In this paper, we present our new teaching and learning materials on gradient descent method, which is widely used in artificial intelligence, available for college mathematics. These materials provide a good explanation of gradient descent method at the level of college calculus, and the presented SageMath code can help students to solve minimization problems easily. And we introduce how to solve least squares problem using gradient descent method. This study can be helpful to instructors who teach various college-level mathematics subjects such as calculus, engineering mathematics, numerical analysis, and applied mathematics.

• Proceedings of the KIEE Conference
• /
• 1987.07b
• /
• pp.1595-1597
• /
• 1987

Two gradient methods, steepest descent method and conjugate gradient descent method, are compar ed through application to vector linear predictors. It is found that the convergence rate of the conju-gate gradient descent method is much faster than that of the steepest descent method.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.21 no.2
• /
• pp.189-194
• /
• 2020

This paper analyzes the gradient descent method, which is the one most used for learning neural networks. Learning means updating a parameter so the loss function is at its minimum. The loss function quantifies the difference between actual and predicted values. The gradient descent method uses the slope of the loss function to update the parameter to minimize error, and is currently used in libraries that provide the best deep learning algorithms. However, these algorithms are provided in the form of a black box, making it difficult to identify the advantages and disadvantages of various gradient descent methods. This paper analyzes the characteristics of the stochastic gradient descent method, the momentum method, the AdaGrad method, and the Adadelta method, which are currently used gradient descent methods. The experimental data used a modified National Institute of Standards and Technology (MNIST) data set that is widely used to verify neural networks. The hidden layer consists of two layers: the first with 500 neurons, and the second with 300. The activation function of the output layer is the softmax function, and the rectified linear unit function is used for the remaining input and hidden layers. The loss function uses cross-entropy error.

• East Asian mathematical journal
• /
• v.32 no.2
• /
• pp.193-215
• /
• 2016

There are three subjects in the study. First, after investigating the development process of the method of infinite descent and the reduction to absurdity, we prove them to be equivalent each other. Second, we apply the method of infinite descent to some problems in textbook and compare it with the reduction to absurdity. Finally, we discuss on teaching proofs with the method of infinite descent.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.11 no.6
• /
• pp.396-401
• /
• 1986

The method of steepest descent is applied to the analysis of electrostatic problems. The differences between iterative method and direct method, e.g. the method of moments, are not lined. It is shown that this method converges monotonically to the exact solution and is suitable for solving a problem of large system. Numerical results are presented for electrostatic case which show a good agreement with momet solution.

• Industrial Engineering and Management Systems
• /
• v.4 no.1
• /
• pp.94-101
• /
• 2005

This paper presents the first known application of a meta-heuristic algorithm, variable neighbourhood descent (VND), to the redundancy allocation problem (RAP). The RAP, a well-known NP-hard problem, has been the subject of much prior work, generally in a restricted form where each subsystem must consist of identical components. The newer meta-heuristic methods overcome this limitation and offer a practical way to solve large instances of the relaxed RAP where different components can be used in parallel. The variable neighbourhood descent method has not yet been used in reliability design, yet it is a method that fits perfectly in those combinatorial problems with potential neighbourhood structures, as in the case of the RAP. A variable neighbourhood descent algorithm for the RAP is developed and tested on a set of well-known benchmark problems from the literature. Results on 33 test problems ranging from less to severely constrained conditions show that the variable neighbourhood descent method provides comparable solution quality at a very moderate computational cost in comparison with the best-known heuristics. Results also indicate that the VND method performs with little variability over random number seeds.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.15 no.12
• /
• pp.7277-7282
• /
• 2014

In this study, the gradient descent algorithm was used for FLC analysis and the algorithm was used to represent the effects of nonlinear parameters, which alter the antecedent and consequence fuzzy variables of FLC. The controller parameters choose the control variable by iteration for gradient descent algorithm. The FLC consists of 7 membership functions, 49 rules and a two inputs - one output system. The system adopted the Min-Max inference method and triangle type membership function with a 13 quantization level.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2009.01a
• /
• pp.460-465
• /
• 2009

This paper presents a new algorithm that includes a mechanism to avoid local solutions in a motion vector detection method that uses the steepest descent method. Two different implementations of the algorithm are demonstrated using two major search methods for tree structures, depth first search and breadth first search. Furthermore, it is shown that by avoiding local solutions, both of these implementations are able to obtain smaller prediction errors compared to conventional motion vector detection methods using the steepest descent method, and are able to perform motion vector detection within an arbitrary upper limit on the number of computations. The effects that differences in the search order have on the effectiveness of avoiding local solutions are also presented.