Journal of the Korean Mathematical Society (대한수학회지)

Korean Mathematical Society (대한수학회)
권호 표지
DOI QR코드

DOI QR Code

MODIFIED LIMITED MEMORY BFGS METHOD WITH NONMONOTONE LINE SEARCH FOR UNCONSTRAINED OPTIMIZATION

  • Yuan, Gonglin (COLLEGE OF MATHEMATICS AND INFORMATION SCIENCE GUANGXI UNIVERSITY) ;
  • Wei, Zengxin (COLLEGE OF MATHEMATICS AND INFORMATION SCIENCE GUANGXI UNIVERSITY) ;
  • Wu, Yanlin (COLLEGE OF MATHEMATICS AND INFORMATION SCIENCE GUANGXI UNIVERSITY)
  • Received : 2008.07.17
  • Published : 2010.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose two limited memory BFGS algorithms with a nonmonotone line search technique for unconstrained optimization problems. The global convergence of the given methods will be established under suitable conditions. Numerical results show that the presented algorithms are more competitive than the normal BFGS method.

Keywords

References

  1. C. G. Broyden, J. E. Dennis Jr, and J. J. More, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl. 12 (1973), 223-245. https://doi.org/10.1093/imamat/12.3.223
  2. R. H. Byrd and J. Nocedal, A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, SIAM J. Numer. Anal. 26 (1989), no. 3, 727-739. https://doi.org/10.1137/0726042
  3. R. H. Byrd, J. Nocedal, and R. B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods, Math. Programming 63 (1994), no. 2, Ser. A, 129-156. https://doi.org/10.1007/BF01582063
  4. R. Byrd, J. Nocedal, and Y. Yuan, Global convergence of a class of quasi-Newton methods on convex problems, SIAM J. Numer. Anal. 24 (1987), no. 5, 1171-1190. https://doi.org/10.1137/0724077
  5. Y. Dai, Convergence properties of the BFGS algorithm, SIAM J. Optim. 13 (2002), no. 3, 693-701. https://doi.org/10.1137/S1052623401383455
  6. Y. Dai and Q. Ni, Testing different conjugate gradient methods for large-scale unconstrained optimization, J. Comput. Math. 21 (2003), no. 3, 311-320.
  7. W. C. Davidon, Variable metric methods for minimization, A. E. C. Research and Development Report ANL-599, 1959.
  8. R. Dembo and T. Steihaug, Truncated Newton algorithms for large-scale unconstrained optimization, Math. Programming 26 (1983), no. 2, 190-212. https://doi.org/10.1007/BF02592055
  9. J. E. Dennis Jr. and J. J. More, Quasi-Newton methods, motivation and theory, SIAM Rev. 19 (1977), no. 1, 46-89. https://doi.org/10.1137/1019005
  10. J. E. Dennis Jr. and J. J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28 (1974), 549-560. https://doi.org/10.1090/S0025-5718-1974-0343581-1
  11. J. E. Dennis Jr. and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall Series in Computational Mathematics. Prentice Hall, Inc., Englewood Cliffs, NJ, 1983.
  12. L. C. W. Dixon, Variable metric algorithms: necessary and sufficient conditions for identical behavior of nonquadratic functions, J. Optimization Theory Appl. 10 (1972), 34-40. https://doi.org/10.1007/BF00934961
  13. E. D. Dolan and J. J. More, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002), no. 2, Ser. A, 201-213. https://doi.org/10.1007/s101070100263
  14. R. Fletcher, Practical Methods of Optimization, Second edition. A Wiley-Interscience Publication. John Wiley & Sons, Ltd., Chichester, 1987.
  15. N. I. M. Gould, D. Orban, and Ph. L. Toint, CUTEr (and SifDec), a constrained and unconstrained testing environment, revisite, ACM Transactions on Mathematical Software 29 (2003), 373-394. https://doi.org/10.1145/962437.962439
  16. A. Griewank, The global convergence of partitioned BFGS on problems with convex decompositions and Lipschitzian gradients, Math. Programming 50 (1991), no. 2, (Ser. A), 141-175. https://doi.org/10.1007/BF01594933
  17. A. Griewank and Ph. L. Toint, Local convergence analysis for partitioned quasi-Newton updates, Numer. Math. 39 (1982), no. 3, 429-448. https://doi.org/10.1007/BF01407874
  18. L. Grippo, F. Lamparillo, and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (1986), no. 4, 707-716. https://doi.org/10.1137/0723046
  19. L. Grippo, F. Lamparillo, and S. Lucidi, A truncated Newton method with nonmonotone line search for unconstrained optimization, J. Optim. Theory Appl. 60 (1989), no. 3, 401-419. https://doi.org/10.1007/BF00940345
  20. L. Grippo, F. Lamparillo, and S. Lucidi, A class of nonmonotone stabilization methods in unconstrained optimization, Numer. Math. 59 (1991), no. 8, 779-805. https://doi.org/10.1007/BF01385810
  21. J. Y. Han and G. H. Liu, Global convergence analysis of a new nonmonotone BFGS algorithm on convex objective functions, Comput. Optim. Appl. 7 (1997), no. 3, 277-289. https://doi.org/10.1023/A:1008656711925
  22. D. Li and M. Fukushima, A modified BFGS method and its global convergence in non-convex minimization, J. Comput. Appl. Math. 129 (2001), no. 1-2, 15-35. https://doi.org/10.1016/S0377-0427(00)00540-9
  23. L. Grippo, F. Lamparillo, and S. Lucidi, On the global convergence of the BFGS method for nonconvex unconstrained optimization problems, SIAM J. Optim. 11 (2001), no. 4, 1054-1064. https://doi.org/10.1137/S1052623499354242
  24. G. Li, C. Tang, and Z.Wei, New conjugacy condition and related new conjugate gradient methods for unconstrained optimization, J. Comput. Appl. Math. 202 (2007), no. 2, 523-539. https://doi.org/10.1016/j.cam.2006.03.005
  25. G. H. Liu and J. Y. Han, Notes on the general form of stepsize selection, OR and Decision Making I (1992), 619-624.
  26. G. H. Liu and J. Y. Han, Global convergence Analysis of the variable metric algorithm with a generalized Wolf linesearch, Technical Report, Institute of Applied Mathematics, Academia Sinica, Beijing, China, no. 029, 1993.
  27. G. H. Liu, J. Y. Han, and D. F. Sun, Global convergence of the BFGS algorithm with nonmonotone linesearch, Optimization 34 (1995), no. 2, 147-159. https://doi.org/10.1080/02331939508844101
  28. G. H. Liu and J. M. Peng, The convergence properties of a nonmonotonic algorithm, J. Comput. Math. 1 (1992), 65-71.
  29. W. F. Mascarenhas, The BFGS method with exact line searches fails for non-convex objective functions, Math. Program. 99 (2004), no. 1, Ser. A, 49-61. https://doi.org/10.1007/s10107-003-0421-7
  30. J. J. More, B. S. Garbow, and K. E. Hillstrome, Testing unconstrained optimization software, ACM Trans. Math. Software 7 (1981), no. 1, 17-41. https://doi.org/10.1145/355934.355936
  31. S. G. Nash, A survey of truncated-Newton methods, Numerical analysis 2000, Vol. IV, Optimization and nonlinear equations. J. Comput. Appl. Math. 124 (2000), no. 1-2, 45-59. https://doi.org/10.1016/S0377-0427(00)00426-X
  32. M. J. D. Powell, On the convergence of the variable metric algorithm, J. Inst. Math. Appl. 7 (1971), 21-36. https://doi.org/10.1093/imamat/7.1.21
  33. M. J. D. Powell, Some global convergence properties of a variable metric algorithm for minimization without exact line searches, Nonlinear programming (Proc. Sympos., New York, 1975), pp. 53–72. SIAM-AMS Proc., Vol. IX, Amer. Math. Soc., Providence, R. I., 1976.
  34. M. J. D. Powell, A new algorithm for unconstrained optimization, 1970 Nonlinear Programming (Proc. Sympos., Univ. of Wisconsin, Madison, Wis., 1970) pp. 31-65 Academic Press, New York.
  35. M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J. Optim. 7 (1997), no. 1, 26-33. https://doi.org/10.1137/S1052623494266365
  36. J. Schropp, A note on minimization problems and multistep methods, Numer. Math. 78 (1997), no. 1, 87-101. https://doi.org/10.1007/s002110050305
  37. J. Schropp, One-step and multistep procedures for constrained minimization problems, IMA J. Numer. Anal. 20 (2000), no. 1, 135-152. https://doi.org/10.1093/imanum/20.1.135
  38. Ph. L. Toint, Global convergence of the partitioned BFGS algorithm for convex partially separable optimization, Math. Programming 36 (1986), no. 3, 290-306. https://doi.org/10.1007/BF02592063
  39. D. J. van Wyk, Differential optimization techniques, Appl. Math. Modelling 8 (1984), no. 6, 419-424. https://doi.org/10.1016/0307-904X(84)90048-9
  40. M. N. Vrahatis, G. S. Androulakis, J. N. Lambrinos, and G. D. Magolas, A class of gradient unconstrained minimization algorithms with adaptive stepsize, J. Comput. Appl. Math. 114 (2000), no. 2, 367-386. https://doi.org/10.1016/S0377-0427(99)00276-9
  41. Z. Wei, G. Li, and L. Qi, New quasi-Newton methods for unconstrained optimization problems, Appl. Math. Comput. 175 (2006), no. 2, 1156-1188. https://doi.org/10.1016/j.amc.2005.08.027
  42. Z. Wei, G. Yu, G. Yuan, and Z. Lian, The superlinear convergence of a modified BFGS-type method for unconstrained optimization, Comput. Optim. Appl. 29 (2004), no. 3, 315-332. https://doi.org/10.1023/B:COAP.0000044184.25410.39
  43. G. L. Yuan, Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems, Optim. Lett. 3 (2009), no. 1, 11-21. https://doi.org/10.1007/s11590-008-0086-5
  44. G. L. Yuan and X. W. Lu, A new line search method with trust region for unconstrained optimization, Comm. Appl. Nonlinear Anal. 15 (2008), no. 1, 35-49.
  45. G. L. Yuan and X. W. Lu, A modified PRP conjugate gradient method, Ann. Oper. Res. 166 (2009), 73-90. https://doi.org/10.1007/s10479-008-0420-4
  46. G. L. Yuan, X. Lu, and Z. Wei, A conjugate gradient method with descent direction for unconstrained optimization, J. Comput. Appl. Math. 233 (2009), no. 2, 519-530. https://doi.org/10.1016/j.cam.2009.08.001
  47. Y. Yuan and W. Sun, Theory and Methods of Optimization, Science Press of China, 1999.
  48. G. L. Yuan and Z. X. Wei, New line search methods for unconstrained optimization, J. Korean Statist. Soc. 38 (2009), no. 1, 29-39. https://doi.org/10.1016/j.jkss.2008.05.004
  49. G. L. Yuan and Z. X. Wei, The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions, Acta Math. Sin. (Engl. Ser.) 24 (2008), no. 1, 35-42. https://doi.org/10.1007/s10114-007-1012-y
  50. G. L. Yuan and Z. X. Wei, Convergence analysis of a modified BFGS method on convex minimizations, Comput. Optim. Appl. doi: 10.1007/s10589-008-9219-0.
  51. J. Z. Zhang, N. Y. Deng, and L. H. Chen, New quasi-Newton equation and related methods for unconstrained optimization, J. Optim. Theory Appl. 102 (1999), no. 1, 147-167. https://doi.org/10.1023/A:1021898630001

Cited by

  1. A modified nonmonotone BFGS algorithm for unconstrained optimization vol.2017, pp.1, 2017, https://doi.org/10.1186/s13660-017-1453-5
  2. A Trust Region Algorithm with Conjugate Gradient Technique for Optimization Problems vol.32, pp.2, 2011, https://doi.org/10.1080/01630563.2010.532273
  3. Gradient trust region algorithm with limited memory BFGS update for nonsmooth convex minimization vol.54, pp.1, 2013, https://doi.org/10.1007/s10589-012-9485-8
  4. An active-set projected trust-region algorithm with limited memory BFGS technique for box-constrained nonsmooth equations vol.62, pp.7, 2013, https://doi.org/10.1080/02331934.2011.603321
  5. A modified Polak–Ribière–Polyak conjugate gradient algorithm for nonsmooth convex programs vol.255, 2014, https://doi.org/10.1016/j.cam.2013.04.032
  6. A quasi-Newton algorithm for large-scale nonlinear equations vol.2017, pp.1, 2017, https://doi.org/10.1186/s13660-017-1301-7
  7. A BFGS algorithm for solving symmetric nonlinear equations vol.62, pp.1, 2013, https://doi.org/10.1080/02331934.2011.564621
  8. An active-set projected trust region algorithm for box constrained optimization problems vol.28, pp.5, 2015, https://doi.org/10.1007/s11424-014-2199-5
  9. A modified three-term PRP conjugate gradient algorithm for optimization models vol.2017, pp.1, 2017, https://doi.org/10.1186/s13660-017-1373-4
  10. The LBFGS quasi-Newtonian method for molecular modeling prion AGAAAAGA amyloid fibrils vol.04, pp.12, 2012, https://doi.org/10.4236/ns.2012.412A138