A new hybrid conjugate gradient algorithm for unconstrained optimization

Document Type : Research Paper

Authors

School of Mathematics and Statistics‎, ‎Southwest University‎, ‎Chongqing 400715‎, ‎P.R‎. ‎China.

Abstract

In this paper, a new hybrid conjugate gradient algorithm is proposed for solving unconstrained optimization problems. This new method can generate sufficient descent directions unrelated to any line search. Moreover, the global convergence of the proposed method is proved under the Wolfe line search. Numerical experiments are also presented to show the efficiency of the proposed algorithm, especially for solving highly dimensional problems.

Keywords

Main Subjects


M. Al-Baali, Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal. 5 (1985) 121--124.
N. Andrei, Numerical comparison of conjugate gradient algorithms for unconstrained optimization, Stud. Inform. Control. 16 (2007) 333--352.
N. Andrei, An unconstrained optimization test functions collection, Adv. Model. Optim. 10 (2008) 147--161.
J. Chen, Q. Ansari, Y. Liu and J.C. Yao, A proximal point algorithm based on decomposition method for cone constrained multiobjective optimization problems, Comput. Optim. Appl. 65 (2016) 289--308.
Y.H. Dai and Q. Ni, Testing different conjugate gradient methods for large-scale unconstrained optimization, J. Comput. Math. 22 (2003) 311--320.
Y.H. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim. 10 (1999) 177--182.
Y.H. Dai and Y. Yuan, An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res. 103 (2001) 33--47.
E.D. Dolan and J.J. Moré, Benchmarking optimization software with performance profiles, Math. Program. 91 (2002) 201--213.
Z. Dai and F.Wen, Another improvedWei-Yao-Liu nonlinear conjugate gradient method with sufficient descent property, Appl. Math. Comput. 218 (2012) 7421--7430.
R. Fletcher and C. Reeves, Function minimization by conjugate gradients, Comput. J. 7 (1964) 149--154.
J.C. Gilbert and J. Nocedal, Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim. 2 (1992) 21--42.
W.W. Hager and H. Zhang, Algorithm 851: CG Descent, a conjugate gradient method with guaranteed descent, ACM Trans. Math. Softw. 32 (2006) 113--137.
W.W. Hager and H. Zhang, A survey of nonlinear conjugate gradient methods, Pac. J. Optim. 2 (2006) 35--58.
M.R. Hestenes and E.L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand. Sec. B 49 (1952) 409--432.
Y.F. Hu and C. Storey, Global convergence result for conjugate gradient methods, J. Optim. Theory Appl. 71 (1991) 399--405.
H. Huang, Z. Wei and S. Yao, The proof of the sufficient descent condition of the Wei-Yao-Liu conjugate gradient method under the strong Wolfe-Powell line search, Appl. Math. Comput. 189 (2007) 1241--1245.
J. Jian, L. Han and X. Jiang, A hybrid conjugate gradient method with descent property for unconstrained optimization, Appl. Math. Model. 39 (2015) 1281--1290.
X. Jiang, L. Han and J. Jian, A globally convergent mixed conjugate gradient method with Wolfe line search, Math. Numer. Sin. 34 (2012) 103--112.
J. Nocedal and S.J. Wright, Numerical Optimization, Springer, New York, 2006.
E. Polak and G. Ribière, Note sur la convergence de directions conjugées, Rev. Fr. Inf. Rech. Oper. 16 (1969) 35--43.
B.T. Polyak, The conjugate gradient method in extremal problems, USSR Comput. Math. Math. Phys. 9 (1969) 94--112.
M.J.D. Powell, Restart procedures for the conjugate gradient method, Math. Program. 12 (1977) 241--254.
M.J.D. Powell, Convergence properties of algorithms for nonlinear optimization, SIAM Rev. 28 (1986) 487--500.
D.F. Shanno and K.H. Phua, Algorithm 500. Minimization of unconstrained multivariate functions, ACM Trans. Math. Softw. 2 (1976) 87--94.
S. Yao, Z. Wei and H. Huang, A note about WYL's conjugate gradient method and its application, Appl. Math. Comput. 191 (2007) 381--388.
Z. Wei, G. Li and L. Qi, New nonliear conjugate gradient formulas for large-scale unconstrained optimization problems, Appl. Math. Comput. 179 (2006) 407--430.
Z. Wei, S. Yao and L. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput. 183 (2006) 1341--1350.
L. Zhang, An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation, Appl. Math. Comput. 215 (2009) 2269--2274.
G. Zoutendijk, Nonlinear programming computational methods, Integer and Nonlinear Programming in: J. Abadie (Ed.), Integer and Nonlinear Programming, North-Holland, Amsterdam, 1970, pp. 37--86.