A Modification of Dai-Yuan's Conjugate Gradient Algorithm for Solving Unconstrained Optimization

The spectral conjugate gradient method is an essential generalization of the conjugate gradient method, and it is also one of the effective numerical methods to solve large scale unconstrained optimization problems. We propose a new spectral Dai-Yuan (SDY) conjugate gradient method to solve nonlinear unconstrained optimization problems. The proposed method's global convergence was achieved under appropriate conditions, performing numerical testing on 65 benchmark tests to determine the effectiveness of the proposed method in comparison to other methods like the AMDYN algorithm and some other existing ones like Dai-Yuan method.

Keywords

unconstrained optimization; conjugate gradient method; spectral conjugate gradient; sufficient descent; global convergence.

References

1. Alhawarat A., Salleh Z. Modification of Nonlinear Conjugate Gradient Method with Weak Wolfe-Powell Line Search. Abstract and Applied Analysis, 2017, vol. 2017, no. 2, pp. 1-6. DOI: 10.1155/2017/7238134
2. Nocedal J., Yuan Ya-Xiang. Analysis of a Self-Scaling Quasi-Newton Method. Mathematical Programming, 1993, vol. 61, no. 1, pp. 19-37. DOI: 10.1007/BF01582136
3. Hestenes M.R., Stiefel E. Methods of Conjugate Gradients for Solving Linear Systems. Journal of Research of the National Bureau of Standards, vol. 49, no. 6, pp. 409-435. DOI: 10.6028/jres.049.044
4. Fletcher R., Reeve C.M., Function Minimization by Conjugate Gradients. The Computer Journal, 1964, vol. 7, no. 2, pp. 149-154. DOI: 10.1093/comjnl/7.2.149
5. Polak E., Ribiere G. Note sur la convergence de m'ethodes de directions conjugu'ees. Mathematical Modelling and Numerical Analysis-Mod'elisation Math'ematique et Analyse Num'erique, 1969, vol. 3, no. 1, pp. 35-43. (in French)
6. Polyak B.T. The Conjugate Gradient Method in Extremal Problems. Computational Mathematics and Mathematical Physics, 1969, vol. 9, no. 4, pp. 94-112. DOI: 10.1016/0041-5553(69)90035-4
7. Dai Yu-Hong, Yuan Yaxiang. A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property. SIAM Journal on Optimization, 1999, vol. 10, no. 1, pp. 177-182.
8. Dai Yu-Hong, Li-Zhi Liao. New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods. Applied Mathematics and Optimization, 2001, vol. 43, no. 1, pp. 87-101. DOI: 10.1007/s002450010019
9. Hassan B., Abdullah Z., Jabbar H. A Descent Extension of the Dai-Yuan Conjugate Gradient Technique. Indonesian Journal of Electrical Engineering and Computer Science, 2019, vol. 16, no. 2, pp. 661-668. DOI: 10.11591/ijeecs.v16.i2.pp661-668
10. Jie Guo, Zhong Wan. A Modified Spectral PRP Conjugate Gradient Projection Method for Solving Large-Scale Monotone Equations and its Application in Compressed Sensing. Mathematical Problems in Engineering, 2019, vol. 2019, pp. 23-27. DOI: 10.1155/2019/5261830
11. Neculai A. New Accelerated Conjugate Gradient Algorithms as a Modification of Dai-Yuan's Computational Scheme for Unconstrained Optimization. Journal of Computational and Applied Mathematics, 2010, vol. 234, no. 12, pp. 3397-3410. DOI: 10.1016/j.cam.2010.05.002
12. Eman H., Rana A.Z., Abbas A.Y. New Investigation for the Liu-Story Scaled Conjugate Gradient Method for Nonlinear Optimization. Journal of Mathematics, 2020, vol. 2020, article ID: 3615208, 10 p. DOI: 10.1155/2020/3615208
13. Gaohang Yu, Lutai Guan, Wufan Chen. Spectral Conjugate Gradient Methods with Sufficient Descent Property for Large-Scale Unconstrained Optimization. Optimization Methods and Software, 2008, vol. 23, no. 2, pp. 275-293. DOI: 10.1080/10556780701661344
14. Ibrahim Sulaiman Mohammed, Yakubu Usman Abbas, Mamat Mustafa. Application of Spectral Conjugate Gradient Methods for Solving Unconstrained Optimization Problems. An International Journal of Optimization and Control: Theories and Applications, 2020, vol. 10, no. 2, pp. 198-205.
15. Wang Li, Cao Mingyuan, Xing Funa, Yang Yueting. The New Spectral Conjugate Gradient Method for Large-Scale Unconstrained Optimisation. Journal of Inequalities and Applications, 2020, vol. 2020, no. 1, pp. 1-11. DOI: 10.1186/s13660-020-02375-z
16. Jian Jinbao, Yang Lin, Jiang Xianzhen, Liu Pengjie, Liu Meixing. A Spectral Conjugate Gradient Method with Descent Property. Mathematics, 2020, vol. 8, no. 2, article ID: 280, 13 p. DOI: 10.3390/math8020280
17. Danhausa A.A., Odekunle R.M., Onanaye A.S. A Modified Spectral Conjugate Gradient Method for Solving Unconstrained Minimization Problems. Journal of the Nigerian Mathematical Society, 2020, vol. 39, no. 2, pp. 223-237.
18. Al-Arbo A., Rana Al-Kawaz. A Fast Spectral Conjugate Gradient Method for Solving Nonlinear Optimization Problems. Indonesian Journal of Electrical Engineering and Computer Science, 2021, vol. 21, no. 1, pp. 429-439. DOI: 10.11591/ijeecs.v21.i1.pp429-439
19. Hassan Basim, Jabbar Hawraz. A New Spectral on the Gradient Methods for Unconstrained Optimization Minimization Problem. Journal of Zankoy Sulaimani, 2020, vol. 22, no. 2, pp. 217-224. DOI: 10.17656/jzs.10822
20. Liu J.K., Feng Y.M., Zou L.M. A Spectral Conjugate Gradient Method for Solving Large-Scale Unconstrained Optimization. Computers and Mathematics with Applications, 2019, vol. 77, no. 3, pp. 731-739.
21. Al-Bayati A.Y. A New Family of Self-Scaling Variable Metric Algorithms for Unconstrained Optimization. Journal of Education and Science, 1991, vol. 12, pp. 25-54.
22. Neculai A. An Unconstrained Optimization Test Functions Collection. Advanced Modeling and Optimization, 2008, vol. 10, no. 1, pp. 147-161.
23. Neculai A. New Accelerated Conjugate Gradient Algorithms for Unconstrained Optimization. Technical Report, 2008, vol. 34, pp. 319-330.
24. Dolan E.D., More J.J. Benchmarking Optimization Software with Performance Profiles. Mathematical Programming, 2002, vol. 91, no. 2, pp. 201-213. DOI: 10.1007/s101070100263