Modification of Three-Term Conjugate Gradient Method for Solving Unconstrained Optimization and Image Restoration Problems

Nonlinear conjugate gradient algorithm is highly effective for optimization due to its low storage requirements and simple structure properties. Expanding on the Barzilai and Borwein conjugate gradient method, we propose a three-term conjugate gradient method with a restart procedure for unconstrained optimization. This method ensures global convergence under standard assumptions and employs a standard Wolfe line search. To evaluate its performance, we carry out comprehensive numerical experiments for large scales to address challenges in unconstrained optimization and image restoration. The numerical results prove that the new method is more effective compared to other classical methods.

Keywords

unconstrained optimization; line search; three-term conjugate gradient method; global convergence; image restoration.

References

1. Hestenes M., Stiefel E. Method of Conjugate Gradients for Solving Linear Systems. Journal of Research of the National Bureau of Standards, 1952, vol. 49, no. 6, pp. 409-436.
2. Fletcher R., Reeves C.M. Function Minimization by Conjugate Gradients. The Computer Journal, 1964, vol. 7, pp. 149-154. DOI: 10.1093/comjnl/7.2.149
3. Polak E., Ribiere G. Note sur la convergence de methodes de directions conjuguees. ESAIM: Mathematical Modelling and Numerical Analysis, 1969, vol. 3, pp. 33-43. DOI: 10.1051/m2an/196903R100351 (in French)
4. Fletcher R. Practical Methods of Optimization. New York, John Wiley and Sons, 1987.
5. Y. Liu, Storey C. Efficient Generalized Conjugate Gradient Algorithms. Part 1: Theory. Journal of Optimization Theory and Applications, 1991, vol. 69, pp. 129-137.
6. Dai Yu-Hong, Yuan Ya-Xiang. A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property. SIAM Journal on Optimization, 1999, vol. 10, no. 1, pp. 177-182. DOI: 10.1137/S1052623497318992
7. Nazareth L. A Conjugate Direction Algorithm without Line Searches. Journal of Optimization Theory and Applications, 1977, vol. 23, pp. 373-387.
8. Zhang Li, Zhou Weijun, Li Donghui. Global Convergence of a Modified Fletcher-Reeves Conjugate Gradient Method with Armijo-Type Line Search. Numerische Mathematik, 2006, vol. 104, no. 4, pp. 561-572. DOI: 10.1007/s00211-006-0028-z
9. Barzilai J., Borwein J.M. Two-Point Step Size Gradient Methods. IMA Journal of Numerical Analysis, 1988, vol. 8, no. 1, pp. 141-148.
10. Zoutendijk G. Nonlinear Programming, Computational Methods. Integer and Nonlinear Programming, 1970, pp. 37-86.
11. Dolan E.D., More J.J. Benchmarking Optimization Software with Performance Profiles. Mathematical Programming, 2002, vol. 91, pp. 201-213. DOI: 10.1007/s101070100263
12. Maulana M., Sulaiman I.M., Abubakar A.B., Ardaneswari G. A New Family of Hybrid Three-Term Conjugate Gradient Method for Unconstrained Optimization with Application to Image Restoration and Portfolio Selection. AIMS Mathematics, 2023, vol. 8, no. 1, pp. 1-28. DOI: 10.3934/math.2023001
13. Alsuliman S.N., Ibrahim N.F., Aizam N.A. A New Conjugate Gradient Parameter via Modification of Liu-Storey Formula for Optimization Problem and Image Restoration. Journal of Advanced Research in Applied Sciences and Engineering Technology, 2024, vol. 35, no. 2, pp. 175-186. DOI: 10.37934/araset.35.2.175186
14. Li Yixin, Li Chunguang, Yang Wei, Zhang Wensheng. A New Conjugate Gradient Method with a Restart Direction and Its Application in Image Restoration. AIMS Mathematics, 2023, vol. 8, no. 12, pp. 28791-28807. DOI: 10.3934/math.20231475