A Spectral Hybrid Conjugate Gradient Approach for Large-Scale Unconstrained Optimization with Applications in Image Restoration

Authors

  • Meysam Ranjbar * Department of Mathematics‎, ‎Faculty of Mathematics‎, ‎Statistics and Computer Science‎, ‎Semnan University‎, ‎Semnan‎, ‎Iran.
  • Ali Ashrafi Department of Mathematics‎, ‎Faculty of Mathematics‎, ‎Statistics and Computer Science‎, ‎Semnan University‎, ‎Semnan‎, ‎Iran.

https://doi.org/10.48314/tsc.v1i2.45

Abstract

This paper introduces a Spectral Hybrid Conjugate Gradient (SHCG) method for large-scale unconstrained optimization. By modifying the Hestenes–Stiefel formula with a correction term and a safeguarded denominator, and embedding it into a spectral framework, the proposed method combines the benefits of hybridization and spectral scaling. The resulting algorithm guarantees sufficient descent independently of line search and ensures global convergence under standard assumptions. Numerical experiments, including applications to image restoration, demonstrate that the SHCG method achieves faster convergence and higher robustness compared with several existing Conjugate Gradient (CG) algorithms.         

Keywords:

Unconstrained optimization, Spectral conjugate gradient method, Hybrid conjugate gradient method, Global convergence, Image restoration

References

  1. [1] Wolfe, P. (1969). Convergence conditions for ascent methods. SIAM review, 11(2), 226–235. https://doi.org/10.1137/1011036
  2. [2] Fletcher, R., & Reeves, C. M. (1964). Function minimization by conjugate gradients. The computer journal, 7(2), 149–154. https://doi.org/10.1093/comjnl/7.2.149
  3. [3] Polak, B. T. (1969). The conjugate gradient method in extreme problems. USSR comput. math. math. phys, 9(4), 94–112. https://doi.org/10.1016/0041-5553(69)90035-4
  4. [4] Hestenes, M. R., Stiefel, E., & others. (1952). Methods of conjugate gradients for solving linear systems. Journal of research of the national bureau of standards, 49(6), 409–436. https://books.google.nl/books
  5. [5] Liu, Y., & Storey, C. (1991). Efficient generalized conjugate gradient algorithms, part 1: theory. Journal of optimization theory and applications, 69(1), 129–137. https://doi.org/10.1007/BF00940464
  6. [6] Dai, Y.-H., & Yuan, Y. (1999). A nonlinear conjugate gradient method with a strong global convergence property. SIAM journal on optimization, 10(1), 177–182. https://doi.org/10.1137/S1052623497318992
  7. [7] Chopra, R., & Saxena, R. R. (2019). Linear Programming Problems with coefficients as Trapezoidal Intuitionistic Fuzzy Numbers. Advances in fuzzy mathematics. issn 0973-533x, 14(1), 41–52. http://www.ripublication.com
  8. [8] Barzilai, J., & Borwein, J. M. (1988). Two-point step size gradient methods. IMA journal of numerical analysis, 8(1), 141–148. https://doi.org/10.1093/imanum/8.1.141
  9. [9] Xu, X., & Kong, F. (2016). New hybrid conjugate gradient methods with the generalized Wolfe line search. SpringerPlus, 5(1), 881. https://doi.org/10.1186/s40064-016-2522-9%0A%0A
  10. [10] Mendoza, H. S. A., Quiroz, E. A. P., & Lengua, M. A. C. (2021). An overview on conjugate gradient methods for optimization, extensions and applications [presentation]. 2021 ieee engineering international research conference (eircon) (pp. 1–4). https://doi.org/10.1109/EIRCON52903.2021.9613264
  11. [11] Salihu, N., Kumam, P., Sulaiman, I. M., Arzuka, I., & Kumam, W. (2024). An efficient Newton-like conjugate gradient method with restart strategy and its application. Mathematics and computers in simulation, 226, 354–372. https://doi.org/10.1016/j.matcom.2024.07.008
  12. [12] Djordjević, S. S. (2016). New hybrid conjugate gradient method as a convex combination of FR and PRP methods. Filomat, 30(11), 3083–3100. https://www.jstor.org/stable/24899318%0A
  13. [13] Djordjević, S. S. (2017). New hybrid conjugate gradient method as a convex combination of LS and CD methods. Filomat, 31(6), 1813–1825. https://www.jstor.org/stable/24902273%0A
  14. [14] Kaelo, P., Narayanan, S., & Thuto, M. V. (2017). A modified quadratic hybridization of Polak-Ribiere-Polyak and Fletcher-Reeves conjugate gradient method for unconstrained optimization problems. An international journal of optimization and control: theories & applications (IJOCTA), 7(2), 177–185. https://pdfs.semanticscholar.org/2883/39f246ac8ca0fc9dd07cbff7a13f6929f313.pdf
  15. [15] Hassan, B. A., Owaid, A. O., & Yasen, Z. T. (2020). A variant of hybrid conjugate gradient methods based on the convex combination for optimization. Indonesian journal of electrical engineering and computer science, 20(2), 1007–1015. https://www.researchgate.net
  16. [16] Alhawarat, A., Salleh, Z., & Masmali, I. A. (2021). A convex combination between two different search directions of conjugate gradient method and application in image restoration. Mathematical problems in engineering, 2021(1), 9941757. https://doi.org/10.1155/2021/9941757
  17. [17] Mohamed, N. S., Mamat, M., Rivaie, M., & Shaharudin, S. M. (2019). A comparison on classical-hybrid conjugate gradient method under exact line search. International journal of advances in intelligent informatics, 5(2), 150–168. http://dx.doi.org/10.26555/ijain.v5i2.356
  18. [18] Zhang, L. (2020). A derivative-free conjugate residual method using secant condition for general large-scale nonlinear equations. Numerical algorithms, 83(4), 1277–1293. https://doi.org/10.1007/s11075-019-00725-7%0A%0A
  19. [19] Cai, J. F., Chan, R., & Morini, B. (2007). Minimization of an edge-preserving regularization functional by conjugate gradient type methods. Image processing based on partial differential equations: proceedings of the international conference on PDE-based image processing and related inverse problems, CMA, Oslo, August 8–12, 2005 (pp. 109-122). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-33267-1_7%0A%0A
  20. [20] Chen, X., & Zhou, W. (2010). Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM journal on imaging sciences, 3(4), 765–790. https://doi.org/10.1137/080740167
  21. [21] Malik, M., Sulaiman, I. M., Abubakar, A. B., Ardaneswari, G., & others. (2023). A new family of hybrid three-term conjugate gradient method for unconstrained optimization with application to image restoration and portfolio selection. AIMS mathematics, 8(1), 1–28. https://doi.org/10.3934/math.2023001
  22. [22] Cao, J., & Wu, J. (2020). A conjugate gradient algorithm and its applications in image restoration. Applied numerical mathematics, 152, 243–252. https://doi.org/10.1016/j.apnum.2019.12.002
  23. [23] Yin, J., Jian, J., & Jiang, X. (2021). A generalized hybrid CGPM-based algorithm for solving large-scale convex constrained equations with applications to image restoration. Journal of computational and applied mathematics, 391, 113423. https://doi.org/10.1016/j.cam.2021.113423
  24. [24] Liu, P., Wu, X., Shao, H., Zhang, Y., & Cao, S. (2023). Three adaptive hybrid derivative-free projection methods for constrained monotone nonlinear equations and their applications. Numerical linear algebra with applications, 30(2), e2471. https://doi.org/10.1002/nla.2471
  25. [25] Yin, J., Jian, J., Jiang, X., Liu, M., & Wang, L. (2021). A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numerical algorithms, 88(1), 389–418. https://doi.org/10.1007/s11075-020-01043-z%0A%0A
  26. [26] Liu, Y., Zhu, Z., & Zhang, B. (2022). Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing. Journal of applied mathematics and computing, 68(3), 1787–1816. https://doi.org/10.1007/s12190-021-01589-8%0A%0A
  27. [27] Liu, P., Shao, H., Wang, Y., & Wu, X. (2022). A three-term CGPM-based algorithm without Lipschitz continuity for constrained nonlinear monotone equations with applications. Applied numerical mathematics, 175, 98–107. https://doi.org/10.1016/j.apnum.2022.02.001
  28. [28] Aminifard, Z., & Babaie-Kafaki, S. (2022). Dai--Liao extensions of a descent hybrid nonlinear conjugate gradient method with application in signal processing. Numerical algorithms, 89(3), 1369–1387. https://doi.org/10.1007/s11075-021-01157-y%0A%0A
  29. [29] Powell, M. J. (2006). Nonconvex minimization calculations and the conjugate gradient method. Numerical analysis: Proceedings of the 10th biennial conference held at dundee, scotland, June 28–July 1, 1983 (pp. 122-141). Berlin, Heidelberg: Springer Berlin Heidelberg. https://doi.org/10.1007/BFb0099521%0A%0A
  30. [30] Dai, Y. H, & Liao, L. Z. (2001). New conjugacy conditions and related nonlinear conjugate gradient methods. Applied mathematics & optimization, 43(1), 87–101. https://doi.org/10.1007/s002450010019%0A%0A
  31. [31] Gould, N. I. M., Orban, D., & Toint, P. L. (2003). CUTEr and SifDec: A constrained and unconstrained testing environment, revisited. ACM transactions on mathematical software (toms), 29(4), 373–394. https://doi.org/10.1145/962437.962439
  32. [32] Dolan, E. D., & Moré, J. J. (2002). Benchmarking optimization software with performance profiles. Mathematical programming, 91(2), 201–213. https://doi.org/10.1007/s101070100263%0A%0A
  33. [33] Hager, W. W., & Zhang, H. (2006). Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent. ACM transactions on mathematical software (TOMS), 32(1), 113-137. https://doi.org/10.1145/1132973.1132979
  34. [34] Aminifard, Z., & Babaie-Kafaki, S. (2019). An optimal parameter choice for the Dai-Liao family of conjugate gradient methods by avoiding a direction of the maximum magnification by the search direction matrix. 4OR, 17(3), 317–330. https://doi.org/10.1007/s10288-018-0387-1%0A%0A
  35. [35] Yuan, G., Li, T., & Hu, W. (2019). A conjugate gradient algorithm and its application in large-scale optimization problems and image restoration. Journal of inequalities and applications, 2019(1), 247. https://doi.org/10.1186/s13660-019-2192-6%0A%0A
  36. [36] Yuan, G., Meng, Z., & Li, Y. (2016). A modified Hestenes and Stiefel conjugate gradient algorithm for large-scale nonsmooth minimizations and nonlinear equations. Journal of optimization theory and applications, 168(1), 129–152. https://doi.org/10.1007/s10957-015-0781-1%0A%0A
  37. [37] Yuan, G., Li, T., & Hu, W. (2020). A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems. Applied numerical mathematics, 147, 129–141. https://doi.org/10.1016/j.apnum.2019.08.022
  38. [38] Yu, G., Huang, J., & Zhou, Y. (2010). A descent spectral conjugate gradient method for impulse noise removal. Applied mathematics letters, 23(5), 555–560. https://doi.org/10.1016/j.aml.2010.01.010
  39. [39] Esmaeili, H., Shabani, S., & Kimiaei, M. (2019). A new generalized shrinkage conjugate gradient method for sparse recovery. Calcolo, 56(1), 1. https://doi.org/10.1007/s10092-018-0296-x%0A%0A

Published

2025-05-23

Issue

Vol. 1 No. 2 (2025): Transactions on Soft Computing

Section

Articles

How to Cite

Ranjbar, M., & Ashrafi, A. (2025). A Spectral Hybrid Conjugate Gradient Approach for Large-Scale Unconstrained Optimization with Applications in Image Restoration. Transactions on Soft Computing , 1(2), 109-126. https://doi.org/10.48314/tsc.v1i2.45

Similar Articles

You may also start an advanced similarity search for this article.