Improvement of Barzilai and Borwein Gradient Method Based on Neutrosophic Logic System with Application in Image Restoration

Abstract

An upgrade to the quasi-Newton (QN) family of methods for solving unconstrained optimization problems is proposed. This research focuses on a detailed investigation of the Barzilai and Borwein (BB) gradient methods. The upgrade involves the use of neutrosophic logic to determine an additional parameter that will be incorporated into an appropriate step size for the BB iterations. Unlike previous research, which incorporated neutrosophic concepts into gradient methods by using only two objective-function values to calculate the input parameter during the neutrophication phase, this study determines the input parameter using three consecutive objective-function values. The main idea is to use appropriately defined membership functions to perform neutrosophication and de-neutrosophication. The set of if–then rules is based on two or more successive values of the objective function. This strategy also directly influences the design of the newly proposed method. Numerical comparisons demonstrate superior performance of the proposed methods with respect to Dolan–Moré performance profiles including the number of iterations, central processing unit (CPU) time, and number of function evaluations. Furthermore, experimental results confirm that the proposed algorithms can be effectively applied to image restoration tasks, particularly for image denoising, where they achieve competitive reconstruction quality and stable convergence behavior.

1. Introduction, Preliminaries, and Motivation

The current research topic focuses on solving the unconstrained optimization problem

$$min\mathfrak{f}\left(x\right),x\in {\mathbb{R}}^n,$$

(1)

where $\mathfrak{f}:{\mathbb{R}}^n\to \mathbb{R}$ is uniformly convex and a twice continuously differentiable objective function. The most common iterative rule used to solve the multivariable unconstrained minimization problem (1) is the family of line search methods

$$x_{k+1}=x_k+t_k{d}_k.$$

(2)

In this equation, $x_{k+1}$ represents a new point, $x_k$ is the previous point, $t_k>0$ is the step size, and $d_k$ is a search (descent) direction. In gradient descent (GD) methods, the direction $d_k$ is defined as $d_k=-g_k$, where $g_k=\nabla \mathfrak{f}\left(x_k\right)$ represents the gradient of f at the point $x_k$ [1].

The minimization problem outlined in (1) can be solved using the quasi-Newton (QN) method supported by line search

$$x_{k+1}=x_k+t_k{d}_k,$$

(3)

where the search direction is determined as

$$d_k=-H_k{g}_k,$$

(4)

under the assumption that $B_k$ is a positive definite symmetric approximation of the Hessian matrix $G_k={▽}^2\mathfrak{f}\left(x_k\right)$ and its inverse is denoted by $H_k=B_k^{-1}$ [2]. The matrix $H_k$ must be positive definite in order to satisfy the QN requirement

$$H_k{y}_{k-1}=s_{k-1},$$

(5)

where $s_{k-1}=x_k-x_{k-1}$ and $y_{k-1}=g_k-g_{k-1}$ [1].

The storage space of QN iterations is at least $O\left(n^2\right)$ and poses challenges when solving large-scale models. Brezinski in [3] classified methods for updating $B_k$ into three common categories: scalar approximations (where $B_k={\lambda }_{k}I$, with I being an appropriate identity matrix), diagonal matrices ($B_k=\mathrm{diag}\left({\lambda }_1,\dots ,{\lambda }_n\right)$), and proper full matrices. In this research, we utilize the scalar Hessian’s estimate

$$B_k={\lambda }_{k}I\approx G_k,{\lambda }_k>0.$$

(6)

Such an approach was investigated in [4,5,6,7,8]. The practical implications of using the scalar Hessian’s estimate for solving large-scale models is minimal storage requirements and a significant reduction in computational complexity when calculating iterations compared to other classes of methods for updating $B_k$.

Barzilai and Borwein, in their work [9], proposed the two-point step-size gradient method, known as the BB method. This method follows the iterative pattern

$$x_{k+1}=x_k-{\lambda }_{k}I{g}_k=x_k-{\lambda }_k{g}_k,$$

(7)

where I denotes the corresponding identity matrix. To comprise second-order information about the goal function, the step size ${\lambda }_k$ should approximately satisfy the subsequent least squares QN property

$$\underset{\lambda >0}{min}∥y_{k-1}-{\displaystyle \frac{1}{\lambda }}{s}_{k-1}∥,$$

(8)

and

$$\underset{\lambda >0}{min}\parallel \lambda y_{k-1}-s_{k-1}\parallel ,$$

(9)

where $s_{k-1}$ and $y_{k-1}$ are consistent with (5). The solutions to (8) and (9), respectively, are equal to

$${\lambda }_k^{BB1}={\displaystyle \frac{s_{k-1}^T{s}_{k-1}}{s_{k-1}^T{y}_{k-1}}},$$

(10)

and

$${\lambda }_k^{BB2}={\displaystyle \frac{s_{k-1}^{\mathrm{T}}{y}_{k-1}}{y_{k-1}^T{y}_{k-1}}}.$$

(11)

Additionally, under the condition $s_{k-1}^{\mathrm{T}}{y}_{k-1}>0$, the parameters defined in (10) and (11) consistently satisfy ${\lambda }_k^{BB1}\ge {\lambda }_k^{BB2}$.

Raydan [10] established that the BB algorithm is globally convergent when applied to a strictly convex objective $\mathfrak{f}\left(x\right)$. Additionally, Dai and Liao [11] showed that the BB algorithm achieves R-linear convergence. However, in scenarios where $\mathfrak{f}$ is continuously differentiable, the step sizes ${\lambda }_k^{BB1}$ and ${\lambda }_k^{BB2}$ in the BB algorithm can be harmful, and a monotonic decrease of $\mathfrak{f}$ is not guaranteed. To address this weakness, Raydan in [12] proposed a security strategy focused on step size and introduced a globally convergent and non-monotonic line search BB algorithm.

However, the classical BB method has an inherent limitation: its step sizes depend solely on two consecutive points, neglecting the broader trajectory of the optimization process. This limitation can lead to oscillatory behavior or inefficient step sizes. Its tri-valued structure models uncertainty in descent progress and enhances the adaptivity of the method. While several BB variants exist, including adaptive step-size, safe BB, and non-monotonic line search methods, our approach is novel in integrating neutrosophic logic to adjust the step size based on multiple previous iterations. This strategy provides a richer, tri-valued assessment of descent trends and improves convergence robustness.

The BB iterative rule has attracted considerable research interest and resulted in several significant outcomes [13,14,15,16,17]. The advantages of the BB algorithm stem from the fact that ${\lambda }_k^{BB1}$ and ${\lambda }_k^{BB2}$ are closely aligned with the QN condition.

Recent studies underscore the value of incorporating quasi-Newton and conjugacy information into gradient-based methods. For instance, ref. [18] introduced spectral hybrid conjugate gradient methods that combine memoryless BFGS directions with the Dai–Liao conjugacy condition, resulting in improved performance and robustness for large-scale unconstrained problems. Similarly, ref. [19] developed a unified conjugate gradient framework with rigorous convergence analysis, demonstrating effectiveness in image restoration applications. The authors of [20] defined a new quasi-Newton method based on the variational procedure aimed to refine the gradient and Hessian matrix approximations.

Fuzzy set (FS), intuitionistic fuzzy set (IFS), and neutrosophic set (NS) are practical algorithmic approaches for solving mathematical models characterized by unreliability, fuzziness, ambivalence, inaccuracy, insufficient confidence, variability, and excessiveness [21]. NSs have been used in denoising, segmentation, clustering, and classification in various image-processing applications. Several applications of neutrosophic systems were discussed in [22].

Definition 1 

([23]). Let $\mathcal{U}$ be a nonempty set with a generic element from $x\in \mathcal{U}$. Then, a FS $\mho \subseteq \mathcal{U}$ is a set of elements

$$\mho =\left\{\langle x,{\mu }_F\left(x\right)\rangle \right|x\in \mathcal{U}\},$$

(12)

in which the membership function (MF) ${\mu }_F\left(x\right)\in [0,1]$ signifies membership degree of the element x to ℧.

Definition 2 

([24]). An IFS $\mho \subseteq \mathcal{U}$ is defined by ordered triples

$$\mho =\left\{\langle x,{\mu }_F\left(x\right),{\nu }_F\left(x\right)\rangle \right|x\in \mathcal{U}\},$$

(13)

such that ${\mu }_F\left(x\right):\mathcal{U}\to [0,1]$ and ${\nu }_F\left(x\right):\mathcal{U}\to [0,1]$ indicate the degree of the membership and the degree of the non-membership of the element x to ℧, respectively, with the restriction $0\le {\mu }_F\left(x\right)+{\nu }_F\left(x\right)\le 1$.

Definition 3 

([25,26,27]). Let $\mathcal{U}$ be the universe of discourse. A single-valued NS $\mho \subseteq \mathcal{U}$ is an arbitrary entity of the pattern

$$\mho =\left\{\langle x,T_{\mathcal{N}}\left(x\right),I_{\mathcal{N}}\left(x\right),F_{\mathcal{N}}\left(x\right)\rangle \right|x\in \mathcal{U}\},$$

(14)

in which $T_{\mathcal{N}}\left(x\right),I_{\mathcal{N}}\left(x\right),F_{\mathcal{N}}\left(x\right):\mathcal{U}\to [0,1]$ are the truth MF, the indeterminacy MF, and the falsity MF, respectively, satisfying $0\le T_{\mathcal{N}}\left(x\right)+I_{\mathcal{N}}\left(x\right)+F_{\mathcal{N}}\left(x\right)\le 3$.

It is known that the determination of ${\lambda }_k^{BB1}$ and ${\lambda }_k^{BB2}$ relies solely on the information of the two points $x_k$ and $x_{k-1}$. Specifically, the quantities $s_{k-1}=x_k-x_{k-1}$ and $y_{k-1}=g_k-g_{k-1}$ are used.

Momentum has been verified as an efficient tool to enhance gradient descent (GD) by incorporating information from previous steps of the algorithm into the next step. Our objective is to implement gradient descent with momentum (GD+M) using fuzzy logic systems (FLSs) and neutrosophic logic systems (NLSs) to create adaptive self-corrective gradient-based optimization methods. These methods include gradient descent, conjugate gradient, and QN methods, as well as the BB gradient methods. The GD+M upgrade in our approach utilizes using adopted fuzzy logic systems and neutrosophic logic systems to determine a corrective parameter that will be used as an adaptive learning quantity in defining multiple step sizes to generate the final step length for solving unconstrained optimization problems. The main idea is to use appropriately defined membership functions to perform neutrosophication and denetrosophication. Appropriate if–then rules in the underlying logic system rely on two or more successive values of the objective function.

Inspired by the findings in [28,29], and research in [30,31] regarding FLSs, we designed multiple step sizes to generate the final step length for solving unconstrained optimization models. We aim to develop and evaluate an improvement in the BB algorithm by applying a suitable fuzzy logic (FL) system. We will refer to these modifications as the Fuzzy Barzilai and Borwein (FBB) algorithms. The composite step size will be generated based on various parameters, which will help determine the final step length in QN algorithms.

In Section 2, we introduce multiple step sizes that are used to determine the final step length for solving unconstrained optimization problems. This approach leads to developing an improved Barzilai and Borwein gradient method based on the neutrosophic logic (NL) system. Section 3 explores the convergence properties of the proposed method. Numerical experiments are conducted in Section 4, aiming to demonstrate the effectiveness of the proposed approach. In addition, Section 4.1 presents a practical application of the proposed methods in image restoration tasks, specifically for image denoising. Some final conclusions and directions for future research are given in Section 5.

2. BB Method Enhanced by the Neutrosophic Logic System

The following idea, originated in [30,31], defines fuzzy GD (FGD) iterations in the form

$$x_{k+1}=x_k+t_k{\nu }_k{d}_k,$$

(15)

where $d_k$ is a descent direction, ${\nu }_k$ is an appropriately defined fuzzy parameter, and the step size $t_k$ is computed using an inexact line search. In general, any if–then rule for ${\nu }_k$ should satisfy the specified conditions

$${\nu }_k\left\{\begin{array}{cc}>1,\hfill & \mathrm{if}\mathfrak{f}\left(x_k\right)>\mathfrak{f}\left(x_{k+1}\right),\hfill \\ <1,\hfill & \mathrm{if}\mathfrak{f}\left(x_k\right)<\mathfrak{f}\left(x_{k+1}\right),\hfill \\ =1,\hfill & \mathrm{if}\mathfrak{f}\left(x_k\right)=\mathfrak{f}\left(x_{k+1}\right).\hfill \end{array}\right.$$

(16)

Starting from general QN iterations

$$x_{k+1}=QN\left(x_k\right)=x_k-t_k{H}_k{g}_k,$$

we define the general fuzzy quasi-Newton (FQN) iterative scheme with line search as the following transformation $\Psi$:

$$x_{k+1}=\Psi \left(\left(QN\right)\left(x_k\right)\right)=\Psi (x_k-t_k{H}_k{g}_k)=x_k-{\nu }_k{t}_k{H}_k{g}_k.$$

(17)

As mentioned in Section 1, we aim to improve the BB iterative formula (7) by introducing additional parameters that influence the step size. Consequently, the FBB gradient method is defined using the function $\Psi$:

$$x_{k+1}=\Psi \left(\left(BB\right)\left(x_k\right)\right)=\Psi (x_k-t_k{\lambda }_k^{BB}{g}_k)=x_k-{\nu }_k{t}_k{\lambda }_k^{BB}{g}_k.$$

(18)

In this equation, ${\nu }_k$ signifies a suitably defined fuzzy parameter, and the ${\lambda }_k^{BB}$ parameter is calculated using (10) or (11). The step size $t_k$ is determined through the backtracking line search outlined in Algorithm 1, as referenced in [7,32].

Algorithm 1 Backtracking line search.

Input:  Goal function $\mathfrak{f}\left(x\right)$, a descent direction $d_k$ at the point $x_k$, and numbers $0<\delta <0.5$ and $\pi \in (0,1)$. 1: $\tau =1$. 2: While $\mathfrak{f}(x_k+\tau d_k)>\mathfrak{f}\left(x_k\right)+\delta \tau g_k^{\mathrm{T}}{d}_k$, take $\tau :=\tau \pi$. 3: Return $t_k=\tau$.

To apply the iterative scheme (18), we must define the fuzzy parameter ${\nu }_k$ following the constraints specified in (16). The basic idea is to introduce some kind of intelligence in the process of defining the total step length based on monitoring the behavior of the objective function in previous iterations. Defining the parameter ${\nu }_k$ involves two phases: Neutrosophication and de-neutrosophication.

(1)

Neutrosophication utilizes three MFs and transforms the input

$${\vartheta }_k:=\left\{\begin{array}{cc}{\displaystyle \frac{l_k}{|l_{k-1}|+|l_k|}},\hfill & \mathrm{if}\mathfrak{f}\left(x_{k-1}\right)\ne 0\mathrm{and}\mathfrak{f}\left(x_{k-2}\right)\ne 0,\hfill \\ 0.5,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(19)

where $l_k={\displaystyle \frac{\mathfrak{f}\left(x_k\right)}{\mathfrak{f}\left(x_{k-1}\right)}}$.

The rationale for using three function values in (19) is to capture the overall trend of the optimization process, rather than just the immediate step. The parameter ${\vartheta }_k$ measures the relative change in function decrease. For instance, if $\mathfrak{f}\left(x_{k-1}\right)$ is much smaller than $\mathfrak{f}\left(x_{k-2}\right)$, and $\mathfrak{f}\left(x_k\right)$ is further smaller compared to $\mathfrak{f}\left(x_{k-1}\right)$, then ${\vartheta }_k$ will reflect strong consistent descent. Conversely, if the function decrease slows down or reverses, ${\vartheta }_k$ will indicate this deterioration. This three-point evaluation provides richer information than the two-point BB formulas, enabling the neutrosophic system to make more informed adaptive corrections.

In Equation (19), we can see that the value of the parameter ${\vartheta }_k$ depends on three values of the objective function: $\mathfrak{f}\left(x_{k-2}\right)$, $\mathfrak{f}\left(x_{k-1}\right)$, and $\mathfrak{f}\left(x_k\right)$. This approach represents a significant difference in the phase of neutrosophication. In contrast, the determination of ${\vartheta }_k$ in references [30,31] relies on only two values of the objective function. Furthermore, the method used to calculate the parameter ${\vartheta }_k$ is also different.

We will examine values ${\nu }_k$ of the gain parameter, estimated in terms of three MFs that represent the percentage of truth, indeterminacy, and falsity.

The Gaussian function is widely used to model the level of indeterminacy in neutrosophic models, especially in fields such as image processing, optimization, classification, and decision-making. The usefulness of this function stems from its ability to smoothly and controllably describe localized phenomena that exhibit a maximum at a specific point and gradually decay with increasing distance from that point. The Gaussian function is ideal for representing regions where values change gradually and where there is overlap among the T, I, and F components, thereby enabling a more realistic handling of uncertainty than functions with abrupt transitions. In neutrosophic systems and related fields, the sigmoid function is commonly used to model membership transitions, soft thresholds, and probabilities, thereby enabling gradual, rather than abrupt, changes between states.

The truth MF is modeled by the sigmoid function

$$\mathrm{T}\left(\vartheta \right)=1/(1+{\mathrm{e}}^{-c_1(\vartheta -c_2)}).$$

(20)

The parameter $c_1$ defines the slope at the crossover point $\vartheta =c_2$. The falsity MF is the sigmoid function

$$\mathrm{F}\left(\vartheta \right)=1/(1+{\mathrm{e}}^{c_1(\vartheta -c_2)}).$$

(21)

The indeterminacy MF is modeled by the Gaussian function

$$\mathrm{I}\left(\vartheta \right)={\displaystyle \frac{1}{\sigma \sqrt{2\pi }}}{\mathrm{e}}^{-{\displaystyle \frac{{(\vartheta -c_2)}^2}{2{\sigma }^2}}}.$$

(22)

The variable $\sigma \in \mathbb{R}$ represents the standard deviation, while $c_2\in \mathbb{R}$ denotes the mean. The value $c_2$ represents the center of the Gaussian function, determining the point at which it attains its maximum value. Changing the value of $c_2$ shifts the curve horizontally without altering its shape. The parameter $\sigma$, on the other hand, controls how rapidly the function decays from its maximum. In other words, $\sigma$ controls the “width of the curve”: a smaller $\sigma$ produces a narrower and sharper curve with a higher peak, while a larger $\sigma$ yields a wider and flatter curve with a lower peak.

Based on these definitions, the neutrosophication of $\vartheta \in \mathbb{R}$ involves transforming a single real quantity $\vartheta$ into the ordered triple $\langle T\left(\vartheta \right),I\left(\vartheta \right),F\left(\vartheta \right)\rangle$. The MFs for this transformation are defined in (20)–(22).

The parameters $c_1$, $c_2$, and $\sigma$ control the shape of the membership functions. Specifically, $c_1$ determines the slope of the sigmoid functions $T\left(\vartheta \right)$ and $F\left(\vartheta \right)$ at their crossover point $\vartheta =c_2$. The parameter $c_2$ determines the intersection point of the $T\left(\vartheta \right)$ and $F\left(\vartheta \right)$ functions and also affects the peak location of $I\left(\vartheta \right)$. On the other hand, $\sigma$ controls the width of the Gaussian indeterminacy function $I\left(\vartheta \right)$.

This paper investigates two combinations for the parameters’ configurations:

$\left({\complement }_1\right)$: $c_1=0.2$, $c_2=0.5,\sigma ={\displaystyle \frac{1}{\sqrt{2\pi }}}$ and $\left({\complement }_2\right)$: $c_1=1$, $c_2=2$, and $\sigma =1$.

Graphs of $\mathrm{T}\left(\vartheta \right),\mathrm{I}\left(\vartheta \right),\mathrm{F}\left(\vartheta \right)$ under the configuration $\left({\complement }_1\right)$ in (20)–(22) are plotted in Figure 1a for $\vartheta \in [-8,8]$. Graphs of $\mathrm{T}\left(\vartheta \right),\mathrm{I}\left(\vartheta \right),\mathrm{F}\left(\vartheta \right)$ for the configuration $\left({\complement }_2\right)$ in (20)–(22) are plotted in Figure 1b for $\vartheta \in [-8,8]$. Figure 1a depicts gradual sigmoid transitions centered at $\vartheta =0.5$ with a narrow indeterminacy peak reaching around $1.0$. Figure 1b exhibits steeper transitions at $\vartheta =2$ and a broader, flatter indeterminacy function with a peak around $0.4$).

To minimize $\mathfrak{f}\left(x\right)$, employing (19) as a measure in the neutrosophic logic controller (NLC), we will take into account the dynamic neutrosophic set (DNS) defined as $\mathfrak{D}:=\left\{\langle \mathrm{T}\left({\vartheta }_k\right),\mathrm{I}\left({\vartheta }_k\right),\mathrm{F}\left({\vartheta }_k\right)\rangle ;{\vartheta }_k\in \mathbb{R}\right\}$ over the real numbers $\mathbb{R}$.

(2)

De-neutrosophication is defined as transformation $\langle {\vartheta }_k:T\left({\vartheta }_k\right),I\left({\vartheta }_k\right),F\left({\vartheta }_k\right)\rangle \to {\nu }_k\left({\vartheta }_k\right)\in \mathbb{R}$ resulting in a single-value $\nu (\vartheta ,t)$. The following de-neutrosophication is adopted to acquire the parameter ${\nu }_k\left({\vartheta }_k\right)$:

$${\nu }_k\left({\vartheta }_k\right)=\left\{\begin{array}{cc}3-(T\left({\vartheta }_k\right)+I\left({\vartheta }_k\right)+F\left({\vartheta }_k\right))\equiv \Delta \left({\vartheta }_k\right),\hfill & \mathfrak{f}\left(x_k\right)<\mathfrak{f}\left(x_{k-1}\right),\hfill \\ {\displaystyle \frac{1}{3-(T\left({\vartheta }_k\right)+I\left({\vartheta }_k\right)+F\left({\vartheta }_k\right))}}\equiv {\displaystyle \frac{1}{\Delta \left({\vartheta }_k\right)}},\hfill & \mathfrak{f}\left(x_k\right)>\mathfrak{f}\left(x_{k-1}\right),\hfill \\ 1,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(23)

The fulfillment of essential constraint ${\nu }_k\left({\vartheta }_k\right)\ge 0$ follows directly from (23) and the fact that all MFs are with codomains in $[0,1]$. Also, ${\nu }_k$ should satisy ${\nu }_k<1$ if $\mathfrak{f}\left(x_k\right)>\mathfrak{f}\left(x_{k-1}\right)$ and ${\nu }_k>1$ if $\mathfrak{f}\left(x_k\right)<\mathfrak{f}\left(x_{k-1}\right)$ and ${\nu }_k=1$ if $\mathfrak{f}\left(x_k\right)=\mathfrak{f}\left(x_{k-1}\right)$.

Graphs of $\Delta \left({\vartheta }_k\right)$ and ${\displaystyle \frac{1}{\Delta \left({\vartheta }_k\right)}}$ for the environment $\left({\complement }_1\right)$ are presented in Figure 2a, while Figure 2b shows the graphs of $\Delta \left({\vartheta }_k\right)$ and ${\displaystyle \frac{1}{\Delta \left({\vartheta }_k\right)}}$ for the configuration $\left({\complement }_2\right)$ in the interval $\vartheta \in [-8,8]$. The purple (respectively, green) graphs in both Figure 2a,b illustrate $\Delta \left({\vartheta }_k\right)$ (respectively, ${\displaystyle \frac{1}{\Delta \left({\vartheta }_k\right)}}$). Clearly, the minimum of $\Delta \left({\vartheta }_k\right)$ and maximum of ${\displaystyle \frac{1}{\Delta \left({\vartheta }_k\right)}}$ are achieved at the point ${\vartheta }_k=c_2$. Reasoning upon (16) and (23), the minimal increase from $t_k$ to $t_k{\nu }_k$ in FGD iterations (15) in the case $\mathfrak{f}\left(x_k\right)>\mathfrak{f}\left(x_{k+1}\right)$ is achieved for ${\theta }_k=c_2$. On the other hand, the minimal decrease from $t_k$ to $t_k{\nu }_k$ in (15) in the case $\mathfrak{f}\left(x_k\right)<\mathfrak{f}\left(x_{k+1}\right)$ is achieved for ${\theta }_k=c_2$. Also, greater values $\sigma$ lead to smaller fluctuations in values $\Delta \left({\vartheta }_k\right)$ and ${\displaystyle \frac{1}{\Delta \left({\vartheta }_k\right)}}$. It is logical to adjust the parameters so that the influence of neutrosophy weakens during iterations and stops near the minimum.

Following this tendency, the parameter’s setting for performed numerical experiments is defined as in $\left({\complement }_1\right)$. This choice is motivated by the convergence behavior of the input variable ${\vartheta }_k$ defined in (19). As the algorithm approaches the minimum, ${\vartheta }_k$ converges toward $0.5$, since $l_k={\displaystyle \frac{\mathfrak{f}\left(x_k\right)}{\mathfrak{f}\left(x_{k-1}\right)}}\to 1$. Setting $c_2=0.5$ aligns the crossover point with this behavior, ensuring balanced membership values near the optimum. Choosing $c_1=0.2$ provides gradual transitions, preventing sharp changes in ${\nu }_k$ that could destabilize convergence. The parameter $\sigma ={\displaystyle \frac{1}{\sqrt{2\pi }}}$ yields a sharp Gaussian peak reaching approximate unity at $\vartheta =c_2$, concentrating high indeterminacy in the crossover region. Preliminary tests confirmed stable convergence with this parameter’s setting.

The bounds $1\le \Delta \left({\vartheta }_k\right)\le 2$ and $0.5\le {\displaystyle \frac{1}{\Delta \left({\vartheta }_k\right)}}\le 1$ are perceptible in Figure 2. Based on this observation, the following estimate applies for the parameter ${\nu }_k$ in (23):

$$0.5\le {\nu }_k\le 2.$$

(24)

Algorithm 2 outlines the general BB method utilizing the neutrosophic logic system (FBB method).

Algorithm 2 FBB method.

Input:  A real function $\mathfrak{f}\left(x\right)$ and selected starting point $x_0\in dom\left(\mathfrak{f}\right)$. 1: Set $k=0$, initialize ${\lambda }_0=1$, ${\nu }_0={\nu }_1=1$, $c_1=0.2$, $c_2=0.5$, $\sigma ={\displaystyle \frac{1}{\sqrt{2\pi }}}$, and calculate $\mathfrak{f}\left(x_0\right)$ and $g_0=\nabla \mathfrak{f}\left(x_0\right)$. 2: (Backtracking) Determine $t_0\in (0,1]$ as the output of Algorithm 1. 3: Compute $x_1$ using (18). 4: Calculate $g_1=\nabla \mathfrak{f}\left(x_1\right)$, $\mathfrak{f}\left(x_1\right)$ and set $k:=1$. 5: If test criteria are fulfilled, stop; otherwise, go to the next step. 6: Calculate $s_{k-1}$, $y_{k-1}$, and ${\lambda }_k$ using (10) or (11). 7: (Backtracking) Determine $t_k\in (0,1]$ as the output of Algorithm 1. 8: Compute $x_{k+1}$ using (18). 9: Calculate $g_{k+1}=\nabla \mathfrak{f}\left(x_{k+1}\right)$ and $\mathfrak{f}\left(x_{k+1}\right)$. 10: Compute ${\vartheta }_{k+1}$ according to (19). 11: Compute $\mathrm{T}\left({\vartheta }_{k+1}\right),\mathrm{I}\left({\vartheta }_{k+1}\right),\mathrm{F}\left({\vartheta }_{k+1}\right)$ using (20)–(22). 12: Compute ${\nu }_{k+1}\left({\vartheta }_{k+1}\right)$ utilizing (23). 13: Set $k:=k+1$ and go to Step 5. 14: Return $\{x_{k+1},\mathfrak{f}\left(x_{k+1}\right)\}$.

Inspired by the previous application of the neutrosophic logic system in Algorithm 2, we want to go one step further. More precisely, we want to show that this neutrosophic logic system can be implemented in other variants of the BB method. Specifically, we want to show the application of the neutrosophic logic system in a hybrid BB-type method for solving large-scale unconstrained optimization problems presented by Gao and Ou in the paper [33]. In the paper [33], the authors based Algorithm HBB on the combination of the BB method with the idea of the IMPBOT Algorithm proposed by Brown and Biggs in [34]. Below, we present a modified version of the HBB Algorithm based on the DNS, that is, fuzzy hybrid BB-type (FHBB) algorithms.

The notation FBB1 refers to the FBB method in which ${\lambda }_k$ is defined in Step 6 of Algorithm 2, utilizing (10). In the dual case, FBB2 refers to the FBB method that determines ${\lambda }_k$ using (11) in Step 6 of Algorithm 2. For the FHBB method, there are also two variants: FHBB1 refers to the version that determines ${\lambda }_k$ in Step 4 of Algorithm 3 using (25), while FHBB2 refers to the variant that determines ${\lambda }_k$ in Step 4 using (26).

Algorithm 3 FHBB method.

Input:  A real function $\mathfrak{f}\left(x\right)$ and selected starting point $x_0\in dom\left(\mathfrak{f}\right)$. 1: Set $k=0$, calculate $\mathfrak{f}\left(x_0\right)$, $g_0=\nabla \mathfrak{f}\left(x_0\right)$, initialize $h_0=10$, ${\nu }_0=1$, $c_1=0.2$, $c_2=0.5$, $\sigma ={\displaystyle \frac{1}{\sqrt{2\pi }}}$, $\rho \in (0,1)$, $0<{\lambda }_{min}<{\lambda }_{max}<+\infty$, ${\theta }_k\in [0,1)$, and $R_0={\mathfrak{f}}_0$. 2: If test criteria are fulfilled, stop; otherwise, go to the next step. 3: If $k=0$, determine $t_0\in (0,1]$ as the output of Algorithm 1, where $d_0=-g_0$, and set $x_1=x_0+t_0{d}_0$, $s_{-1}:=s_0$, $y_{-1}:=y_0$; otherwise, go to Step 4. 4: Calculate ${\lambda }_k$ using $${\lambda }_k^{HBB1}={\displaystyle \frac{s_{k-1}^{\mathrm{T}}{\overline{y}}_{k-1}}{{\overline{y}}_{k-1}^T{\overline{y}}_{k-1}}}$$ (25) or $${\lambda }_k^{HBB2}={\displaystyle \frac{s_{k-1}^T{s}_{k-1}}{s_{k-1}^T{\overline{y}}_{k-1}}},$$ (26) where ${\overline{y}}_{k-1}={\tilde{y}}_{k-1}+{\displaystyle \frac{s_{k-1}}{h_k}}$, ${\tilde{y}}_{k-1}=y_{k-1}+{\tilde{\vartheta }}_k{s}_{k-1}$, ${\tilde{\vartheta }}_k={\displaystyle \frac{{(g_{k-1}+g_k)}^T{s}_{k-1}+2({\mathfrak{f}}_{k-1}-{\mathfrak{f}}_k)}{\parallel s_{k-1}{\parallel }^2}}$.      If ${\lambda }_k<0$, set ${\lambda }_k={\lambda }_{max}$; otherwise, set ${\lambda }_k=min\{{\lambda }_{max},max\{{\lambda }_{min},{\lambda }_k\}\}$. 5: Compute $d_k=-{\displaystyle \frac{{\lambda }_k{h}_k}{{\lambda }_k+h_k}}{g}_k$. 6: If the condition $\mathfrak{f}(x_k+d_k)\le R_k+\rho g_k^T{d}_k$, where $$R_k={\theta }_k{R}_{k-1}+(1-{\theta }_k){\mathfrak{f}}_k,$$ $${\theta }_{k+1}=\left\{\begin{array}{ll}{\displaystyle \frac{{\theta }_k}{2}},& \mathrm{if}k=0,\\ {\displaystyle \frac{{\theta }_k+{\theta }_{k-1}}{2}},& \mathrm{if}k\ge 1\end{array}\right.$$ holds, set $x_{k+1}=x_k+{\nu }_k{d}_k$ and $h_{k+1}=2h_k$, and go to Step 7; otherwise, find the step size $t_k$ along the direction $d_k$ by using the modified backtracking line search rule $\mathfrak{f}(x_k+t_k{d}_k)\le R_k+\delta t_k{g}_k^T{d}_k$ and set $x_{k+1}=x_k+{\nu }_k{t}_k{d}_k$ and $h_{k+1}={\displaystyle \frac{1}{2}}{h}_k$. 7: Calculate $g_{k+1}=\nabla \mathfrak{f}\left(x_{k+1}\right)$, $\mathfrak{f}\left(x_{k+1}\right)$, $s_k$, and $y_k$. 8: Compute ${\vartheta }_{k+1}$ according to (19). 9: Compute $\mathrm{T}\left({\vartheta }_{k+1}\right),\mathrm{I}\left({\vartheta }_{k+1}\right),\mathrm{F}\left({\vartheta }_{k+1}\right)$ using (20)–(22). 10: Compute ${\nu }_{k+1}\left({\vartheta }_{k+1}\right)$ utilizing (23). 11: Set $k:=k+1$ and go to Step 2. 12: Return $\{x_{k+1},\mathfrak{f}\left(x_{k+1}\right)\}$.

3. Convergence Analysis

This section focuses on the convergence properties of the FBB method outlined in Algorithm 2. To establish global convergence results, we require the following assumptions on the objective function $\mathfrak{f}\left(u\right)$.

Assumption 1.  $\left(A1\right)$ The level set $\mathcal{M}=\{u\in {\mathbb{R}}^n|\mathfrak{f}\left(u\right)\le \mathfrak{f}\left(u_0\right)\}$ is bounded, where $u_0$ of (2) is defined in the context of the initial iterate.

$\left(A2\right)$ The objective function $\mathfrak{f}\left(u\right)$ is continuously differentiable in a neighborhood $\mathcal{N}$ inside $\mathcal{M}$. Furthermore, the gradient g is Lipschitz continuous, meaning the existence of a constant $L>0$, such that the following condition holds:

$$\parallel g\left(u\right)-g\left(v\right)\parallel \le L\parallel u-v\parallel ,\forall u,v\in \mathcal{N}.$$

(27)

Additionally, Assumption 1 implies the existence of constants $C>0$ and $\kappa >0$ that satisfy

$$\parallel u-v\parallel \le C,\forall u,v\in \mathcal{N}$$

(28)

and

$$\parallel g\left(u\right)\parallel \le \kappa ,\forall u\in \mathcal{N}.$$

(29)

A key aspect of demonstrating convergence is the inequality

$$s_{k-1}^T{y}_{k-1}\ge \chi {\parallel s_{k-1}\parallel }^2,$$

(30)

where $\chi >0$ is the strong convexity constant. This inequality is a direct consequence of the uniform (strong) convexity of the objective function and follows from ([2], Theorem 1.3.16). It should be emphasized that inequality (30) is not part of Assumption 1 but requires the additional assumption that $\mathfrak{f}$ is uniformly convex, as stated in Proposition 2. From (27) and (30), one can conclude

$$\chi \parallel s_{k-1}{\parallel }^2\le s_{k-1}^T{y}_{k-1}\le L{\parallel s_{k-1}\parallel }^2,$$

(31)

and consequently, $\chi \le L$.

Moreover, several auxiliary results from [32,35,36,37,38] are employed in the subsequent analysis.

Proposition 1 

([32,38]). Let $d_k$ be selected as a descent direction, and the gradient $g\left(u\right)=\nabla \mathfrak{f}\left(u\right)$ satisfies the Lipschitz condition (27). Then, the step length $t_k$ derived from the backtracking line search Algorithm 1 satisfies

$$t_k\ge min\left\{1,-{\displaystyle \frac{\pi (1-\delta )}{L}}{\displaystyle \frac{g_k^{\mathrm{T}}{d}_k}{\parallel d_k{\parallel }^2}}\right\}.$$

(32)

The proofs of Proposition 2 and Lemma 1 can be found in [36,37]. For the sake of completeness, we nevertheless present the proof of the following proposition.

Proposition 2. 

If the objective function $\mathfrak{f}:{\mathbb{R}}^n\to \mathbb{R}$ is twice continuously differentiable and uniformly convex on ${\mathbb{R}}^n$, then Assumption 1 is satisfied.

Proof. 

Since $\mathfrak{f}$ is uniformly convex and twice continuously differentiable on ${\mathbb{R}}^n$, there exists $m>0$ such that

$$\mathfrak{f}\left(u\right)\ge \mathfrak{f}\left(u^{*}\right)+{\displaystyle \frac{m}{2}}{\parallel u-u^{*}\parallel }^2,\forall u\in {\mathbb{R}}^n,$$

where $u^{*}$ is the unique minimizer of $\mathfrak{f}$. Hence, for any initial point $u_0$, the level set

$$\mathcal{M}=\{u\in {\mathbb{R}}^n\mid \mathfrak{f}\left(u\right)\le \mathfrak{f}\left(u_0\right)\}$$

satisfies

$$\mathcal{M}\subseteq \left\{u\in {\mathbb{R}}^n\mid {\parallel u-u^{*}\parallel }^2\le {\displaystyle \frac{2(\mathfrak{f}\left(u_0\right)-\mathfrak{f}\left(u^{*}\right))}{m}}\right\},$$

which is a bounded set. This proves Assumption 1 (A1).

Moreover, since $\mathfrak{f}\in C^2\left({\mathbb{R}}^n\right)$ and $\mathcal{M}$ is bounded, the gradient $\nabla \mathfrak{f}$ is Lipschitz continuous on any closed neighborhood $\mathcal{N}$ containing $\mathcal{M}$, which establishes Assumption 1 (A2). □

Lemma 1. 

Under the assumptions of Proposition 2, there exist real numbers m, M, satisfying

$$0<m\le 1\le M,$$

(33)

such that the function $\mathfrak{f}\left(u\right)$ has a unique minimizer $u^{*}$ and

$$\begin{array}{c}\hfill {m\parallel v\parallel }^2\le v^{\mathrm{T}}{\nabla }^2\mathfrak{f}\left(u\right)v\le M{\parallel v\parallel }^2,\forall u,v\in {\mathbb{R}}^n;\end{array}$$

(34)

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{2}}m\parallel u-u^{*}{\parallel }^2\le \mathfrak{f}\left(u\right)-\mathfrak{f}\left(u^{*}\right)\le {\displaystyle \frac{1}{2}}M{\parallel u-u^{*}\parallel }^2,\forall u\in {\mathbb{R}}^n;\end{array}$$

(35)

$$\begin{array}{c}\hfill {m\parallel u-v\parallel }^2\le {(g\left(u\right)-g\left(v\right))}^{\mathrm{T}}(u-v)\le M{\parallel u-v\parallel }^2,\forall u,v\in {\mathbb{R}}^n.\end{array}$$

(36)

Theorem 1 examines the convergence of the $FBB$ iterative sequence.

Theorem 1. 

Assuming that the conditions $\left(A1\right)$ and $\left(A2\right)$ are satisfied, and that $\mathfrak{f}:{\mathbb{R}}^n\to \mathbb{R}$ is a uniformly convex function, the $FBB$ sequence induced by (18) satisfies

$$\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x_{k+1}^{FBB}\right)\ge {\mu }_{{\nu }_k}{\parallel g_k\parallel }^2,$$

(37)

such that

$${\mu }_{{\nu }_k}=min\left\{{\displaystyle \frac{\delta {\nu }_k\chi }{L^2}},{\displaystyle \frac{\delta (1-\delta )}{L}}\pi \right\}.$$

(38)

Proof. 

The update from the $FBB$ iteration is given by $x_{k+1}^{FBB}=x_k^{FBB}-{\nu }_k{t}_k{\lambda }_k^{BB}{g}_k$, as presented in Equation (18). This follows the general pattern outlined in (2), where $d_k=-{\nu }_k{\lambda }_k^{BB}{g}_k$. From the stopping criterion used in Algorithm 1, it follows

$$\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x_{k+1}^{FBB}\right)\ge -\delta t_k{g}_k^{\mathrm{T}}{d}_k,\forall k\in \mathbb{N}.$$

(39)

In the situation $t_k<1$, using (39) with $d_k=-{\nu }_k{\lambda }_k^{BB}{g}_k$, one obtains

$$\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x_{k+1}^{FBB}\right)\ge -\delta t_k{g}_k^{\mathrm{T}}{d}_k=-\delta t_k{g}_k^{\mathrm{T}}\left(-{\nu }_k{\lambda }_k^{BB}{g}_k\right).$$

(40)

Equation (32) indicates that

$$\begin{array}{ll}{t}_k\ge -{\displaystyle \frac{\pi (1-\delta )}{L}}·{\displaystyle \frac{g_k^{\mathrm{T}}{d}_k}{\parallel d_k{\parallel }^2}}& =-{\displaystyle \frac{\pi (1-\delta )}{L}}·{\displaystyle \frac{g_k^{\mathrm{T}}\left(-{\nu }_k{\lambda }_k^{BB}{g}_k\right)}{{∥-{\nu }_k{\lambda }_k^{BB}{g}_k∥}^2}}\\ & ={\displaystyle \frac{\pi (1-\delta )}{L}}·{\displaystyle \frac{{\nu }_k{\lambda }_k^{BB}{\parallel g_k\parallel }^2}{{\nu }_k^2{\left({\lambda }_k^{BB}\right)}^2{\parallel g_k\parallel }^2}}\\ & ={\displaystyle \frac{\pi (1-\delta )}{L}}·{\displaystyle \frac{1}{{\nu }_k{\lambda }_k^{BB}}}.\end{array}$$

Now, combining (40) with the previous inequality, we obtain

$$\begin{array}{ll}\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x_{k+1}^{FBB}\right)& \ge \delta t_k{\nu }_k{\lambda }_k^{BB}\parallel g_k{\parallel }^2\\ & \ge \delta {\displaystyle \frac{\pi (1-\delta )}{L}}·{\displaystyle \frac{1}{{\nu }_k{\lambda }_k^{BB}}}{\nu }_k{\lambda }_k^{BB}\parallel g_k{\parallel }^2\\ & \ge \delta {\displaystyle \frac{(1-\delta )\pi }{L}}{\parallel g_k\parallel }^2.\end{array}$$

(41)

In the case $t_k=1$, using the fact $s_{k-1}^{\mathrm{T}}{y}_{k-1}>0$, ${\lambda }_k^{BB1}\ge {\lambda }_k^{BB2}$ from [39], (27), and (31), we have

$${\lambda }_k^{BB1}\ge {\lambda }_k^{BB2}={\displaystyle \frac{s_{k-1}^T{y}_{k-1}}{y_{k-1}^T{y}_{k-1}}}={\displaystyle \frac{s_{k-1}^T{y}_{k-1}}{\parallel y_{k-1}{\parallel }^2}}\ge {\displaystyle \frac{\chi \parallel s_{k-1}{\parallel }^2}{\parallel y_{k-1}{\parallel }^2}}\ge {\displaystyle \frac{\chi \parallel s_{k-1}{\parallel }^2}{L^2{\parallel s_{k-1}\parallel }^2}}={\displaystyle \frac{\chi }{L^2}}.$$

(42)

From Equations (39) and (42), we derive

$$\begin{array}{ll}\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x_{k+1}^{FBB}\right)\ge & -\delta g_k^{\mathrm{T}}{d}_k=-\delta g_k^{\mathrm{T}}(-{\nu }_k{\lambda }_k^{BB}{g}_k)\\ & =\delta {\nu }_k{\lambda }_k^{BB}\parallel g_k{\parallel }^2\\ & \ge {\displaystyle \frac{\delta {\nu }_k\chi }{L^2}}{\parallel g_k\parallel }^2.\end{array}$$

(43)

Finally, from Equations (41) and (43), we obtain

$$\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x_{k+1}^{FBB}\right)\ge min\left\{{\displaystyle \frac{\delta {\nu }_k\chi }{L^2}},{\displaystyle \frac{\delta (1-\delta )}{L}}\pi \right\}{\parallel g_k\parallel }^2,$$

which confirms (37) and (38). □

In Theorem 2, we demonstrate the linear convergence of the $FBB$ method on uniformly convex functions.

Theorem 2. 

If the objective function $\mathfrak{f}:{\mathbb{R}}^n\to \mathbb{R}$ is twice continuously differentiable and uniformly convex on ${\mathbb{R}}^n$, then the series $\left\{x_k^{FBB}\right\}$ generated by Algorithm 2 fulfills

$$\underset{k\to \infty }{lim}\parallel g_k^{FBB}\parallel =0.$$

(44)

This implies that the sequence $\left\{x_k^{FBB}\right\}$ converges to $x^{*}$ at least linearly.

Proof. 

According to Lemma 1, we know that the objective function $\mathfrak{f}$ is bounded below and that $\mathfrak{f}\left(x_k^{FBB}\right)$ decreases. Therefore, it is evident that

$$\underset{k\to \infty }{lim}\left(\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x_{k+1}^{FBB}\right)\right)=0.$$

By applying estimations (37) and (38) from Theorem 1, we obtain

$$\underset{k\to \infty }{lim}\parallel g_k^{FBB}\parallel =0.$$

We need to prove that the $\underset{k\to \infty }{lim}\parallel x_k^{FBB}-x^{*}\parallel =0$. In other words, we must demonstrate that the sequence $\left\{x_k^{FBB}\right\}$ defined by (18) converges to $x^{*}$.

We substitute $x^{*}=v$ into the inequality (36) and apply the mean value theorem along with the Cauchy–Schwartz inequality to draw conclusions

$$m\parallel u-x^{*}{\parallel }^2\le \parallel g\left(u\right)\parallel \le M\parallel u-x^{*}{\parallel }^2,\forall u\in {\mathbb{R}}^n.$$

(45)

Estimation (45) and the inequality (35) lead to

$${\mu }_{{\nu }_k}\parallel g\left(x_k^{FBB}\right){\parallel }^2\ge {\mu }_{{\nu }_k}{m}^2{\parallel x_k^{FBB}-x^{*}\parallel }^2\ge 2{\mu }_{{\nu }_k}{\displaystyle \frac{m^2}{M}}\left(\mathfrak{f}\left(x_k^{FBB}\right)-\mathfrak{f}\left(x^{*}\right)\right){\to }_{k\to \infty }0.$$

As a result, it follows that $\underset{k\to \infty }{lim}\parallel x_k^{FBB}-x^{*}\parallel =0$, which indicates the convergence of the sequence $\left\{x_k^{FBB}\right\}$ to $x^{*}$.

To finalize the proof, we must establish that the sequence $\left\{x_k^{FBB}\right\}$ is generated by Algorithm 2, linearly convergent under the condition $L\ge M\ge \sqrt{2}$, where L defined by (27) in Assumption 1, and M is specified by (33) in Lemma 1.

To demonstrate the linear convergence, it is essential to show

$$\rho =\sqrt{2{\mu }_{{\nu }_k}{\displaystyle \frac{m^2}{M}}}<1.$$

Since the parameter ${\mu }_{{\nu }_k}$ in (38) depends on two values, there are two cases.

In the case where ${\mu }_{{\nu }_k}={\displaystyle \frac{\delta {\nu }_k\chi }{L^2}}$, using $0.5\le {\nu }_k\le 2$ in (24) and $\chi \le L$ in (31), we derive

$${\rho }^2=2{\mu }_{{\nu }_k}{\displaystyle \frac{m^2}{M}}=2{\displaystyle \frac{\delta {\nu }_k\chi }{L^2}}{\displaystyle \frac{m^2}{M}}\le 2{\displaystyle \frac{\delta {\nu }_{k}L}{L^2}}{\displaystyle \frac{m^2}{M}}=2{\displaystyle \frac{\delta {\nu }_k}{L}}{\displaystyle \frac{m^2}{M}}\le 2\delta <1.$$

Similarly, in the case ${\mu }_{{\nu }_k}={\displaystyle \frac{\delta (1-\delta )}{L}}\pi$, it follows that

$${\rho }^2=2{\mu }_{{\nu }_k}{\displaystyle \frac{m^2}{M}}=2\pi {\displaystyle \frac{\delta (1-\delta )}{L}}{\displaystyle \frac{m^2}{M}}<{\displaystyle \frac{m^2}{LM}}<1,$$

because from (33) and assumptions $L\ge M\ge \sqrt{2}$, inequality $m\le M\le L$ holds.

Applying Theorem 4.1 from [38], we get

$$\parallel x_k^{FBB}-x^{*}\parallel \le \sqrt{{\displaystyle \frac{2\left(\mathfrak{f}\left(x_0^{FBB}\right)-\mathfrak{f}\left(x^{*}\right)\right)}{m}}}{\sqrt{1-{\rho }^2}}^k,$$

thus completing the proof. □

4. Numerical Experience

We will evaluate the numerical efficiency of the proposed FBB and FHBB methods.

Seven methods will be compared using standard test functions, with given initial points sourced from [40,41], involving FBB1, FBB2, FHBB1, and FHBB2 and well-known HBB1 and HBB2 methods from [33] and the FMSM method from [30]. The comparison will focus on the following three criteria: the number of iterative steps (NIs), the number of function evaluations (NFEs), and the CPU time in seconds. Ten test problems are selected for evaluation with problem dimensions set at [100, 500, 1000, 2000, 3000, 5000, 7000, 8000, 10,000, 15,000]. The tests will be conducted in MATLAB R2017a and a LAP’s (Intel(R) Core(TM) i3-7020U, up to 2.30 GHz, 8 GB Memory) with the Windows 10 Education operating system.

All algorithms use the stopping criterion

$${\displaystyle \frac{|\mathfrak{f}\left(x_{k+1}\right)-\mathfrak{f}\left(x_k\right)|}{1+\left|\mathfrak{f}\right(x_k\left)\right|}}\le \zeta \mathrm{and}\parallel g_k\parallel \le ϵ,$$

(46)

where $\zeta ={10}^{-16}$ and $ϵ={10}^{-6}$. The stopping criteria (46) are chosen to ensure that the algorithms terminate only when the iterates are sufficiently close to a stationary point. These strict tolerances help prevent premature termination and enable a reliable comparison of convergence behavior among different variants of the method. Algorithms FBB1, FBB2, and FMSM are tested using backtracking line search with parameters $\delta =0.0001$ and $\pi =0.8$, while algorithms HBB1, HBB2, FHBB1, and FHBB2 are tested using backtracking line search and the modified backtracking line search with parameters $\delta =0.01$ and $\pi =0.5$. The neutrosophic logic system parameters for the FBB1, FBB2, FHBB1, and FHBB2 methods are defined as ($c_1=0.2$, $c_2=0.5, \sigma ={\displaystyle \frac{1}{\sqrt{2\pi }}}\approx 0.398942280$), while the parameters used in the HBB1, HBB2, and FMSM algorithms are set to be the same as those in the related paper [30,33].

A comprehensive sensitivity analysis on the choice of $c_1,c_2,\sigma$ is planned as future research. All numerical experiments in current research employ the primary setting $\left({\complement }_1\right)$.

A summary of numerical results from the competition between FBB1, FBB2, HBB1, HBB2, FHBB1, FHBB2, and FMSM is presented after testing 27 test functions (270 tests) by monitoring criteria such as NI, NFE, and CPU time, as arranged in

Table 1. Summary test results of FBB1, FBB2, HBB1, HBB2, FHBB1, FHBB2, and FMSM methods for the NI.

Test Function	No. of Iterations
	FBB1	FBB2	HBB1	HBB2	FHBB1	FHBB2	FMSM
Extended Penalty	460	463	459	448	431	438	373
Perturbed Quadratic	57,888	10,846	7776	13,943	7523	14,474	58,008
Raydan 2	65	65	69	69	60	60	87
Diagonal 2	32,020	5607	4684	8575	4312	9283	20,764
Hager	789	663	710	671	677	704	743
Extended Tridiagonal 1	9015	426	232	457	152	158	4233
Extended TET	111	80	100	161	110	107	237
Diagonal 5	40	40	60	60	51	51	105
Extended Himmelblau	257	238	138	136	241	257	376
Perturbed quadratic diagonal	40,954	6127	5520	15,521	6659	17,943	40,791
Extended quadratic penalty QP2	141	78	704	753	998	1759	2077
Extended quadratic exponential EP1	40	40	34	34	33	33	92
Extended Tridiagonal 2	427	394	384	400	373	363	441
ENGVAL1 (CUTE)	303	300	319	325	289	299	302
QUARTC (CUTE)	192	192	166	166	10	10	216
Diagonal 6	120	120	69	69	91	91	87
Generalized Quartic	203	191	116	115	171	169	148
Diagonal 7	60	60	60	60	50	50	114
Diagonal 8	60	60	50	50	50	50	81
Extended BD1(Block Diagonal)	320	267	150	167	277	282	206
Extended Cliff	736	299	505	1426	445	2215	17,963
NONDIA (CUTE)	222,837	2663	584	284	823	1445	378,685
DQDRTIC (CUTE)	1860	838	461	567	871	1579	687
Extended Freudenstein and Roth	5486	319	322	925	365	653	19,612
Extended White and Holst	12,512	660	760	876	913	2874	8552
Extended Beale	1567	457	310	435	522	1037	905
EDENSCH (CUTE)	252	256	291	300	240	244	313

,

Table 2. Summary test results of FBB1, FBB2, HBB1, HBB2, FHBB1, FHBB2, and FMSM methods for the NFE.

Test Function	No. of Funct. Evaluation
	FBB1	FBB2	HBB1	HBB2	FHBB1	FHBB2	FMSM
Extended Penalty	2064	2259	1716	1522	1479	1914	2320
Perturbed Quadratic	324,714	24,657	16,465	49,688	15,940	51,578	334,615
Raydan 2	160	160	168	168	150	150	231
Diagonal 2	175,307	13,455	11,655	31,228	10,840	33,441	119,819
Hager	3131	1775	2262	2181	1853	2425	3129
Extended Tridiagonal 1	81,588	1862	541	1634	354	366	38,232
Extended TET	422	310	260	422	284	274	662
Diagonal 5	110	110	150	150	132	132	299
Extended Himmelblau	704	695	356	352	562	594	1119
Perturbed quadratic diagonal	417,437	18,056	12,586	64,416	15,286	74,392	407,013
Extended quadratic penalty QP2	3436	2274	2646	3307	3437	7059	14,051
Extended quadratic exponential EP1	404	404	178	178	176	176	579
Extended Tridiagonal 2	2155	1689	1756	1841	1566	1515	2070
ENGVAL1 (CUTE)	2153	2316	1500	1469	1211	1412	2942
QUARTC (CUTE)	454	454	382	382	70	70	492
Diagonal 6	270	270	168	168	212	212	334
Generalized Quartic	558	519	312	310	402	398	466
Diagonal 7	160	160	160	160	140	140	570
Diagonal 8	190	190	150	150	150	150	272
Extended BD1 (Block Diagonal)	760	664	390	424	584	614	700
Extended Cliff	9796	4090	2598	9319	2091	14,190	146,314
NONDIA (CUTE)	2,736,952	11,099	3124	1666	3724	8259	4,555,897
DQDRTIC (CUTE)	10,192	2689	1126	1772	2022	5322	3130
Extended Freudenstein and Roth	39,918	2307	1337	3619	1506	3810	77,132
Extended White and Holst	109,712	3148	2160	3736	2434	12,667	64,083
Extended Beale	8441	2024	790	1370	1374	3374	4433
EDENSCH (CUTE)	888	880	920	966	771	716	1207

and

Table 3. Summary test results of FBB1, FBB2, HBB1, HBB2, FHBB1, FHBB2, and FMSM methods for the CPU time.

Test Function	CPU Time (s)
	FBB1	FBB2	HBB1	HBB2	FHBB1	FHBB2	FMSM
Extended Penalty	1.156	1.391	1.563	0.984	0.938	1.641	1.297
Perturbed Quadratic	181.328	18.359	11.000	27.938	11.359	29.703	184.438
Raydan 2	0.266	0.141	0.313	0.250	0.109	0.141	0.250
Diagonal 2	238.391	17.906	14.359	35.594	12.672	35.031	140.297
Hager	8.453	4.906	4.938	4.719	4.094	5.688	6.813
Extended Tridiagonal 1	240.219	5.250	2.766	4.469	1.172	2.000	81.359
Extended TET	0.766	0.406	0.438	0.734	0.406	0.516	1.000
Diagonal 5	0.391	0.328	0.500	0.500	0.313	0.438	1.047
Extended Himmelblau	0.438	0.328	0.203	0.281	0.156	0.359	0.563
Perturbed quadratic diagonal	182.313	10.750	8.563	31.922	9.859	37.109	181.406
Extended quadratic penalty QP2	2.281	1.234	2.438	2.313	2.734	5.250	9.000
Extended quadratic exponential EP1	0.422	0.219	0.188	0.281	0.172	0.344	0.578
Extended Tridiagonal 2	1.234	0.766	0.969	1.031	1.016	0.922	1.219
ENGVAL1 (CUTE)	1.125	1.031	1.031	0.969	0.734	0.953	1.516
QUARTC (CUTE)	3.078	2.688	1.875	2.063	0.125	0.266	2.391
Diagonal 6	0.438	0.219	0.281	0.250	0.266	0.344	0.313
Generalized Quartic	0.469	0.406	0.219	0.266	0.203	0.344	0.359
Diagonal 7	0.297	0.203	0.359	0.281	0.188	0.266	0.734
Diagonal 8	0.438	0.234	0.281	0.250	0.203	0.281	0.500
Extended BD1 (Block Diagonal)	0.813	0.750	0.438	0.531	0.594	0.656	0.609
Extended Cliff	4.469	1.594	2.063	4.859	1.328	4.906	77.969
NONDIA (CUTE)	664.313	5.297	1.734	0.828	1.813	3.156	1723.219
DQDRTIC (CUTE)	3.859	1.094	0.875	1.125	1.047	2.172	1.578
Extended Freudenstein and Roth	9.031	0.672	0.813	1.453	0.688	1.234	23.578
Extended White and Holst	209.938	8.359	5.906	8.719	6.391	26.531	188.328
Extended Beale	25.438	6.875	3.250	4.859	4.594	10.094	14.141
EDENSCH (CUTE)	4.359	3.781	3.891	3.938	3.094	3.594	5.094

.

An analysis of the results presented in Table 1, Table 2 and Table 3 reveals that the FBB1, FBB2, HBB1, HBB2, FHBB1, FHBB2, and FMSM methods successfully solved all 27 test problems. Furthermore, a closer examination of these results in Table 1, Table 2 and Table 3 highlights the performance of each method in terms of NI, NFE, and CPU time. It was found that the FHBB1 method solved 18.52% (5 out of 27) of all test functions in the conducted experiments with the fewest iterations compared to FBB1, FBB2, HBB1, HBB2, FHBB1, and FMSM, which resolved only 0% (0 out of 27), 22.22% (6 out of 27), 14.81% (4 out of 27), 11.11% (3 out of 27), 3.70% (1 out of 27), and 3.70% (1 out of 27), respectively. Additionally, it is noteworthy that 14.81% of the problems (4 out of 27) were solved with the same minimum number of iterations by both the FHBB1 and FHBB2 methods, and 3.70% of the problems (1 out of 27) were solved with the same minimum number of iterations using four methods (HBB1, HBB2, FHBB1, FHBB2). In total, the FHBB1 method solved 37.04% of the problems (10 out of 27) with the minimum number of iterations.

The result analysis shows that the FHBB1 method solved 22.22% (6 out of 27) of all problems with the minimum number of function evaluations compared to the FBB1, FBB2, HBB1, HBB2, FHBB2, and FMSM methods, which recorded 0% (0 out of 27), 7.41% (2 out of 27), 25.93% (7 out of 27), 11.11% (3 out of 27), 7.41% (2 out of 27), and 0% (0 out of 27), respectively. Furthermore, Table 2 shows that 14.81% (4 out of 27) of the problems were solved with an equal least number of function evaluations by the two (FHBB1 and FHBB2) methods in the experiments, and 3.70% of the problems (1 out of 27) were solved an equal least number of function evaluations using four (HBB1, HBB2, FHBB1, FHBB2) methods. All in total the FHBB1 method solved 40.74% of the problems (11 out of 27) with the minimum number of function evaluations.

The summarized results in Table 3 indicate that the FHBB1 method successfully solved 55.56% of the problems (15 out of 27) in the shortest CPU time. In contrast, the FBB1, FBB2, HBB1, HBB2, FHBB2, and FMSM methods are solved 0% (0 out of 27), 14.81% (4 out of 27), 22.22% (6 out of 27), 3.70% (1 out of 27), 0% (0 out of 27), and 0% (0 out of 27), respectively. One problem, that is, 3.70% of the problems was solved in the shortest CPU time using the two (FBB2 and FHBB1) methods. In total, the FHBB1 method solved 59.26% of the problems (16 out of 27) with the minimum processing time.

The performance profiles presented in [42] are utilized to compare considered methods for the criteria NFE, CPU time, and NI in relation to FBB1, FBB2, HBB1, HBB2, FHBB1, FHBB2, and FMSM methods. Interpreting these profiles is simple: a higher curve signifies better algorithmic performance; the method corresponding to the upper performance profile graph is considered the winner [42].

In Figure 3 and Figure 4, we compare the performance profiles of NI, NFE, and CPU time for the FBB1, FBB2, HBB1, HBB2, FHBB1, FHBB2, and FMSM methods based on quantities provided in Table 1, Table 2 and Table 3. From Figure 3a,b and Figure 4, it is obvious that the FHBB1 graph reaches the value of one first, indicating that FHBB1 outperforms FBB1, FBB2, HBB1, HBB2, FHBB2, and FMSM methods with respect to NI, NFE, and CPU time.

The numerical results indicate that the BB gradient methods based on the dynamic neutrosophic decision achieve better results. While the neutrosophic approach introduces additional calculations for determining the input parameters based on three consecutive function values, the overall increase in computational cost is minimal compared to the BB method. Our numerical experiments show that the extra operations do not significantly affect the total runtime, as the dominant cost remains the function evaluations. This leads to the conclusion that incorporating dynamic neutrosophic sets into gradient methods substantially enhances numerical outcomes.

4.1. Image Restoration via FBB and FHBB Methods

Image restoration is a fundamental problem in image processing, where the goal is to reconstruct an unknown clean image $x\in {\mathbb{R}}^{n\times n}$ from a noisy observation b. In the additive Gaussian noise model, we have

$$b=x+\eta ,\eta \sim \mathcal{N}(0,{\sigma }^2),$$

(47)

where $\eta$ represents white Gaussian noise with variance ${\sigma }^2$.

Over the years, a wide range of denoising approaches has been proposed. Classical filtering methods, such as Gaussian smoothing or median filtering, are computationally efficient but tend to oversmooth edges and remove fine details. More advanced statistical and transform-based methods include wavelet thresholding, sparse coding, and non-local means (NLMs), which exploit self-similarity in images to improve restoration quality. More recently, learning-based approaches, especially deep neural networks, have achieved state-of-the-art performance, but they require large amounts of training data and often lack interpretability and stability guarantees. Deep neural network denoisers such as DnCNN often achieve excellent quantitative performance by learning complex nonlinear mappings from large training datasets. However, these models require substantial training effort and GPU resources, and their performance may degrade when the noise statistics differ from the training data. In contrast, variational TV-based methods provide transparent control of regularization and stable behavior across noise levels without training. For these reasons, and given the optimization-focused scope of this work, we restrict our comparisons to classical variational methods while acknowledging the strong empirical results of DnCNN-type approaches.

In contrast to these data-driven approaches, variational methods provide a principled framework for image denoising by explicitly formulating an optimization problem that balances data fidelity and regularization. A widely adopted model is based on total variation (TV) regularization, introduced by Rudin, Osher, and Fatemi in [43]. The TV regularizer promotes piecewise smooth reconstructions while preserving important edge structures, which are crucial in natural images. Unlike quadratic smoothness priors, TV regularization avoids excessive blurring across edges, making it particularly effective for images with sharp transitions.

A common variational formulation for image denoising is given by

$$\underset{x}{min}{\displaystyle \frac{1}{2}}{\parallel x-b\parallel }_2^2+\lambda TV\left(x\right),$$

(48)

where $TV\left(x\right)$ denotes the total variation of the image, and $\lambda >0$ is a regularization parameter that balances data fidelity and smoothness. The TV term is defined as

$$TV\left(x\right)=\sum _{i,j}\sqrt{{(x_{i+1,j}-x_{i,j})}^2+{(x_{i,j+1}-x_{i,j})}^2}.$$

(49)

In summary, the TV-based denoising framework provides a robust and interpretable mechanism for image restoration. It effectively preserves important edge structures by penalizing oscillatory components while allowing sharp discontinuities, which is crucial for maintaining natural image details. At the same time, the regularization efficiently suppresses Gaussian noise without introducing pronounced artifacts, ensuring visually smooth yet accurate reconstructions. Furthermore, TV-based models are grounded in solid mathematical principles, offering interpretability and analytical transparency that data-driven approaches often lack. Finally, the same variational formulation exhibits strong generality and can be naturally extended to a wide range of inverse imaging problems, including deblurring, inpainting, and compressed sensing.

4.1.1. Application to Image Denoising

The denoising problem (48) can be solved by applying proposed FBB/FHBB to the objective function

$$f\left(x\right)={\displaystyle \frac{1}{2}}{\parallel x-b\parallel }_2^2+\lambda TV\left(x\right).$$

(50)

At each iteration, the gradient is computed as

$$\nabla f\left(x\right)=(x-b)+\lambda \nabla TV\left(x\right),$$

(51)

where $\nabla TV\left(x\right)$ is obtained from the divergence of normalized gradients.

In practice, we will demonstrate the effectiveness of these algorithms on standard test images, such as Barbara, baboon, and other benchmark datasets. Different levels of Gaussian noise $\sigma \in \{0.03,0.05,0.08,0.1\dots \}$ and a range of regularization parameters $\lambda$ will be considered.

4.1.2. Comparative Analysis

We first introduce two variants of the FBB method, which differ in how the step-size parameter ${\nu }_k$ is applied during the iterations. Global FBB computes a single scalar value of ${\nu }_k$ for the entire image. This approach is simple and computationally efficient, providing stable convergence and consistent restoration quality, but it may oversmooth fine details, especially in regions with high noise or complex textures. Pixel-wise FBB, in contrast, computes a spatially varying step size ${\nu }_k(i,j)$, allowing the algorithm to adapt to the local properties of each pixel. This enhances edge and texture preservation, while maintaining fine structures, at the cost of higher computational complexity and potentially more complex stability behavior.

In the experiments, we compare these two variants along with the previously mentioned FHBB method. In the FHBB algorithm, a slight modification was made by omitting h from the descent step, using $d=-\lambda g$ for improved stability and consistency with the implementation. We also evaluate the influence of the two standard BB step-size rules (10) and (11), assessing their impact on convergence speed, numerical stability, and restoration quality within both global and pixel-wise settings.

Performance is assessed using PSNR and SSIM as quality metrics, along with the number of iterations and CPU time measured across multiple runs and noise realizations. This evaluation provides a comprehensive view of the trade-offs between global and pixel-wise strategies, the effect of BB step-size variants, and the overall performance of all three algorithms in denoising tasks.

Before proceeding to detailed performance comparisons, we first analyze the relationship between the parameters $\sigma$ (noise level) and $\lambda$ (regularization weight). The goal is to identify suitable $\lambda$ values for each level of noise, such that the reconstructed images achieve optimal visual and quantitative quality across all algorithms. We evaluate three methods: FBB Pixel, FBB Global, and FHBB. Initial experiments indicate that the optimal values of $\lambda$ are typically close to the corresponding noise level $\sigma$, especially for smaller $\sigma$. Based on this observation, for each $\sigma$, we tested three to four nearby values of $\lambda$ (slightly smaller, equal, and slightly larger). All methods were tested on standard benchmark images—Barbara, baboon, pepper, boats, and others. In addition to the basic stopping criterion based on a maximum number of iterations, an additional condition was included to ensure convergence stability, defined as

$${\displaystyle \frac{\parallel x_k-x_{k-1}\parallel }{\parallel x_{k-1}\parallel }}<{10}^{-3}\mathrm{and}{\displaystyle \frac{|f_k-f_{k-1}|}{|f_{k-1}|}}<{10}^{-3},$$

where $x_k$ and $f_k$ denote the current iterate and objective-function value, respectively. The detailed numerical results are summarized in

Table 4. Quantitative results for the FBB Pixel, Global, and Hybrid methods across different images and parameter settings. For each fixed noise level $\sigma$, multiple values of $\lambda$ were tested, and the best-performing entries within each $\sigma$ block (in terms of PSNR, SSIM, average iterations (Iter), and time) are highlighted in bold. Horizontal lines separate result groups corresponding to different $\sigma$ values.

Image	Max	$\mathit{\sigma }$	$\mathit{\lambda }$	FBB Pixel				FBB Global				FHBB
	Iter			PSNR	SSIM	Iter	Time	PSNR	SSIM	Iter	Time	PSNR	SSIM	Iter	Time
			0.01	33.57	0.843	6.7	1.22	33.64	0.845	6.3	0.88	33.66	0.845	10.7	1.31
pepper	100	0.03	0.03	33.91	0.852	9.3	2.08	33.85	0.85	8.3	1.46	33.67	0.847	16.7	2.33
			0.05	33.31	0.839	9.7	2.13	33.12	0.834	9.3	1.6	32.7	0.816	20.3	2.9
			0.03	31.80	0.789	32.0	7.02	31.80	0.789	30.0	5.23	31.93	0.795	22.0	3.03
pepper	100	0.05	0.05	32.01	0.801	45.3	9.72	31.96	0.8	41.7	6.59	31.95	0.798	36.7	4.33
			0.07	31.76	0.797	42.7	8.72	31.65	0.795	38.3	5.62	31.38	0.786	56.3	6.54
			0.06	28.23	0.779	47	9.92	28.22	0.777	44.3	7.08	28.22	0.779	28	3.26
barbara	100	0.08	0.08	27.99	0.784	55.3	9.42	27.97	0.784	61.3	9.58	27.89	0.781	59.3	8.69
			0.10	27.71	0.778	64.0	13.62	27.66	0.777	65.0	9.50	27.55	0.774	87.7	10.87
			0.08	27.33	0.741	52.0	10.91	27.33	0.741	54.3	8.27	27.33	0.743	42.0	4.71
barbara	100	0.10	0.10	27.27	0.751	58.7	10.16	27.27	0.750	47	6.21	27.18	0.751	74.7	7.82
			0.12	27.13	0.752	68.3	12.14	27.12	0.752	59.3	7.98	26.95	0.748	89.3	9.39
			0.08	23.26	0.677	38	7.31	23.24	0.677	40.7	5.60	23.22	0.675	20.7	2.42
baboon	100	0.12	0.10	22.96	0.652	55.3	9.46	22.96	0.654	54.0	8.21	22.89	0.648	27.0	4.48
			0.12	22.66	0.626	59.0	12.57	22.63	0.625	62.0	9.61	22.51	0.614	52.7	6.87
			0.14	22.39	0.600	64.3	13.74	22.35	0.599	63.0	8.95	22.17	0.581	86.3	9.32
			0.08	22.25	0.624	26.7	5.39	22.24	0.624	26	4.10	22.29	0.627	19.3	2.65
baboon	100	0.15	0.10	22.85	0.622	46.0	8.07	22.36	0.623	47.7	6.52	22.36	0.621	22.3	2.68
			0.12	22.26	0.607	59.0	10.24	22.25	0.608	62.3	8.33	22.21	0.603	29.3	3.24

.

Based on Table 4, we further analyze the optimal selection of the regularization parameter $\lambda$ for each fixed noise level $\sigma$. For every block of results corresponding to a specific $\sigma$, several values of $\lambda$ were tested, and the quantitative metrics (PSNR, SSIM, number of iterations, and computational time) were recorded for all three algorithms. Within each block, the best-performing entries—highlighted in bold—represent the most favorable trade-off between denoising quality and computational efficiency for that particular $\sigma$.

By examining the highlighted results for each fixed noise level $\sigma$, a nearly linear relationship between $\sigma$ and the optimal regularization parameter $\lambda$ can be observed. For lower noise levels, the best results are typically obtained when $\lambda \approx \sigma$, while for higher $\sigma$ values, the optimal $\lambda$ tends to stabilize around $0.08$–$0.10$. This behavior is consistent with the expected characteristics of TV-based regularization, where stronger noise requires slightly higher regularization to balance smoothness and detail preservation.

Optimal Parameter Pairs. From the highlighted results in Table 4, the following $(\sigma ,\lambda )$ pairs were identified as empirically optimal across the tested images and algorithms:

$$(\sigma ,\lambda )\in \left\{\right(0.03,0.03),(0.05,0.05),(0.08,0.06),(0.1,0.08),(0.12,0.08),(0.15,0.08\left)\right\}.$$

These values will be used in the subsequent analysis to further investigate the convergence behavior and robustness of the proposed methods under different noise conditions.

After identifying the optimal $(\sigma ,\lambda )$ pairs, we proceed to compare the three FBB methods under a fixed number of iterations. The motivation for this analysis stems from the observation that all three methods eventually reach a similar target accuracy and saturate in terms of restoration quality; the primary difference lies in the number of iterations required to achieve that accuracy. For instance, one method may reach near-optimal performance in 30 iterations, while another may require 100 iterations. Clearly, the method reaching the desired quality in fewer iterations is more efficient.

To investigate this, we fix the number of iterations at several predefined values, namely 10, 20, 30, 50, and 80, and compare the performance of all three methods. For this initial comparison, we use the same step-size rule ($B{B}_1$) for all methods; the effect of $B{B}_2$ will be examined in subsequent experiments. For each selected $(\sigma ,\lambda )$ pair, two random images are chosen from our benchmark set, and the algorithms are run for the specified iteration counts. Performance is evaluated using standard metrics such as PSNR, SSIM, and CPU time, allowing a direct comparison of efficiency and restoration quality across the different methods (see

Table 5. Quantitative comparison of $FB{B}_1$ Pixel, $FB{B}_1$ Global, and $FHB{B}_1$ methods for fixed iteration counts under the $B{B}_1$ step-size rule. Blocks correspond to different $(\sigma ,\lambda )$ settings, with best-performing values highlighted in bold.

Parameters				${\mathit{FBB}}_1$ Pixel			${\mathit{FBB}}_1$ Global			${\mathit{FHBB}}_1$ Hybrid
Image	Iter	$\mathbf{\sigma }$	$\mathbf{\lambda }$	PSNR	SSIM	Time	PSNR	SSIM	Time	PSNR	SSIM	Time
	10			34.90	0.909	3.3	34.75	0.905	2.32	34.71	0.905	1.94
	20			34.87	0.91	6.90	34.74	0.905	4.65	34.69	0.905	4.28
boats	30	0.03	0.03	34.83	0.909	10.98	34.70	0.905	8.58	34.67	0.904	6.63
	50			34.72	0.906	19.05	34.63	0.904	11.42	34.63	0.904	10.16
	10			28.63	0.883	1.76	28.53	0.881	1.38	28.42	0.875	1.13
	20	0.03	0.03	28.55	0.879	4.33	28.48	0.878	3.42	28.39	0.873	2.68
baboon	30			28.51	0.877	6.82	28.45	0.876	5.57	28.38	0.872	4.67
	50			28.44	0.874	10.8	28.4	0.872	7.02	28.36	0.871	5.86
	10			26.07	0.81	2.71	25.99	0.807	2.48	25.82	0.796	1.54
	20			25.98	0.803	5.46	25.93	0.802	3.62	25.77	0.791	2.95
baboon	30	0.05	0.05	25.93	0.8	7.56	25.89	0.799	4.38	25.76	0.79	3.49
	50			25.86	0.795	12.18	25.83	0.794	8.61	25.73	0.788	7.14
	10			29.81	0.841	1.81	29.78	0.843	1.46	29.61	0.84	1.2
	20			29.75	0.843	3.74	29.72	0.844	2.88	29.57	0.84	2.48
barbara	30	0.05	0.05	29.72	0.844	6.56	29.7	0.844	5.06	29.56	0.84	3.94
	50			29.65	0.843	10.2	29.64	0.843	8.14	29.54	0.839	6.41
	10			29.7	0.715	1.77	29.64	0.714	1.58	29.88	0.729	1.51
	20			29.87	0.727	4.08	29.78	0.723	2.95	29.98	0.736	2.55
pepper	30	0.08	0.06	29.94	0.731	5.84	29.85	0.728	4.59	30	0.737	3.85
	50			30.03	0.738	8.87	29.95	0.734	6.93	30.04	0.74	6.49
	10			28.13	0.759	1.84	28.14	0.761	1.4	28.2	0.772	1.12
	20			28.19	0.77	3.4	28.18	0.769	2.87	28.22	0.778	2.42
barbara	30	0.08	0.06	28.21	0.774	5.51	28.2	0.773	5.26	28.22	0.78	5.8
	50			28.23	0.78	10.59	28.22	0.779	7.62	28.23	0.782	6.19
	10			28.39	0.699	3.57	28.27	0.698	2.55	28.48	0.719	2.06
	20			28.57	0.718	6.54	28.41	0.712	4.84	28.58	0.731	4.1
boats	30	0.1	0.08	28.63	0.725	10.82	28.46	0.719	6.99	28.6	0.734	5.57
	50			28.69	0.734	14.69	28.54	0.729	11.49	28.63	0.739	9.12
	10			27.81	0.748	2.12	27.77	0.750	1.66	27.95	0.767	1.38
	20			27.93	0.765	3.27	27.86	0.762	2.50	27.99	0.776	2.27
yacht	30	0.1	0.08	27.97	0.770	4.89	27.89	0.767	3.76	28.0	0.778	3.25
	50			28.01	0.777	10.13	27.94	0.775	7.77	28.02	0.782	6.29
	10			27.16	0.599	2.22	27.09	0.598	1.72	27.47	0.624	1.43
	20			27.42	0.619	4.28	27.29	0.613	3.3	27.62	0.636	2.71
pepper	30	0.12	0.08	27.51	0.625	5.28	27.38	0.619	4.1	27.65	0.639	3.26
	50			27.63	0.635	8.52	27.51	0.629	6.53	27.71	0.644	5.38
	10			26.85	0.616	3.61	26.73	0.613	2.41	27.07	0.64	2.3
	20			27.06	0.635	6.39	26.88	0.626	5.77	27.18	0.651	3.99
boats	30	0.12	0.08	27.12	0.641	8.77	26.95	0.632	6.71	27.2	0.654	5.31
	50			27.21	0.65	14.39	27.06	0.642	12.29	27.25	0.659	8.84
	10			24.93	0.485	2.1	24.9	0.486	1.49	25.26	0.51	1.32
	20			25.14	0.501	3.43	25.06	0.497	2.66	25.38	0.519	2.2
pepper	30	0.15	0.08	25.22	0.506	5.11	25.13	0.502	3.92	25.41	0.521	3.19
	50			25.33	0.514	8.71	25.25	0.51	6.7	25.46	0.525	5.73
	80			25.43	0.521	12.27	25.39	0.52	9.85	25.52	0.529	8.33
	10			24.24	0.541	2.21	24.28	0.543	1.71	24.55	0.562	1.49
	20			24.41	0.554	3.4	24.4	0.552	2.65	24.63	0.57	2.44
barbara	30	0.15	0.08	24.47	0.557	5.1	24.45	0.557	4.06	24.65	0.572	3.42
	50			24.55	0.565	8.56	24.54	0.563	6.55	24.69	0.575	5.25
	80			24.66	0.573	15.78	24.64	0.572	10.98	24.73	0.578	8.97

).

Based on the results presented in Table 5, it can be clearly observed that for smaller $(\sigma ,\lambda )$ pairs, the first algorithm FBB Pixel consistently outperforms the others. For each row in the table, the best-performing values among the three algorithms are highlighted in bold, considering PSNR, SSIM, and Time as evaluation criteria. It is evident that for $\sigma \le 0.05$, the FBB Pixel method achieves superior reconstruction quality, reflected in higher PSNR and SSIM values. Conversely, for higher noise levels and, consequently, larger regularization weights $\lambda$, the third algorithm FHBB demonstrates the best overall performance across all evaluated metrics (PSNR, SSIM, and Time). Indeed, most of the bolded results in the table correspond to this method, indicating its robustness and efficiency under more challenging noise conditions. Hence, additional statistical testing is unnecessary, as the superiority of the FHBB algorithm is clearly evident from the presented data.

Another noteworthy observation is that the FBB Global variant generally achieves shorter computational times compared to FBB Pixel, whereas the latter attains higher reconstruction accuracy in terms of PSNR and SSIM. This outcome is consistent with expectations, as the pixel-wise strategy enables finer local adaptation to spatial variations in noise and image structure, albeit at a higher computational cost.

Furthermore, when the same experiments are repeated using the alternative $B{B}_2$ step-size rule, similar trends are observed across all tested noise levels. The relative performance ranking among the three algorithms remains stable, with FHBB maintaining its leading position. To maintain clarity and avoid unnecessary redundancy, the detailed $B{B}_2$ tables are omitted here. Instead, the subsequent section provides a focused comparison of the FHBB and FBB Pixel algorithms under both $B{B}_1$ and $B{B}_2$ formulations, aiming to identify which configuration offers the most favorable balance between convergence rate, reconstruction accuracy, and computational efficiency.

4.1.3. Comparative Analysis of $B{B}_1$ and $B{B}_2$ Step-Size Rules for FHBB and FBB Pixel Variants

Since the FHBB and FBB Pixel variants exhibited the most favorable overall performance—the former for smaller values of $\sigma$ and the latter for larger ones—we further compare each of these two algorithms under different selections of the BB parameter, specifically $B{B}_1$ and $B{B}_2$, given by Formulas (10) and (11).

Table 6. Quantitative results for $FHB{B}_1$ and $FHB{B}_2$. Bold values indicate the better result for each metric (PSNR and SSIM) between the two methods.

Parameters				${\mathit{FHBB}}_1$		${\mathit{FHBB}}_2$
Image	Iter	$\mathbf{\sigma }$	$\mathbf{\lambda }$	PSNR	SSIM	PSNR	SSIM
	10			26.45	0.818	26.42	0.818
motobikes	20	0.08	0.06	26.44	0.818	26.4	0.816
	30			26.43	0.818	26.39	0.816
	10			28.2	0.772	28.18	0.771
barbara	20	0.08	0.06	28.22	0.778	28.22	0.784
	30			28.22	0.78	28.22	0.785
	10			28.48	0.719	28.42	0.714
boats	20	0.1	0.08	28.58	0.731	28.64	0.746
	30			28.6	0.734	28.67	0.75
	10			27.9	0.766	27.82	0.76
yacht	20	0.1	0.08	27.96	0.776	27.96	0.785
	30			27.97	0.779	27.96	0.788
	20			27.62	0.636	27.84	0.655
pepper	30	0.12	0.08	27.65	0.639	27.89	0.659
	50			27.71	0.644	27.93	0.662
	20			27.15	0.648	27.33	0.669
boats	30	0.12	0.08	27.18	0.651	27.36	0.672
	50			27.22	0.656	27.39	0.676
	20			24.81	0.594	24.95	0.606
yacht	30	0.15	0.08	24.83	0.596	24.98	0.609
	50			24.86	0.598	25	0.611
	80			24.9	0.602	25.03	0.613
	20			23.3	0.672	23.34	0.675
motorbikes	30	0.15	0.08	23.3	0.673	23.35	0.676
	50			23.31	0.673	23.35	0.676
	80			23.33	0.674	23.36	0.676

summarizes the comparative performance of the FHBB algorithm for the two BB configurations, whereas

Table 7. Quantitative results for $FB{B}_1$ Pixel and $FB{B}_2$ Pixel. Bold values indicate the better result for each metric (PSNR and SSIM) between the two methods.

Parameters				${\mathit{FBB}}_1$ Pixel		${\mathit{FBB}}_2$ Pixel
Image	Iter	$\mathbf{\sigma }$	$\mathbf{\lambda }$	PSNR	SSIM	PSNR	SSIM
	5			32.11	0.888	32.25	0.895
barbara	10	0.03	0.03	32.12	0.901	32.24	0.902
	20			32.05	0.901	32.17	0.902
	5			33.69	0.843	33.87	0.851
pepper	10	0.03	0.03	33.85	0.849	33.91	0.852
	20			33.79	0.845	33.87	0.849
	10			31.44	0.881	31.42	0.872
yacht	20	0.05	0.05	31.41	0.884	31.45	0.879
	30			31.37	0.885	31.45	0.883
	10			32.32	0.854	32.19	0.839
boats	20	0.05	0.05	32.34	0.859	32.28	0.848
	30			32.32	0.861	32.32	0.854
	10			26.55	0.819	26.53	0.814
motorbikes	20	0.08	0.06	26.54	0.821	26.55	0.817
	30			26.53	0.82	26.56	0.819
	10			28.13	0.759	27.95	0.736
barbara	20	0.08	0.06	28.19	0.77	28.04	0.747
	30			28.21	0.774	28.1	0.755

reports the corresponding analysis for the FBB Pixel variant. In accordance with the previous observations, slightly higher $\sigma$ values were employed for the FHBB experiments, while lower $\sigma$ values were chosen for the FBB Pixel case to ensure consistency across evaluations.

For each $(\sigma ,\lambda )$ pair, two random images from the test set were selected. From the results presented in Table 6, no definitive conclusion can be drawn regarding the superiority of $FHB{B}_1$ or $FHB{B}_2$. Nevertheless, a consistent tendency can be identified: $FHB{B}_1$ achieves marginally better performance for smaller $\sigma$ values, whereas $FHB{B}_2$ becomes more effective as $\sigma$ increases (see Table 6).

A comparable pattern is observed in Table 7, where neither $FB{B}_1$ Pixel nor $FB{B}_2$ Pixel decisively outperforms the other. However, the trend is reversed compared to the previous analysis—the $FB{B}_2$ Pixel variant shows a slight advantage for smaller values of $\sigma$, while the $FB{B}_1$ Pixel variant exhibits clear superiority for larger $\sigma$. Overall, a modest yet consistent advantage can be attributed to the $FB{B}_1$ Pixel variant when $\sigma$ is high.

4.1.4. Visual Comparison of Reconstruction Quality

To complement the quantitative evaluation presented in the previous subsections, we provide a visual comparison of the reconstructed images obtained using the three algorithms—FBB Pixel, FBB Global, and FHBB. The aim of this analysis is to highlight the qualitative behavior of each approach in terms of texture recovery, edge sharpness, and visual consistency under representative parameter settings.

Figure 5 illustrates the results for a moderate noise configuration with parameters $\sigma =0.12$, $\lambda =0.08$, and $\mathrm{num}\_\mathrm{iters}=30$. This setup corresponds to the first Barzilai–Borwein step-size rule determined by (10). As confirmed by the quantitative metrics, the FHBB method provides the visually most convincing reconstructions, preserving image structure while effectively removing Gaussian noise. We observe that the FBB Pixel variant slightly outperforms the FBB Global approach in terms of both PSNR and SSIM, which is consistent with the quantitative analysis presented earlier. This behavior is expected, as the pixel-wise formulation allows local step-size adaptation, enabling better preservation of fine details and sharp transitions within the image. Nevertheless, both variants demonstrate stable performance and comparable overall denoising quality, confirming the consistency of the FBB framework across different parameter regimes.

A similar trend is observed for the second configuration shown in Figure 6, where the noise level is increased to $\sigma =0.15$, while $\lambda =0.08$ and $\mathrm{num}\_\mathrm{iters}=30$ are kept fixed.

The second Barzilai–Borwein rule (11) is used in this case. The higher noise intensity further accentuates the differences between the methods. Once again, FHBB demonstrates superior visual quality, maintaining structural details and preventing oversmoothing, consistent with its performance observed for quantitative measures. The FBB Pixel and FBB Global methods exhibit alternating advantages depending on the local image characteristics, confirming that their performance is image-dependent and complementary rather than hierarchical.

Overall, the visual analysis aligns with the conclusions drawn from the quantitative evaluation: the FHBB algorithm consistently outperforms both FBB Pixel and FBB Global for both Barzilai–Borwein step-size schemes, demonstrating robustness and superior visual fidelity. The observed variations between the two FBB variants are expected and reflect the trade-offs between localized adaptation and global smoothness inherent to their formulations.

5. Conclusions and Discussion of Further Research

We propose the inclusion of an additional parameter in the step size of the BB gradient methods for solving unconstrained optimization problems. This corrective parameter is derived from a suitably defined neutrosophic logic system. As a result, we introduce a composite step size for the BB iterations that incorporates the corrective parameter, enabling the method to adapt based on previous iteration behavior. Theoretical analysis demonstrates that the proposed iterations converge under the same conditions as the original methods. Furthermore, numerical comparisons indicate that the proposed methods outperform the standard BB variants according to Dolan–Moré performance profiles, particularly in terms of CPU time, the number of function evaluations, and the number of iterations required. In addition, experimental results confirm that the proposed methods perform well on practical applications such as image restoration, particularly in image denoising tasks.

The use of a backtracking line search guarantees the global convergence of the FBB method; however, it substantially increases CPU time and the number of function evaluations. To reduce both CPU time and the number of function evaluations in the iterative scheme (18), future research will explore a variant that omits the parameter $t_k$. This modification leads to a new iterative scheme of the form

$$x_{k+1}=x_k-{\nu }_k{\lambda }_k^{BB}{g}_k.$$

(52)

The iterative scheme (52) relies on two parameters, ${\nu }_k$ and ${\lambda }_k^{BB}$, whereas the original scheme (18) depends on three. This difference provides greater flexibility in defining the fuzzy parameter ${\nu }_k$. The method (52), referred to as the modified BB gradient method based on the neutrosophic logic system (MFBB), will be the subject of future investigation.

Additionally, a promising direction for future research is to conduct a systematic sensitivity analysis of the neutrosophic parameters $(c_1,c_2,\sigma )$. Such an investigation could provide deeper insight into their influence on the performance of neutrosophic-based gradient methods and help further optimize the proposed framework.