Hybridised Network of Fuzzy Logic and a Genetic Algorithm in Solving 3-Satisfiability Hopfield Neural Networks

Abstract

This work proposed a new hybridised network of 3-Satisfiability structures that widens the search space and improves the effectiveness of the Hopfield network by utilising fuzzy logic and a metaheuristic algorithm. The proposed method effectively overcomes the downside of the current 3-Satisfiability structure, which uses Boolean logic by creating diversity in the search space. First, we included fuzzy logic into the system to make the bipolar structure change to continuous while keeping its logic structure. Then, a Genetic Algorithm is employed to optimise the solution. Finally, we return the answer to its initial bipolar form by casting it into the framework of the hybrid function between the two procedures. The suggested network’s performance was trained and validated using Matlab 2020b. The hybrid techniques significantly obtain better results in terms of error analysis, efficiency evaluation, energy analysis, similarity index, and computational time. The outcomes validate the significance of the results, and this comes from the fact that the proposed model has a positive impact. The information and concepts will be used to develop an efficient method of information gathering for the subsequent investigation. This new development of the Hopfield network with the 3-Satisfiability logic presents a viable strategy for logic mining applications in future.

1. Introduction

Artificial intelligence (AI) technology has advanced significantly with the development of a hybrid system that combines neural networks, logic programming models, and SAT structures. The artificial neural network (ANN) is a robust analysis operating model that has previously seen extensive investigation and application by specialists and researchers due to its capacity to manage and display complex challenges. The Hopfield neural network (HNN), established by Hopfield & Tank [1], is one of the many neural networks. The HNN is a recurrent network that performs similarly to the human brain and can solve various challenging mathematical problems. A symbolised representing structure that governs the data stored in the HNN was produced by using Wan Abdullah’s (WA) approach that describes logic rules [2]. Abdullah suggested one of the earliest approaches to incorporate SAT into the HNN. The work was explained by embedding the SAT structure into the HNN by reducing the cost function to a minimum value. Abdullah offered a unique logic programming as a symbolic rule to represent the HNN to solve the issue by determining the symbol’s reduced cost function using the final Lyapunov energy function. Therefore, the method successfully surpassed the traditional learning method like Hebbian learning and can intelligently compute the relationship between the reasoning.

The work is expanded by Sathasivam when the SAT structure, specifically Horn Satisfiability (HornSAT), is utilised [3]. The proposed logic demonstrates how well the HornSAT can converge to the least amount of energy. Years later, Kasihmuddin et al. [4] pioneered the work of 2-Satisfiability (2SAT). The 2SAT structure is being implemented in the HNN as the systematic logical rule. According to the previous work, the proposed 2SAT in the HNN manages to reach a high global minima ratio in a respectable amount of time. Two literals are used in every clause of the suggested logical structure, and a disjunction joins all clauses. By contrasting the cost function and the Lyapunov energy function, this logical rule was incorporated into the HNN. Mansor et al. [5] then expanded the logical rule’s order by suggesting a systematic logical rule with a higher order. This work improves the previous work by adding three literals per clause, namely 3-Satisfiability (3SAT) in the HNN. The third-order Lyapunov energy function is evaluated with the cost function to determine the third-order synaptic weight. Despite the suggested study’s associative memory of the HNN with 3SAT expanding exponentially, it defined the global minimum ratio’s superior value. The proposed 3SAT increases the network’s storage capacity in the HNN because the neuron’s ability to produce many local minima solutions is still limited. The use of systematic SAT in ANNs was made possible in large part by this research. The rigidity of the logical composition, which added to overfitting results in the HNN, is the drawback of any kSAT. As soon as the number of neurons is elevated, the literals in a clause result in low synaptic weight values, which reduces the likelihood of finding global minimum solutions. Various options in the search space solutions are required to ensure a thorough exploration of the neurons. Thus, to overcome the downside of the structure, we offer a hybridised network of 3SAT logic that uses fuzzy logic and a metaheuristic algorithm to significantly widen the search space and improve the presentation of logic programming in the HNN.

The word fuzzy refers to things that appear not to be adequate or ambiguous [6]. Numerous-valued logic allows an idea to have some degree of reality [7]. It can simulate human experience and decision-making. Fuzzy logic has been used in numerous scientific fields, including neural networks, machine learning, artificial intelligence, and data mining [8,9,10,11]. This method can handle insufficient, inaccurate, inconsistent, or uncertain data. Since then, a broad range of research has shown the benefits of this approach, including the fact that the fuzzy sets are straightforward to use, the method does not require a sophisticated mathematical model, and the cost of computing is little [12]. Fuzzy logic builds the proper rules via a fuzzification and defuzzification process. Incorporating the fuzzy logic process with the HNN lowers the processing cost and enables the suggested scheme to accommodate more accurate neurons. As part of the defuzzifying procedure, any suboptimal neuron will be modified using the alpha-cut technique until the ideal neuron state is found. Then, we integrate the network with a metaheuristic algorithm, a Genetic Algorithm (GA). The Darwinian Theory of Survival of the Fittest is replicated by the GA, a form of natural selection. The GA was proposed by Holland [13] as a widely used optimisation technique founded on the population of metaheuristic algorithms. A population of potential solutions is improved over time as part of the implementation of the GA, which is an iterative process. The paradigm of computer intelligence known as metaheuristic algorithms is particularly effective at handling challenging optimisation problems.

Thus, the objectives of this research are listed as follows: (1) To construct a continuous search space by applying a fuzzy logic system in the training phase of 3-Satisfiability logic. (2) To create a design structure integrating a Genetic Algorithm to improve all the logical combinations of reasoning that will minimise the cost function. (3) To produce a new performance metric to evaluate the method’s efficiency in obtaining the highest clauses of satisfying assignments in the learning phase. (4) To establish a thorough assessment of the suggested hybrid network using the generated datasets that can perform significantly with the Hopfield neural network model. Our findings indicated that the suggested HNN-3SATFuzzyGA exhibits the finest performance in conditions of error analysis, efficiency evaluation, energy analysis, similarity index performance, and computational time. The layout of this article is as follows: Section 2 provides an outline structure of 3-Satisfiability. Section 3 describes the application of 3SAT into the Hopfield Neural Network. The general formulation of fuzzy logic and Genetic Algorithm is explained in Section 4 and Section 5, respectively. Section 6 discusses the execution of the suggested model of HNN-3SATFuzzyGA. Additionally, Section 7 listed the experimental and parameter sets, along with the performance evaluation metrics used. Section 8 reviewed the outcomes and discussion on the findings of HNN-3SATFuzzyGA. Section 9 concludes the paper’s results and suggestions for future research.

2. 3-Satisfiability

Satisfiability signifies a logical rule that includes several clauses $\left({\beta }_m\right)$ with literals $\left({\omega }_i\right)$. The SAT problem explores whether a specific SAT formula is satisfactory when it has a value that evaluates it as accurate. Our study will focus on 3-Satisfiability, also known as 3SAT. Hence, the following describes the principles structure of a 3SAT problem: (1) a set of a clause, ${\beta }_m$, is connected by AND $\left(\wedge \right)$ with, ${\beta }_1\wedge {\beta }_2\wedge \dots \wedge {\beta }_m$ where $m\ge 3$. If $m=3$, the Boolean SAT will have three clauses. The value of $m$ may increase according to the number of clauses specified. (2) For each clause of the Boolean SAT, there will be $i$ literals that are connected by OR $\left(\vee \right).$ The literals are ${\omega }_1,{\omega }_2, \dots , {\omega }_i$ for every $\omega \in \left\{-1,1\right\}$. Moreover, since this study focuses on 3SAT, it will consist of only three literals per clause. (3) The choice of the literal’s condition can be either negative or positive. The principal formula for 3SAT is

$${\theta }_{3SAT}={\wedge }_{i=1}^l{\beta }_i^{\left(3\right)} {\wedge }_{i=1}^m{\beta }_i^{\left(2\right)}{\wedge }_{i=1}^n{\beta }_i^{\left(1\right)}$$

(1)

where ${\beta }_i$ signifies a set of a clause. Meanwhile, $l,m \mathrm{and} n$ indicate the number of clauses in the third, second and first order, respectively. The value of 3 (third), 2 (second) and 1 (first) indicates the order of clauses in ${\theta }_{3SAT}$. Given that, the overall amount of clauses $\left(nc\right)$ in ${\theta }_{3SAT}$ is $nc=l+m+n$and the entire literals $\left(\tau \right)$ included in ${\theta }_{3SAT}$ for all order of clauses are $\tau =3\times nc$. Since this study focuses on 3SAT, Equation (2) below is an example that signifies the logical rule with exactly three literals per clause:

$${\theta }_{3SAT}=\left({\omega }_1\vee \neg {\omega }_2\vee \neg {\omega }_3\right)\wedge \left({\omega }_4\vee {\omega }_5\vee \neg {\omega }_6\right)\wedge \left({\omega }_7\vee {\omega }_8\vee {\omega }_9\right)$$

(2)

The assignments that correlate to the 3SAT can be expressed as $\left\{\mathrm{True}, \mathrm{False}\right\}=\left\{1,-1\right\}$ [5]. Determining a list of assignments that will translate the entire formulation into TRUE is the main objective of the 3SAT. If the logical rule provides the actual value, the ${\theta }_{3SAT}=1$ is considered fulfilled; if it provides a false value, the ${\theta }_{3SAT}=-1$ is not fulfilled.

3. 3-Satisfiability in the Hopfield Neural Network

The HNN is one of the subdivisions of the ANN that has a feedback mechanism where every single neuron output is sent back into the input [1]. Because of the non-symbolic nature of the HNN structure, the logical conceptualisation of 3SAT can improve its capacity for great storage. The general mathematical formula for the neuron’s activation is presented as follows:

$$S_i=\left\{\begin{array}{c}1, {\displaystyle \sum }_j{W}_{ij}{S}_j>{\xi }_i \\ -1, \mathrm{otherwise}\end{array}\right.$$

(3)

where ${\xi }_i$ signifies the threshold value and $W_{ij}$ indicates the weight for component $j$to $i$.

The 3SAT algorithm used in this study, known as HNN-3SAT, uses three neurons for each sentence. Additionally, the HNN’s synaptic weight connection is symmetrical with $W_{ij}=W_{ji}$ and there is no self-connection of $W_{ii}=W_{jj}=0$ between any of the neurons. The structure serves as the framework for the compelling content addressable memory (CAM), which was used to store and recover every necessary pattern from handling the optimisation problem. All the variables will be assigned to the specified cost function, $E_{{\theta }_{3SAT}}$ to embed the ${\theta }_{3SAT}$. In the meantime, the following formula is used to generate the cost function for 3SAT:

$$E_{{\theta }_{3SAT}}=\frac{1}{2^3}{\displaystyle \sum }_{i=1}^{nc}\left({\displaystyle \prod }_{j=1}^3{V}_{ij}\right)$$

(4)

where $nc$ is the sum of clauses and $V_{ij}$ in Equation (5) below can be shifted parallel to the corresponding state of the literal:

$$V_{ij}=\left\{\begin{array}{c}\left(1-S_X\right) \mathrm{if} \neg X_1 \\ \left(1+S_X\right) \mathrm{if} X_1\end{array}\right.$$

(5)

By assessing the inconsistencies of the 3SAT structure rule, the minimum cost function $E_{{\theta }_{3SAT}}$ must be achieved [14]:

$$min\left[E_{{\theta }_{3SAT}}\right]=0$$

(6)

Note that the amount of $E_{{\theta }_{3SAT}}$ grows in direct proportion to the number of inconsistencies in the clauses [14]. In this case, when the number of unsatisfied clauses rises, $E_{{\theta }_{3SAT}}$ will rise as well. $E_{{\theta }_{3SAT}}=0$ is the lowest amount that can be attained for the cost function. The HNN will use the local field equation during the retrieval phase to execute the iterative updating of neurons. The parameters of the synaptic weight, such as $W_{ijk}$, can be decided by associating the cost function with the Lyapunov energy function of HNN using Wan Abdullah’s (WA) approach [2]. The stable condition of the neurons is represented by the minimal values from the energy function [3]. Each variant of the HNN-3SAT has a different generalised Lyapunov final energy function.

$$E=-\frac{1}{3}{\displaystyle \sum }_i{\displaystyle \sum }_j{\displaystyle \sum }_k{W}_{ijk}{S}_i{S}_j{S}_k-\frac{1}{2}{\displaystyle \sum }_i{\displaystyle \sum }_j{W}_{ij}{S}_i{S}_j-{\displaystyle \sum }_i{W}_i$$

(7)

4. Fuzzy Logic

The motivation of fuzzy logic appears when there are wide-ranging ways humans use imprecise information to make judgments. Traditional Boolean logic only will not always be able to unravel a problem. Zadeh [15] proposed fuzzy logic and fuzzy set to apply vagueness and control better with human reasoning errors, while Mamdani [16] was the first to bring these ideas into the control theory. Both accomplishments make it possible to use fuzzy logic in various applications successfully.

4.1. Fuzzy Set

Zadeh [15] created the theory of a fuzzy set to represent vague concepts and handle informational uncertainty. The fuzzy set can be thought of as a smooth set, unlike the classical sets, in which the fuzzy set chooses whether to belong to or not belong to a particular set gradually or progressively instead of just sharply. Distinguishing a fuzzy set $A$ in $X$ to realise the involvement of any element $x$, where $x\in X$, with a membership degree ${\mu }_A\left(x\right)$ that corresponds to the range between zero and one and ${\mu }_A\in \left[0,1\right]$ is a process of characterising a membership function $A$. The membership function of $A$ is symbolised by the symbol ${\mu }_A$ where the ordered pair can likewise be used to describe this fuzzy set $A$ in $X$:

$$A=\left\{\left(x, {\mu }_A\left(x\right)\right)\left|x\in X\right.\right\}$$

(8)

in which $0\le {\mu }_A\left(x\right)\le 1$ where ${\mu }_A\left(x\right)=0$means a portion $x$ does not fit into a fuzzy set and ${\mu }_A\left(x\right)=1$ means $x$ is a component of a fuzzy set. The notation of the set can be demonstrated as a discrete universe $X$ [17].

$$A=\frac{{\mu }_A\left(x_1\right)}{x_1}+\frac{{\mu }_A\left(x_2\right)}{x_2}+\dots +\frac{{\mu }_A\left(x_n\right)}{x_n}={\displaystyle \sum }_{x\in X}\frac{{\mu }_A\left(x\right)}{x}$$

(9)

where $\sum$ indicates union through all $x\in X$. The degree of membership of $x$ in $A$ is identified as ${\mu }_A\left(x\right)$.

4.2. Fuzzy Logic System

The critical components of a fuzzy logic system (FLS) are presented in Figure 1. It includes the fuzzifier stage, which converts crisp inputs into fuzzy sets. Then, the rules-based converts fuzzy sets from the backgrounds to equivalent fuzzy sets. Lastly, the defuzzifier stage will receive a crisp assessment from the fuzzy sets.

5. The Genetic Algorithm

The GA replicates the Darwinian Theory of Survival of the Fittest, a form of natural choice commonly utilised as an optimisation technique based on the population of metaheuristic algorithms. The GA is executed by refining the population of possible solutions over time. It was first proposed by Holland [13] as a method for finding good-quality answers to the combinatorial optimisation problem. Holland adopted the natural GA processes of selection, crossover, and mutation. In line with the findings, Zamri et al. [18] used a multi-discrete GA to enhance the functionality of the weighted random k satisfiability in the HNN. It works remarkably well in various optimisation issues, mainly when attempting a global search in a discrete space. It is an iterative process that must create two definitions before the iteration may initiate. The search space is defined by (1) a genetic model of the search area and (2) a fitness function which frequently represents the goal function of the optimisation problem and calculates how suitable a candidate solution is. There are five main steps in GA: (1) initialisation, (2) fitness evaluation, (3) selection, (4) crossover and (5) mutation. The flowchart for the GA method is shown in Figure 2.

6. Fuzzy Logic System and Genetic Algorithm in 3-Satisfiability Logic Programming

The discovery of the HNN and SAT models, satisfying assignments of embedded logical structure, results in an optimal training phase (when the cost function is zero). A metaheuristic algorithm was used as the training method to carry out the kSAT minimisation objective in the HNN [14]. The optimisation operators enable the algorithm to enhance the network’s convergence towards cost function minimisation [19]. $S_i\in \left\{-1,1\right\}$ is used as the instance for the state of neurons in the training phase’s representation of the 3SAT’s potentially satisfying assignments. The drawback of the previous approach is the usage of $\left\{-1,1\right\}$ representations. In this study, the process is to employ the neuron’s expression continuously so that every component in the exploration universe will be computed with various fitness functions. As a result, we embedded fuzzy logic in the system in the first stage. Additionally, the domain of fuzzy sets and functions is introduced when the binary (Boolean) problem is made continuous while maintaining its logical nature. A metaheuristic algorithm is employed in the second stage to obtain the best solution for the system to converge optimally. We cast it into the framework of the hybrid function between two techniques and then return (transform) the solution to the initial binary structure. This section explains how fuzzy logic incorporates the GA in 3SAT, namely the HNN-3SATFuzzyGA. The proposed technique is allocated into two stages: (1) the Fuzzy logic system’s stage and (2) the Genetic algorithm’s stage.

Firstly, the fuzzy logic control system is used in determining the output $\left({\beta }_i\right)$ with three inputs $\left({\omega }_i\right)$. For example, three literals represent the input of ${\omega }_1,{\omega }_2$ and ${\omega }_3$ and the system produces ${\beta }_1$ as its output. The procedure is repeated for all number of clauses. Three membership functions are set up as the inputs. Meanwhile, there are two membership functions used in the output. In the first phase, each ${\omega }_i$ will be randomly assigned. After that, fuzzy logic will take place to evaluate the value of the first output ${\beta }_i$. The first step in the fuzzy logic system is fuzzification, where each literal will be assigned to its membership function of:

$${\tau }_i=\left\{\left({\omega }_i, {\mu }_{\tau }\left({\omega }_i\right)\right)\left|{\omega }_i\in X\right.\right\}$$

(10)

in which $0\le {\mu }_{\tau }\left({\omega }_i\right)\le 1$.

Then, fuzzy rules will be applied using the first clause block, ${\beta }_i$, e.g., ${\beta }_1={\omega }_1\vee \neg {\omega }_2\vee \neg {\omega }_3$. By enabling ${\mu }_{\tau }\left({\omega }_i\right)$to accept any value in [0, 1], we will employ Zadeh’s operators, such as the logical operator AND holds the function $max\left\{{\mu }_{\tau }\left({\omega }_i\right),{\mu }_{\tau }\left({\omega }_j\right)\right\}$ and logical operator OR holds the function $min\left\{{\mu }_{\tau }\left({\omega }_i\right),{\mu }_{\tau }\left({\omega }_j\right)\right\}$. This conversion creates the fuzzy function logically consistent with the binary function (Boolean function). This process will be repeated for a total number of clauses $\left(nc\right)$. Defuzzification steps will be employed to convert fuzzy values to its crips output. The alpha-cut method will modify neuron clauses through defuzzification until the proper neuron state is attained [17]. The improved alpha-cut formula has an estimation model of:

$$\mathrm{if} {\beta }_i<\alpha , \mathrm{then} {\beta }_i^{\prime }=0, \mathrm{otherwise} {\beta }_i\ge \alpha , \mathrm{then} {\beta }_i^{\prime }=1$$

(11)

The second phase of the learning part is applying the Genetic Algorithm (GA). The initialisation of the chromosome, $C{r}_i$ is obtained from the previous phase (fuzzy logic system solutions). The fitness of each chromosome, $C{r}_i$ is evaluated using a fitness function:

$${\mathsf{\Phi }}_{C{r}_i}={\displaystyle \sum }_{i=1}^l{\beta }_i^{\left(3\right)}+{\displaystyle \sum }_{i=1}^m{\beta }_i^{\left(2\right)}+{\displaystyle \sum }_{i=1}^n{\beta }_i^{\left(1\right)}$$

(12)

where the conditions meet

$${\beta }_i^{\left(k\right)}=\left\{ \begin{array}{c}1, \mathrm{fulfilled}\\ 0, \mathrm{otherwise}\end{array}\right.$$

(13)

The sum of all clauses in ${\theta }_{3SAT}$ implies the optimum fitness ${\mathsf{\Phi }}_{C{r}_i}$ of each $C{r}_i$ [18]. Equation (14) denotes the objective function of GA:

$$\max \left[{\mathsf{\Phi }}_{C{r}_i}\right]$$

(14)

The genetic information $\left({\beta }_i\right)$ of two parents (chromosome, $C{r}_i$) is exchanged using crossover operators to create the offspring (new chromosome, $C{r}_i$). The parents’ genetic information is replaced according to the segments produced in the crossover stage, which selects random crossover points [20]. One mechanism that keeps genetic diversity from one population to the next is mutation. The mutation procedure entails the flipping of a single gene from another. A gene $\left({\beta }_i\right)$ of a given chromosome $\left(C{r}_i\right)$ is disseminated within itself by the displacement mutation operator. The location is randomly selected from the provided gene $\left({\beta }_i\right)$ for displacement to guarantee that both the final result and a random displacement mutation are valid. Note that the GA process will be repeated until the $max\left[{\mathsf{\Phi }}_{C{r}_i}\right]$ is achieved. Algorithm 1 shows the pseudocode for FuzzyGA.

Algorithm 1: Pseudocode for FuzzyGA

Begin the first stage Step 1: Initialisation Initialise the inputs${\omega }_i$, output${\beta }_i$, membership function${\mu }_{\tau }\left({\omega }_i\right)$ and rule lists $RL$. Step 2: Fuzzification Assign the literal, ${\omega }_i$ to it membership function which $0\le {\mu }_{\tau }\left({\omega }_i\right)\le 1$. Step 3: Fuzzy rules Apply rules lists, $RL$ using the first clause block, ${\beta }_i$, e.g., ${\beta }_1={\omega }_1\vee \neg {\omega }_2\vee \neg {\omega }_3.$ Step 4: Defuzzification Implement the defuzzification step and estimation model. end Begin the second stage Step 1: Initialisation Initialise random populations of the instances, ${\beta }_i$. Step 2: Fitness Evaluation Evaluate the fitness of the population of the instances, ${\beta }_i$. Step 3: Selection Select best $S_i$ and save as $S_i^{\prime }$. Step 4: Crossover Randomly select two bits, $x_i$ from ${\beta }_i$. Generate new solutions and save them as ${\beta }_i^{″}$. Step 5: Mutation Select a solution $x_i$ from ${\beta }_i^{″}$. Mutates the solution by flipping the bits randomly and generates $x_i^{\prime }$. if $x_i^{\prime }<criterion$,   Update $x_i^{\prime }$. end Update $x_i=$ $x_i^{\prime }$. end

7. Experimental Setup

Matlab 2020b software is used in this experimental simulation to perform the fuzzy logic and GA as a training tool in the HNN-3SAT. The HNN models in this work produced 3SAT clauses with varying degrees of difficulty using simulated datasets. The experiment will be terminated if the CPU time used to generate data surpasses 24 h. In addition, we choose HTAF due to its stability, and notably, an excellent approach to an activation function developed in the HNN. The final energy implementation requirements value is 0.001 because it effectively reduced statistical errors [3]. The effectiveness of the HNN-3SATFuzzyGA will be assessed by comparing it with another two models, the HNN-3SAT and HNN-3SATFuzzy, for accuracy and efficacy. This section evaluates the performance of each phase of the proposed model using the relevant performance measures. The training phase and testing phase are defined as the two key phases. The relevant objectives for each phase, which are stated below, determine the best solution. The training phase involves minimising Equation (4). The evaluation metrics used in this work are training error analysis and efficiency evaluation.

Meanwhile, the testing phase includes producing global minima solutions within the desired tolerance value and calibre of the returned final neuron states. The evaluation metrics involved in this phase are testing error analysis, energy analysis, similarity index and computational time.

Table 1. Listing of related parameters in the HNN-3SAT.

Parameter	Parameter Value
Number of neurons	$9\le NN\le 180$
Total of combinations	100
Tolerance measurement	0.001
CPU time threshold	24 h
Activation function	HTAF
Synaptic weight method	Wan Abdullah (WA) method
$\mathrm{Final} \mathrm{neuron} \mathrm{states}, S_i$	$\left\{-1, 1\right\}$

,

Table 2. Listing of related parameters in the 3SATFuzzy.

Parameter	Parameter Value
$\mathrm{Fuzzy} \mathrm{membership} \mathrm{neuron}, {\mu }_{\tau }$	$\left[0,1\right]$
$\mathrm{Input} \mathrm{parameter}, {\omega }_i$	$\left[0,1\right]$
$\mathrm{Output} \mathrm{parameter}, {\beta }_i$	$\left[0,1\right]$
Number of inputs	3
Number of outputs	1

and

Table 3. Listing of related parameters in the 3SATFuzzyGA.

Parameter	Parameter Value
$\mathrm{Populations}, pop$	100
$\mathrm{Selection} \mathrm{rate}, {\mathsf{\Gamma }}_S$	0.1
$\mathrm{Crossover} \mathrm{rate}, {\mathsf{\Gamma }}_C$	1
$\mathrm{Mutation} \mathrm{rate}, {\mathsf{\Gamma }}_M$	1

shows the list of parameters in the execution of evaluation metrics.

7.1. Performance Metric

Several metrics are used to analyse the performance of all the presented HNN-3SAT models. The effectiveness of each HNN-3SAT model will be evaluated using various criteria. The following are the goals for each phase:

• Training phase—The evaluation includes how well the suggested model performs in locating the most satisfied clauses in the learning phase of the 3SAT assignments. In this phase, the computation involves error analysis to verify the performances of each model. It also assesses the efficiency of the optimisation method in the training phase.
• Testing phase—The evaluation in the testing phase comprises computing the error analysis to validate the satisfied clauses. Besides that, it calculates the similarity degree to which the final neuron state differs from or resembles the benchmark states. It also verifies the robustness of the suggested model by energy analysis and computational time.

7.1.1. Training Phase Metric

This evaluation shows that the finest form of every iteration is substantial when executing an exploring procedure [21]. The error analysis metrics for each HNN-3SAT model in the training phase include RMSE, MAE, SSE and MAPE in Equations (15)–(18). The Root Mean Square Error (RMSE) reports the real difference between the anticipated amounts and the calculated value [22]. The Mean Absolute Error (MAE) is a valuable metric for evaluating the case by assessing the change in the mean gap between the determined and expected values [23]. The ideal HNN network attempt for a smaller MAE value. Since $I_{highest} \ge 0$, there will be only positive values of the MAE [24]. The Sum Squared Error (SSE) is a statistical technique for calculating how much the data deviates from expected values and makes it empirically validated [25]. The SSE is extremely sensitive to minor inaccuracy. Therefore, it will increase incorrect final solutions [26]. The Mean Absolute Percentage Error (MAPE) is a modified form of the MAE in which the results are standardised to a percentage [27]. If the measured value is zero, the MAPE cannot be employed because doing so will result in a division by zero. The MAPE evaluation with the lowest value will be the most effective network [14].

$$RMS{E}_{train}={\displaystyle \sum }_{i=1}^n\sqrt{\frac{1}{n}{\left(I_{highest}-I_x\right)}^2}$$

(15)

$$MA{E}_{train}={\displaystyle \sum }_{i=1}^n\frac{1}{n}\left|I_{highest}-I_x\right|$$

(16)

$$SS{E}_{train}={\displaystyle \sum }_{i=1}^n{\left(I_{highest}-I_x\right)}^2$$

(17)

$$MAP{E}_{train}={\displaystyle \sum }_{i=1}^n\frac{100}{n}\frac{\left|I_{highest}-I_x\right|}{\left|I_x\right|}$$

(18)

where $I_{highest}$ refers to the maximum fitness attained by the proposed model and $I_x$ is the obtained fitness value of the network and $n$ denoted the total number of iterations in the learning phase.

The two newly proposed efficiency performance metrics in Equations (19) and (20) are inspired by Djebedjian et al. [28] for calculating the efficiency of an optimisation algorithm as the computational effort required to find an optimal solution. The iteration efficiency evaluates the number of iterations the models require to compute in the fuzzy-meta learning phase.

$$\mathrm{Iteration} \mathrm{efficiency}, {\eta }_{iter}=1-\frac{\log N_{iter}}{\log n}$$

(19)

where $N_{iter}$ is the total iteration of the networks to enter the fuzzy-meta learning phase and $n$ is the total iteration of the network.

The evaluation efficiency calculates the number of successful iterations of the model in obtaining the highest clauses of satisfying assignments in the fuzzy-meta learning phase.

$$\mathrm{Evaluation} \mathrm{efficiency}, {\eta }_{eval}=1-\frac{\log N_{eval}}{\log N_{iter}}$$

(20)

where $N_{eval}$ is the total of satisfying solution in the fuzzy-meta learning phase in achieving the maximum clauses and $N_{iter}$ is the total iteration the networks enter the fuzzy-meta learning phase.

7.1.2. Testing Phase Metric

In evaluating the error analysis for each HNN-3SAT model in the testing phase, the same metrics of RMSE, MAE, SSE and MAPE in Equations (21)–(24) are used.

$$RMS{E}_{test}={\displaystyle \sum }_{i=1}^n\sqrt{\frac{1}{\gamma .\epsilon }{\left({\varphi }_i-z_i\right)}^2}$$

(21)

$$MA{E}_{test}={\displaystyle \sum }_{i=1}^n\frac{1}{\gamma .\epsilon }\left|{\varphi }_i-z_i\right|$$

(22)

$$SS{E}_{test}={\displaystyle \sum }_{i=1}^n{\left({\varphi }_i-z_i\right)}^2$$

(23)

$$MAP{E}_{test}={\displaystyle \sum }_{i=1}^n\frac{100}{\gamma .\epsilon }\frac{\left|{\varphi }_i-z_i\right|}{\left|z_i\right|}$$

(24)

where ${\varphi }_i$ refers to the maximum optimal solution attained by the proposed model and $z_i$ is the obtained optimal solution value of the network and $n$ denoted the total number of iterations in the learning phase.

In order to analyse the diversification of the 3SAT produced by the suggested models, the final neuron state, $S_i$ can be compared with the ideal neuron state, $S_i^{ideal}$ [29]. The similarity metrics of the Jaccard Index [29] in Equation (25) and the Gower & Legendre Index [30] in Equation (26) are applied using the benchmark information in

Table 4. Benchmark information in the similarity index.

Parameter	${\mathit{S}}_{\mathit{i}}^{\mathit{i}\mathit{d}\mathit{e}\mathit{a}\mathit{l}}$	${\mathit{S}}_{\mathit{i}}$
$\rho \rho$	$1$	$1$
$\rho \eta$	$1$	$-1$
$\eta \rho$	$-1$	$1$
$\eta \eta$	$-1$	$-1$

to examine the similarity between the positive and negative of $S_i^{ideal}$ with $S_i$. The following are the formulas for both similarity metrics:

$$\mathrm{Jaccard} \mathrm{Index}, JI=\frac{\rho \rho }{\rho \rho +\rho \eta +\eta \rho }$$

(25)

$$\mathrm{Gower} \mathrm{and} \mathrm{Legendre} \mathrm{Index}, GLI=\frac{pp+\eta \eta }{\rho \rho +0.5\left(\rho \eta +\eta \rho \right)+\eta \eta }$$

(26)

Moreover, the energy analysis is calculated to test the robustness of the proposed network. The global minima ratio ($z{M}_{Global}$) and local minima ratio $\left(y{M}_{Local}\right)$ are evaluated during the testing phase to check the accuracy of the final neuron states retrieved by the model. $z{M}_{Global}$ and $y{M}_{Local}$ in Equations (27) and (28), respectively, are employed to thoroughly investigate the energy analysis [3]. The computational time analysis is used to determine the productivity of the network. In Equation (29), processing time generally refers to the total time it takes to finish a simulation.

$$z{M}_{Global}=\frac{1}{\gamma .\epsilon }{\displaystyle \sum }_{i=1}^n{z}_i$$

(27)

$$y{M}_{Local}=\frac{1}{\gamma .\epsilon }{\displaystyle \sum }_{i=1}^n{y}_i$$

(28)

$$\mathrm{Processing} \mathrm{time}=\mathrm{Training} \mathrm{phase} \mathrm{time}+\mathrm{Testing} \mathrm{Phase} \mathrm{time}$$

(29)

where $\gamma .\epsilon$ is the product of the number of combinations and trials. Meanwhile $z_i$ and $y_i$ refers to the number of global minima and local minima, respectively.

Table 5. List of parameters implicated in the learning and testing phases.

Parameter	Parameter Notion
$I_{highest}$	Maximum fitness
$I_x$	Current fitness
$n$	Number of learning
$N_{iter}$	Number of iterations
$N_{eval}$	Number of successful iterations
$\gamma$	Number of combinations
$\epsilon$	Number of trials
${\varphi }_i$	Number of total solutions
$z_i$	Number of global minima
$y_i$	Number of local minima

below shows the list of parameters used in the learning and testing phases.

8. Results and Discussion

The results of three distinct methods will be discussed in this section. HNN-3SAT uses a basic random searching algorithm without any improvisation in the training phase. Meanwhile, HNN-3SATFuzzy employs fuzzy logic alone, and HNN-3SATFuzzyGA uses fuzzy logic and a Genetic algorithm in the training phase. Finding satisfying ${\theta }_{3SAT}$ assignments that match the reduction of Equation (6) is the purpose of the training and testing phase in the HNN. This strategy uses the WA method to achieve appropriate synaptic weight regulation [31]. A simulation of the proposed model is conducted with the dynamics of different numbers of neurons to verify the prediction. This study limits the number of neurons to $9\le \mathrm{NN}\le 180$ to determine our suggested model’s success. Additionally, we tested training error, training efficiency, testing error, energy analysis, similarity index and computational time. Results of all the three HNN-3SAT methods in the training and testing phases are shown in Figure 3, Figure 4, Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9.

Figure 3 displays the results of RMSE_train, MAE_train, SSE_train and MAPE_train during the training stage for the three different methods: HNN-3SAT, HNN-3SATFuzzy, and HNN-3SATFuzzyGA, respectively. According to Figure 3a,b, RMSE_train and MAE_train for HNN-3SATFuzzyGA outperformed those of other networks during the training phase. Despite an increase in the number of neurons (NN), the results demonstrate that the RMSE_train and MAE_train values for the HNN-3SATFuzzyGA network are the lowest. The HNN-3SATFuzzyGA solutions deviated from the potential solutions less. Initially, the outcomes for all networks seemed to have little difference during $9\le \mathrm{NN}\le 45$. However, once it reached $\mathrm{NN}=54$, the results started to deviate more for RMSE_train and MAE_train, with performance for HNN-3SATFuzzyGA remaining low. The other two methods considerably increased toward the end of the simulations, especially HNN-3SAT. Based on RMSE_train and MAE_train calculations, the suggested method, HNN-3SATFuzzyGA, achieved $E_{{\theta }_{3SAT}}$ at lower results. The leading cause of this is that fuzzy logic, and GA resolve a better solution in the training phase, enabling the achievement of $E_{{\theta }_{3SAT}}=0$ in a shorter number of cycles. The reason is that the fuzzy logic system is more likely to widen the search during training using fuzzification for precise interpretations. Then, the HNN-3SATFuzzyGA utilised a systematic approach by employing the defuzzification procedure. The usage of GA also helps to optimise the solutions better. Additionally, HNN-3SATFuzzyGA could efficiently verify the proper interpretation and handle more restrictions than the other networks. In another work by Abdullahi et al. [32], a method for extracting the logical rule in the form of 3SAT to characterise the behaviour of a particular medical data set is tested. The goal is to create a reliable algorithm that can be used to extract data and insights from a set of medical records. The 3SAT method extracts logical rule-based insights from medical data sets. The technique will integrate 3SAT logic and HNN as a single data mining paradigm. Medical data sets like the Statlog Heart (ST) and Breast Cancer Coimbra (BCC) are tested and trained using the proposed method. The network has recorded good performance evaluation metrics like RMSE and MAE based on the simulation with various numbers of neurons. The results of Abdullahi et al. [32] can conclude that the HNN-3SAT models have a considerably lower sensitivity in error analysis results. According to the optimal CPU time measured with various levels of complexity, the networks can also accomplish flexibility.

Figure 3c, d demonstrates that HNN-3SATFuzzyGA has lower SSE_train and MAPE_train values. Since HNN-3SATFuzzyGA has a lower SSE_train value, it has a more reliable ability to train the simulated data set. With a lower SSE_train value for all hidden neuron counts, HNN-3SATFuzzyGA was found to have evident good-quality results. Even though the results for SSE_train during $9\le \mathrm{NN}\le 45$have not deviated far from each other’s result, HNN-3SATFuzzyGA remained low until the end of the simulations compared to the other methods. Associated with HNN-3SATFuzzyGA during the final NN, the outcomes of HNN-3SAT increased drastically. The results show the MAPE_train settings’ output for all networks. The MAPE_train value has also provided compelling evidence of fuzzy logic and GA compatibility in 3SAT. The results for HNN-3SATFuzzyGA were significantly at a low level even after $\mathrm{NN}=54$. Compared to HNN-3SATFuzzyGA, the results of HNN-3SAT converged higher, starting from $\mathrm{NN}=54$. As a result, the HNN-3SATFuzzyGA approach performs noticeably better than the other two models. The effectiveness of the network is because the training phase’s operators of fuzzy logic and GA boosted the solutions’ compatibility. Compared to different networks, HNN-3SATFuzzyGA can recover a more precise final state.

Figure 4a,b shows the iteration efficiency $\left({\eta }_{iter}\right)$ and evaluation efficiency $\left({\eta }_{eval}\right)$ for the methods in the training part, respectively. A lower value of ${\eta }_{iter}$ and ${\eta }_{eval}$ stands to be the best result. Generally, a lower ${\eta }_{iter}$ and ${\eta }_{eval}$ score denotes a higher efficiency level. Based on Figure 4a, HNN-3SAT maintains the value of ${\eta }_{iter}=1$ since it has no optimisation operator to improve the learning part. HNN-3SATFuzzyGA and HNN-3SATFuzzy have a lower value of ${\eta }_{iter}$ compared to HNN-3SAT. The reason is that most of the iteration goes into the fuzzy logic and (or) GA optimisation phase. In order to search for the best maximum fitness, the training must go through the operator to locate the best solution. Meanwhile, in Figure 4b, ${\eta }_{eval}$ show the results for the highest successful iterations in obtaining the highest clauses of satisfying assignments in the fuzzy-meta learning phase. HNN-3SAT remains at ${\eta }_{eval}=1$ at all levels since it does not have an optimiser. In the meantime, HNN-3SATFuzzyGA obtained the lowest score since it can achieve the highest successful iterations to obtain maximum fitness. Fuzzy logic is responsible for widening the development effort to obtain an ideal state with satisfying assignments. Owing to the addition of a metaheuristic algorithm, such as GA, HNN-3SATFuzzyGA performed well with the help of operators like crossover and mutation. The crossover is a subclass of global search operators that maximises the search for potential solutions. In this instance, chromosomal crossover (possible satisfying assignments) leads to almost ideal states that might support 3SAT. The mutation operator is vital in locating the best solution for a particular area of the search space. Meanwhile, HNN-3SAT has no capability for exploration and exploitation since it uses a random search technique by the “trial and error” mechanism.

Figure 5 display the results of RMSE_test, MAE_test, SSE_test and MAPE_test in the testing error analyses. Examining the HNN-3SAT behaviour based on error analysis concerning global or local minima solutions is the significance of testing error analysis. The WA approach will construct the synaptic weight after completing the HNN-3SAT clause satisfaction (minimisation of the cost function). Figure 5a,b shows that the testing error for RMSE_test and MAE_test in HNN-3SAT is the highest, indicating that the model cannot induce global minima solutions. The model is overfitted to the test sets due to the unsatisfied clauses. Therefore, we need to add an optimiser such as fuzzy logic to widen the search space and GA optimise the solutions better. Based on RMSE_test and MAE_test calculations, the HNN-3SAT attains $E_{{\theta }_{3SAT}}$ at higher results. The two methods, with an optimiser, maintain the error at a lower level. Additionally, in Figure 5c,d, we consider SSE_test and MAPE_test as two performance indicators in the testing phase when evaluating the error analysis. Notably, an increased likelihood of optimal solutions will be attributed to lower SSE_test and MAPE_test values. Lower SSE_test and MAPE_test scores denote a higher accuracy (fitness) level. The SSE_test and MAPE_test values for the HNN-3SAT model rise noticeably when the number of neurons increases. As a result, the HNN-3SAT model with greater SSE_test and MAPE_test values cannot optimise the required fitness value during the testing phase. After $NN=9$, the values for SSE_test and MAPE_test for the HNN-3SAT model begin to rise. The event occurred due to the HNN-3SAT trial-and-error approach, which resulted in a less ideal testing phase.

Next, we used the similarity index to analyse the similarity and dissimilarity of the generated final neuron states in Figure 6a,b. This indexing is used for divergence investigations. It is particular to binary variables [33]. According to Equations (25) and (26), we searched for the Jaccard Index (JI) and the Gower-Legendre Index (GLI) in the similarity index analysis. A comparison of the JI and GLI values obtained by several HNN-3SAT models is shown in Figure 6a, b. The JI and GLI calculate how the computed states resemble the baseline state. Examining the value of the JI in Figure 6a is best done with lower JI and GLI values. In our investigation, the HNN-3SATFuzzyGA model recorded lower JI values for lower NN between $9<NN<180$. Based on Figure 6a,b, HNN-3SAT has the highest overall similarity index values. In HNN-3SATFuzzy and HNN-3SATFuzzyGA, there is fuzzy logic or (and) GA optimisation in the training phase, resulting in a non-redundant neuron state that achieves the global minima energy due to diversification of the search space. As a result, the final neuron state differs significantly from the benchmark state, causing it to have a lower value. However, HNN-3SAT produces a similar neuron state corresponding to the global minima energy because of the limitation of neuron choice for the variable. That interprets the higher value of the JI and GLI in Figure 6a,b, respectively. Gao et al. [34] used a higher-order logical structure that can exploit the random property of the first, second, and third-order clauses. Alternative parameter settings, including varying clause orderings, ratios between positive and negative literals, relaxation and different numbers of learning trials, were used to assess the logic in the HNN. The similarity index measures formulation is used since the similarity index measurements aim to analyse neuron variation caused by the random 3SAT. The simulation indicated that the random logic was adaptable and offered various solutions. According to Karim et al. [29], the experimental results showed that random 3SAT may be implemented in the HNN and that each logical combination had unique properties. In the entire solution space, random 3SAT can offer neuron possibilities. Third-order clauses were added to the random 3SAT to boost the possible variability in the reasoning learning and retrieval stages. The simulations were used to determine the best value for each parameter to get the best outcomes. Therefore, the model’s flexible architecture can be combined with fuzzy logic and the GA to offer an alternate viewpoint on potential random dynamics for applying practical bioinformatics issues.

Figure 7 and Figure 8 show the global and local minima ratio detected by the other two networks for different numbers of neurons. A correlation between the global and local minima values and the type of energy gained in the program’s final section was found by Sathasivam [3]. The outcomes in the system have arguably reached global minima energy, except HNN-3SAT, given that the suggested hybrid network’s global minima ratio is approaching a value of 1. 3SATFuzzyGA certainly can offer more precise and accurate states when compared to the HNN. The outcome is due to how effectively the fuzzy logic algorithm and GA searches. The HNN-3SATFuzzyGA solution attained the best global minimum energy of value 1. The obtained results are from utilising the HNN-3SAT network with the fuzzy logic technique and the genetic algorithm. The fuzzy logic lessens computation load by fuzzifying and defuzzifying the state of the neurons to distinguish the right states. The suggested method can accept additional neurons. Additionally, unfulfilled neuron clauses are improved by utilising the alpha-cut process in the defuzzification stage until the proper neuron state is found. This property makes it possible for defuzzification and fuzzification algorithms to converge successfully to global minima compared to the other network. With the aid of the GA, the optimisation process is enhanced. When the number of neurons rises, the HNN-3SAT network gets stuck in a suboptimal state. It has been demonstrated that the fuzzy logic method makes the network less complex. From the results, HNN-3SATFuzzy and HNN-3SATFuzzyGA global minimum converged to ideal results with satisfactory outcomes. Calculating the time is crucial for assessing our proposed method’s efficacy. The robustness of our techniques can be roughly inferred from the effectiveness of the entire computation process. The computing time, also known as the CPU period, is the time it took for our system to complete all the calculations for our inquiry [3]. The computation technique involves training and producing our framework’s most pleasing sentences. Figure 9 shows the total computation time for the network to complete the simulations. The likelihood of the same neuron being implicated in other phrases increased as the number of neurons grew [3]. Furthermore, Figure 9 demonstrates unequivocally how superior HNN-3SATFuzzyGA is to other networks. The HNN-3SATFuzzyGA network improved more quickly than the other network after the number of clauses grew. The time taken by all techniques for more minor clauses is not significantly different. The CPU time was quicker when the fuzzy logic technique and the GA were applied, showing how effectively the methods worked.

In another application by Mansor et al. [35], the HNN can be used to optimise the pattern satisfiability (pattern SAT) problem when combined with 3SAT. The goal is to examine the pattern formed by the algorithms’ accuracy and CPU time. Based on global pattern-SAT and running time, the specific HNN-3SAT performance of the 3SAT method in performing pattern-SAT is discussed by Mansor et al. [35]. The outcomes of the simulation clearly show how well HNN-3SAT performs pattern-SAT. This work was done by applying logic programming in the HNN to prove the pattern satisfiability using 3SAT. The network in pattern-SAT works excellently as measured by the number of correctly recalled patterns and CPU time. As confirmed theoretically and experimentally, the network increased the probability of recovering more accurate patterns as the complexity increased. The excellent agreement between the global pattern and CPU time obtained supports the performance analysis. In another work by Mansor et al. [36], a very large-scale integration (VLSI) circuit design model also can be used with the HNN as a circuit verification tool to implement the satisfiability problem. In order to detect potential errors earlier than with human circuit design, a VLSI circuit based on the HNN is built. The method’s effectiveness was assessed based on runtime, circuit accuracy, and overall VLSI configuration. The early error detection in our circuit has led to the observation that the VLSI circuits in the HNN-3SAT circuit generated by the suggested design are superior to the conventional circuit. The model increases the accuracy of the VLSI circuit as the number of transistors rises. The result of Mansor et al. [36] achieved excellent outcomes for the global configuration, correctness, and runtime, showing that it can also be implemented with fuzzy logic and the GA.

9. Conclusions

The central insight of this study provides an efficient approach to creating a continuous search space that considers the optimal training phase (when the cost function is zero). The proposed model, HNN-3SATFuzzyGA, displays exciting behaviour when the network is hybridised between fuzzy logic and the GA. The application of fuzzy logic design in the first phase of the training phase demonstrates that the optimisation process is more effective at diversifying the neuron state in a bigger search space. The second contribution is designing a hybridised network between the fuzzy logic system with the GA as the second phase in the training phase, namely HNN-3SATFuzzyGA, to determine the global solutions. The effectiveness of the suggested model was then effectively contrasted with other methods. The GA performs well since it requires a few iterations to maximise the global solutions. The execution of evaluation metrics discussed in this article proves that the findings of the suggested model, HNN-3SATFuzzyGA, are excellent compared to the HNN-3SAT and HNN-3SATFuzzy. The robustness of HNN-3SATFuzzyGA architecture offers insight into potential dynamics for usage in real-world issues. The proposed HNN-3SATFuzzyGA can be included in logic mining to extract the best logical rule for real datasets problems. The sturdy architecture of HNN-3SATFuzzyGA offers a solid foundation for use in real-life settings. For instance, the proposed HNN-3SATFuzzyGA in logic mining can extract the optimum logical rule for categorising the death occurrence based on comorbidity cases linked to COVID-19 disease. Other bioinformatics applications can also be implemented in logic mining to obtain the optimum logical rule for classifying the attributes, such as identifying the characteristic of a patient with Alzheimer’s disease, recognising an early sign of a heart disease patient and many more attributes. The HNN-3SATFuzzyGA can lead to a large-scale design with classification and forecasting capabilities. Additionally, for future work, fuzzy logic integrated with metaheuristic algorithms can be applied in solving other types of logical rules, such as Random 3-Satisfiability (RAN-3SAT) and Maximum 3-Satisfiability (MAX-3SAT).