3.2. Detection Using the MHS-BiLSTM Model
For deceptive content detection, the BAINLP-DCD technique uses the MHS-BiLSTM model. LSTM-NN is a kind of RNN that presents a ‘gate’ model. It is capable of capturing the long-term semantic dependence and preventing the gradient-disappearing problems of the classical RNN, due to its long sequence [
24]. Thus, the LSTM approach is applied for sentiment classification tasks. The computing method of the LSTM method is shown Equations (1)–(6).
Here, the input at time, the cell layer, the value of the forget gate, the value of the input gate, the value of the output gate, the layer of the candidate cell, and the outcome of the LSTM units are represented by and respectively. The sigmoid activation function is represented by , while indicates the dot product function among the weighted matrices and shows the parameter set from the LSTM unit. Hence, the BLSTM-NN technique is commonly utilized as the building block of the DL sentiment classification method to attain a better classification efficiency.
Word2vec translates the text data of the assessment information into a vector representation that is fed as an input into the BLSTM. The final result of the sample sentiment classification is attained by passing the BLSTM to the sigmoid layer.
Considering the sentence
, a pretrained model and a standard tokenizer are used to attain
dimensional embedding for a single word in the sentence, where
and
from the input to the model [
25]. It is necessary to recognize certain words to identify the sarcasm in sentence
that provide relevant clues like negative emotions and sarcastic connotations. The importance of such a cue word corresponds to various factors based on dissimilar contexts. In this work, Multi-Head Self-Attention (MHSA) is leveraged to detect the cue words from the input texts.
Here, the attention module is used to determine the design in the input that is critical for determining the presented task. The self-attention model helps in learning a task’s particular connection amongst various modules to generate the best series representation. In this self-attention model, three linear projections exist, where
Key (
), Value (
), and Query (
) of the given input order are created. The attention map is calculated according to the comparison among
,
, and the outcomes of these modules.
denotes the scaled Dot-product between
and learned softmax attention
as explained in Equation (7) as follows.
Various copies of the self-attention model are utilized in parallel to MHSA. Every head captures the dissimilar connections among the individual keywords that support classification and identifies the words from the input text. In this work containing different heads
in all the layers, a sequence of MHSA layers
is used.
Figure 2 demonstrates the framework of the attention BiLSTM layer.
3.3. Hyperparameter Tuning Using AVOA
In the current study, the AVOA is used for optimal fine-tuning of the hyperparameters for the MHS-BiLSTM model. AVOA is a new meta-heuristic swarm-based optimization approach inspired by the hunting style of the African vulture [
26]. The African vulture is a kind of hunter that preys upon weak animals as its food. The AVOA is particularly inspired by its feeding and orienting behaviors. The algorithm consists of powerful operators while it also maintains a balance of exploration and efficiency in solving the continuous optimization problems.
In this method, there is an
count of the population of vultures and its values are adjusted to suit the problems that need to be resolved. The fitness of the vultures is measured after its arbitrary initialization. The best vulture is the vulture with the optimum solution, chosen to lead the first group; the second-best vulture is the vulture with next best solution, chosen to lead the rest of the groups. The remaining population is disseminated to make up both the groups, as in Equation (8). By applying the roulette wheel mechanism, the probability of choosing the group is calculated. The
and
parameters are predefined parameters so that the value lies in the range of [0, 1], where their sum is equivalent to
. When
is closer to
and
lies near
, the intensification increases in AVOA. If
is closer to 1 and
is closer to
, this increases the diversification. The following equation provides more details in this regard.
The starved vulture becomes aggressive; this stage defines the starvation rate of the vulture. A satiated vulture has abundant energy to travel a long distance, foraging for food; when it becomes starved, it becomes highly aggressive and finds food near other vultures. Equation (9) computes the satiation rate, which switches between the exploration and exploitation stages. Equation (10) ensures that the exploration stage reaches the exact estimate of the overall optimal solution and also that no early convergence takes place.
Here, is the rate of starvation, represents the existing iteration, shows the max iteration count, and indicates a random integer within []; if is negative, the vulture is starved, and if positive, the vulture is satisfied. denotes a random integer in [] and shows the random value within []. The last iteration of the AVOA performs the exploitation and exploration stages.
The transformation between the exploration and exploitation phases can be accomplished by shifting the probability of entering the exploration stage at the last stage. If the parameter reduces, then the probability of turning towards the exploration phase in the end stage reduces. At last, if is less than 1, then the vulture is starved and finds prey in the following region, and accordingly, the AVOA enters the exploitation stage. If exceeds one, then the vulture is satisfied and pursues food foraging in other regions. Therefore, the AVOA enters the exploration stage.
In the exploration stage, the vulture can find food and travel for a long distance. The vulture identifies a new area based on the satiation level, and a parameter
is used for the selection process, as shown in the Equations (11) and (12). Two strategies are used to ensure a wider exploration of the search range. The initial strategy considers the exploration of the search region, adjacent to a better vulture from the group. This technique enables the localized exploration from the neighborhood of the present optimum performance. On the other hand, the next approach allows exploration across the whole search range while following the specific upper and lower limitations. This method permits a wider exploration range without surpassing the limits.
Here denotes the random integer that lies in the range of [], indicates the leader, and shows the rate of starvation, as set out in (9). co-efficient is used for increasing the arbitrary movement and alteration with all the iterations. and indicate the upper and lower boundaries, respectively. , , and are random integers within [0, 1].
When the rate of starvation is
, then it enters the exploitation stage, which has two phases. If the values of the rate of starvation range between 1 and 0.5, then the AVOA technique enters the first exploitation stage. However, the vulture remains comparatively satisfied. At this point, a random number
is created in [0, 1] for choosing that specific approach to follow, and later, it is compared to a predetermined parameter
. The siege fight strategy is applied if
is equal to or greater than
. In this strategy, the vulture behaviors are simulated, i.e., when a strong vulture refuses to share food, the weaker vultures tend to tire this strong vulture out by surrounding and attacking it. The rotating flight is chosen if
is less than
i.e., spiral motion of a vulture while finding food. Equations (13) and (14) represent the siege and rotation flights, respectively.
Here is evaluated in (10), denotes the rate of starvation, indicates the vulture’s leader, stands for the present location of the vultures, and , and denote the random numbers within [0, 1], which, in turn, increase the arbitrariness. The values of or form an array vector of dimension , whereas indicates the count of units generated.
If
, the vulture is starved and aggressive in the second exploitation phase. Firstly, a random integer
within [0, 1] is compared to a predetermined parameter
. When
is equal to or greater than
, different kinds of vultures accumulate over the food. In other terms, an aggressive siege fight is selected as follows.
Here, the first and the second best vultures in the existing iteration are represented as
and
respectively.
is the problem dimension.
denotes the existing location of the vultures.
, and
were described earlier, while the levy flight increases the efficacy of the AVOA and is evaluated as follows.
In Equation (17), is a set number that is equivalent to 1.5, whereas and are randomly generated values within [0, 1].
The AVOA system progresses a Fitness Function (FF) to realize the best classifier solution. It defines a positive integer to denote the best solution for candidate efficiencies. In this case, FF corresponds to the reduction in classifier errors, as expressed in Equation (18).