Multimodal Age and Gender Estimation for Adaptive Human-Robot Interaction: A Systematic Literature Review

Abstract

Identifying the gender of a person and his age by way of speaking is considered a crucial task in computer vision. It is a very important and active research topic with many areas of application, such as identifying a person, trustworthiness, demographic analysis, safety and health knowledge, visual monitoring, and aging progress. Data matching is to identify the gender of the person and his age. Thus, the study touches on a review of many research papers from 2016 to 2022. At the heart of the topic, many systematic reviews of multimodal pedagogies in Age and Gender Estimation for Adaptive were undertaken. However, no current study of the theme concerns connected to multimodal pedagogies in Age and Gender Estimation for Adaptive Learning has been published. The multimodal pedagogies in four different databases within the keywords indicate the heart of the topic. A qualitative thematic analysis based on 48 articles found during the search revealed four common themes, such as multimodal engagement and speech with the Human-Robot Interaction life world. The study touches on the presentation of many major concepts, namely Age Estimation, Gender Estimation, Speaker Recognition, Speech recognition, Speaker Localization, and Speaker Gender Identification. According to specific criteria, they were presented to all studies. The essay compares these themes to the thematic findings of other review studies on the same topic such as multimodal age, gender estimation, and dataset used. The main objective of this paper is to provide a comprehensive analysis based on the surveyed region. The study provides a platform for professors, researchers, and students alike, and proposes directions for future research.

1. Introduction

With the world’s latest tremendous development, technological advancement, and information age, we started accessing this study which is referred to the main terms in the study, i.e., each of Multimodal Age Estimation [1,2]. Typically, it is more challenging to assume the age of a speaker based on their speech [3], Gender Estimation [4], and Human-Robot Interaction [5,6,7]. The first term, Multimodal, refers to the theory of communication between social auditors that represent communication between audiovisual, visual, and spatial resources. The second term is age and gender estimation. Therefore, many studies have presented Multimodal in meta-learning [8], English language [9], a comprehensive presentation of Vocal sacs [10], deep learning fields of vision, language, and speech [11,12,13,14,15].

One study [16] used three-dimensional image analysis using mandibular third molar apices to estimate age which also includes gender, race, and age image style transfer [17]. The age and gender estimations were based on a large database [18], in which the Gender determination was conducted based on pain [19]. Age and gender estimation were conducted by sphenoid bone pterygoid processes [20] and the Deep Residual Learning Network [21]. In addition, the analysis of race was conducted in [22] the Frequency domain [23]. Infants’ neural speech processing [24], brain aging [25,26], and old speakers’ ability to understand one another [27]. Emotion recognition [28,29,30]. The conversion of audio to textured visuals to recognize speech emotions [31,32]. The third term of the Human-Robot Interaction (HRI) is a promising technology in service, social, and industrial perspectives. Therefore, developers of HRI systems should evaluate the effectiveness of the proposed system in terms of its function and whether it will satisfy individual, group, and production requirements (i.e., its quality). Engineering places a lot of emphasis on the concept of quality, which generally indicates how well a system, service, product, or service provider process complies with established requirements and operates within established parameters and conditions. [33,34,35,36,37,38,39,40,41,42,43].

Physical human-robot interaction has emerged as a critical issue in many fields, including human–robot collaboration learning by demonstration and rehabilitation [33,39,40,41,42,43,44,45].

Multimodal is a mode of communication that includes Sign Language, Writing, Typing, Body Language, Sign Language, Speech & Vocalizations, Gestures, Facial Expressions, High Tech AAC systems, Light Tech Devices, and Low-Tech boards.

Biometrics is used as an identifier for an individual using physiological data or behavioural attributes. Everyone is known to be unique, therefore, biometric identifiers are permanently associated with the user. As a result, they are more trustworthy than token or knowledge-based authentication techniques, as illustrated in Figure 1. There are three classifications of biometric modalities such as:

1.1. Physical Biometrics

The unique modality of an individual includes face, fingerprints, iris scans, and hand geometry (see Figure 1). [46].

1.2. Chemical Biometrics

One of the recent areas that involves the estimation of chemical components associated with humans, such as odor and perspiration.

1.3. Behavioral Biometrics

These are commonly temporal in nature and involve analyzing how a user accomplishes a specified task. Examples of modalities include speech, signature, functional mobility, and keyboard dynamics.

This study represented a comprehensive presentation of the study of literature, with 48 discreet scientific articles within a specific period. It is a comprehensive guide and assistant for researchers in the same field that included symptoms of technologies, data, processing methods, analysis, and extraction of features in four important databases, Web of Science database, IEEE Xplore, Science Direct, and Taylor & Francis Group, (25%, 12%, 56%, 6%), respectively.

This paper is outlined as the following, Section 2 Relevant Works represents and compares our A Systematic Literature Review study to the existing literature studies. Section 3 is the Method used in SRL and provides the concept of the study and Research Methodology. Section 4 Results. Section 5, evaluation metrics includes the categories of the studies and discusses a comprehensive and detailed, Section 6, challenges and limitations, Section 7, view of the dataset used in this field; and Section 8 Conclusions of paper review.

2. Relevant Works

This section discusses the existing literature under the scope of multimodal age detection and gender estimation associated with our main research. The main topics included are current systematic reviews, surveys, and reviews. In this work, statistics pertaining to the use of the reviewed studies were also listed (e.g., features, research design, feature selection, deep learning, dataset, speech-dependent, text-dependent, multimodal, language, protocol, and feature direction). The authors also discussed a variety of age and gender estimates and offered some recommendations for potential future research areas, although only in part.

Table 1. Comparison between the proposed survey and existing literature.

Survey Paper	Year	Features	Research Design	Feature Selection	Deep Learning	Dataset	Speech Dependent	Text Dependent	Multimodal	Language	Protocol	Feature Direction
[47]	2018	√	√	√	√	√	√	√	√	√	√	√
[48]	2019	√	$\times$	$\times$	$\times$	√	√	$\times$	√	$\times$	$\times$	$\times$
[49]	2019	√	√	$\times$	$\times$	√	√	$\times$	√	$\times$	$\times$	$\times$
[6]	2020	√	$\times$	$\times$	$\times$	√	√	$\times$	√	√	$\times$	$\times$
[50]	2020	√	$\times$	√	√	√	√	$\times$	√	$\times$	$\times$	√
[51]	2020	√	√	$\times$	$\times$	√	√	$\times$	√	√	$\times$	√
[52]	2020	√	√	$\times$	$\times$	$\times$	$\times$	$\times$	√	$\times$	$\times$	$\times$
[53]	2020	√	√	√	√	√	√	√	√	$\times$	√	$\times$
[54]	2020	√	√	$\times$	$\times$	√	√	$\times$	$\times$	$\times$	√	$\times$
[5]	2021	√	√	$\times$	$\times$	√	√	$\times$	√	$\times$	$\times$	$\times$
[55]	2021	√	√	$\times$	$\times$	√	√	$\times$	√	$\times$	$\times$	$\times$
[56]	2021	√	√	$\times$	√	√	√	$\times$	√	√	√	√
[57]	2022	√	√	$\times$	√	√	√	$\times$	√	$\times$	√	$\times$
This Survey	2022	√	√	√	√	√	√	$\times$	√	√	√	√

represents and compares our A Systematic Literature Review study to the existing literature studies.

Table 1 describes the significance of our work against other existing work in terms of various metrics, such as features, research design, feature selection, deep learning, dataset, speech-dependent, text-dependent, multimodal, language, protocol, and feature direction. The authors studied and reviewed the body of knowledge on age and gender estimation before establishing these parameters. This work includes a comprehensive discussion of the most recent developments in the field and is compared to 13 prior relevant studies. The scope of the study shows that speech recognition is the extraction of data that represents the identity of the speaking person, to become the person who can, through his voice, reach a set of services, control systems, or databases.

3. Method

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

This study designed a group of questions. The study covered the review of inputs, methods of treatment, analysis, and directing of features to the results. We also offered some of the techniques and algorithms. The methods used in the process contribute to each study. The methods were distinguished by the collection of properties and methods of treatment and access, and the presentation of the offer is the most accurate result.

The requirements asserted in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses must be fulfilled (PRISMA) [58,59,60,61,62]. The current study was carried out, meanwhile, PRISMA advises against depending solely on a database. When doing a systematic review, look for the literature because it is unlikely that any database would include all pertinent references. Consequently, much study is practically required, and it is possible to conduct an accurate and detailed survey on several databases to include a variety of articles. This study covers five comprehensive questions for each:

• RQ1: What methods were used and what techniques in data analysis are age estimation and gender knowledge?
• RQ2: Does the data depend on the multi-means of the species? What kind of data is used to estimate age and known gender? Do you rely on multimedia?
• RQ3: What are the challenges? Types, methods of conclusion, challenge it, classify it. Ways to overcome them, how to overcome the challenges?
• RQ4: What essential features of the datasets employed in this investigation are there? Do their traits seem to have an impact on the outcomes?
• RQ5: What are the probable difficulties that exist in the studies in developing an adaptive human-robot interaction’s multimodal age and gender estimation?

The most pivotal keywords, which are listed below in italics, describe the scope of this study. Multimodal, Multimodality, Multimedia, Multi-media, Combined, System can recognize, Age and Gender Estimation, Gender Age and Estimation, estimate, rating, assessment, appreciation, respect, Adaptive, Adapt, Adjust, fit, conform, Human-Robot Interaction, interaction man-robot, Robot-Human Interaction, Data mining, Deep learning, Speech, Voice, and sound. Specifically, our study is limited to English-language publications. Our study included only selected digital databases such as Scopus, the Web of Science (WoS) database, Taylor & Francis Group, IEEE Xplore, and Science Direct. This research was conducted using our academic search engine accounts to gain access to all open and closed access papers. The first iteration excluded all duplicated articles and selected only articles published prior to the last five and a half years, from 2016 to 2022 (completed June 2022). In this study, the Mendeley-Desktop 1.19.4-win32 was used to collect the papers. The second iteration sorted the articles by titles, abstracts, and articles outside of our domain. The third iteration filtered all articles by reading the full text. Unrelated articles and those that did not meet our research requirements were eliminated.

In this study, we searched the Web of Science (WoS) database, IEEE, Science Direct, and Taylor & Francis Group, using the search string ((Multimodal OR Multimodality OR Multimedia OR Multimedia OR Combined and System can recognize) AND (Age and Gender Estimation OR Gender Age and Estimation) AND (estimate OR rating OR assessment OR appreciation OR respect) AND (Adaptive OR Adapt OR Adjust OR fit OR conform) AND (Human-Robot Interaction OR interaction man-robot OR Robot-Human Interaction) AND (Data mining OR Deep learning)) AND (Speech OR Voice OR sound)). We could not find any Systematic Literature Reviews during our thorough investigation between 2016–2022 that focuses on Age and Gender Estimation for Adaptive Human-Robot Interaction.

3.1. Properties Criteria Search (Eligibility Requirements)

We started to input keywords, and the words of their approach to them in the same sense, to reveal all articles in the same field and within the specified period (see

Table 2. Original words and alternative words used extensively in the study.

Original Words	Alternative Words
Multimodal	Multimodality Multimedia Multi-media Efficiently manage Combined gdp System can recognize
Age and Gender Estimation	Age and Gender estimate, Age and Gender rating, Age and Gender assessment, Age and Gender appreciation, Age and Gender estimation, Age and Gender respect, Age appreciation and gender appreciation
Adaptive	Adapt Adjust Fit Conform
Human-Robot Interaction Robot-Human Interaction	Interaction man—robot. The interaction between man is a robot. The interaction between a human robot.
Speech	Voice, sound

). Any article does not apply to the conditions below. We included the papers in Figure 2 that matched our criteria based on our analysis. The primary goal is to address the overall scope of the study. The coarse-grained taxonomy has two levels. Categories identified by prior research in the unrestricted literature (only three categories were emphasized in this study). We examined two websites to study the directions cited in the literature. After that, we excluded the articles through three cycles in order to eliminate duplicate papers across databases. When the items failed to fulfil our eligibility requirements for filtering and screening, the following conditions had to be met in order for an article to be disqualified: (1) it must be clear and focus on a single issue; (2) it must be written in English; (3) it must be redundant with other research websites, one of which had to be disqualified; and (4) it must concentrate on the overview. Therefore, we emphasize our efforts on the domains discovered in Figure 2 since they include a wealth of information relevant to the investigation.

3.2. Process for Gathering and Taxonomy of Data

The information was gathered from the four scientific websites—WOS, IEEE Xplore, Science Direct, and Taylor & Francis—and secured using the authors’ user credentials. The preliminary screening process was then carried out, consisting of the following three steps: First, duplicates are eliminated, then titles and abstracts are examined, and finally, the full texts are reviewed. Last but not least, the Mendeley-Desktop-1.19.4-win32 software, Word, and Excel were used to establish initial classifications, summaries, and infographics based on their materials after reading, evaluating, and summarizing the articles. All academic articles that followed the taxonomy were divided into two groups. Then, it concentrated on three major categories, including the 48 research papers, models, parameters, type, input, output, average accuracy, and problems, that extensively addressed the issue of multimodal Age and Gender Estimation. We will talk about the other categories in another article.

4. Results

The study recommended that each of the Web of Science database (577), IEEE Xplore (142), Science Direct (376), and Taylor and Francis Group (289), obtain a total of 1384 articles from research sites that were approved in the study between 2016 and 2022. However, two scientific articles from WoS could not be downloaded due to accessibility issues. A total of 546 papers were published in the most recent six years (from 2016 to 2022), and 368 papers appeared in all three databases, generating 178 papers. The articles were filtered using two categories and the research’s sequence. Following a careful examination of their titles and abstracts, 67 more papers were eliminated. A total of 53 papers were in the complete set. Five additional research papers were not uploaded after that. As a result, the study now includes 48 academic articles in its entire sample. The study covered two major fields that were divided into two subfields: Multimodal Age and Gender Estimation, and Human-Robot Interaction. The first category consists of 13 papers (27.083%) such as review, survey, and systematic review. Meanwhile, the second category consists of 35 papers (72.91%) that include techniques and algorithms used for age and gender estimation for adaptive Human-Robot Interaction.

4.1. Distribution Outcomes

Figure 3a demonstrates that the four digital databases published many publications. IEEE Xplore published six papers, Web of Science published 12, Science Direct published twenty-seven, and Taylor & Francis published three. Next, we extracted the percentage number of articles from the total number of features in four important databases, Web of Science database, IEEE Xplore, Science Direct, and Taylor & Francis Group (25%, 12%, 56%, 6%), respectively, shown Figure 3b.

4.2. Distribution by Articles’ Publication Years

Figure 4 shows the distribution of the articles from 2016 to 2022 by the years of their publication. Approximately, there have been five published since 2016. In 2017, four articles were published. Since 2018, 23 articles have been published. Since 2019, one article has been published, three have been published since 2020, ten have been published since 2021, and two have been published until May 2022. The lack of articles created in 2019 and 2020 can be attributed to the spread of COVID-19, while the year 2022 included the first four months of it.

Figure 5 shows how triage is used in 25 different countries and nationalities. We discovered that certain countries were the focus of literature studies or the circumstances they addressed. The most productive writers come from the United Kingdom (UK), Germany, the United States (USA), Brazil, Australia, Malaysia, Spain, Japan, Iraq, Indonesia, Italy, Turkey, Norway, Iran, Switzerland, Canada, Hungary, India, Francis, Mexico, Chile, Singapore, South Korea, and Switzerland, in that order, in terms of both numbers and percentages (1 each).

In Figure 6a, the word that appears the biggest in a word cloud reflects the term that occurs the most frequently in the dataset, whereas the word that appears the smallest in a word cloud represents the word that occurs the least frequently in the dataset. While it represents Figure 6b, which was developed to assist instructors in managing the demands of academic language and vocabulary in their text resources.

4.3. Classification

Studies differed from one individual to another by the method of treating inputs, and by what are the approved parameters on which the study was based in terms of input, model, parameters, types, and many other various factors. Some of them used accurate and other learning algorithms. Before entering the topic, it is necessary to include an introduction to deep learning, machine learning, and many methods of age and gender estimation.

4.4. Artificial Intelligence (AI)

Today, artificial intelligence (AI) is known as a fast-expanding discipline that is engaged in various subdomains and adapted to many practical applications. Over decades, AI has been associated with various applications such as self-driving cars, speech and image recognition, machine translation, as well as competing against world chess champions [63].

Some AI systems can learn without being explicitly programmed by extracting patterns from raw data [64]. This capacity is known as Machine Learning (ML). Al-based systems learn through trial and error and develop over time, in contrast to conventional programming that follows step-by-step coding instructions based on logic, if-then rules, and decision trees [65]. A subsidiary of machine learning called deep learning (DL) makes use of artificial neural networks (ANN) with many hidden layers.

A.

Machine learning (ML)

Computer algorithms that can learn to perform tasks more effectively based on experience are the subject of machine learning (ML). Essentially, it is related to statistical analysis and pattern recognition. ML has become increasingly mathematical and successful in applications over the last 20 years (Makridakis, Spyr). ML has a wide range of applications, including web searches, commercials, credit scores [66,67], stock market forecasting [68], gene sequencing analysis, behavioral analyses, time forecasting, and in the analysis of large amounts of data. Artificial neural networks have been used to predict time series in recent decades due to ideal properties that allow for nonlinear models to be used [69]. Similarly, the artificial network continues to grow in terms of the development of applications that make it easier to work with when performing simulations with networks.

B.

Deep learning (DL)

This is known as an automatic learning technique that uses many layers of nonlinear information processing to extract and transform characteristics with and without supervision, as well as analyze and classify patterns.

An artificial neuron is modeled after a biological structure that serves as a basic information processing unit. An artificial neuron is made up of a linear combination of weights and input values that is processed by activation functions. As shown in Figure 7a, the classical artificial neuron is a binary classifier from a supervised learning algorithm [70].

The simplest ANNs generate backward propagation, which allows for better learning patterns to be obtained over time. Therefore, ANNs can match a certain pattern (some pattern of data) over time. f represents the activation function of this unit, which corresponds to the function chosen to convert input x to output value y = f(x,w), and it depends on the specific characteristics of each network.

$$Y_t=f\left(T_t,S_t,E_t\right)$$

(1)

$$y=f\left(x,w\right)$$

(2)

where x = input neural network, w = weight.

The basic structure of artificial neurons is a multilayered network. This structure is commonly implemented, and it consists of the first level being the input neuron layers, which receive the values of some of the patterns represented as vectors and act as the input of the network. Then, many hidden layers whose modules respond to specific features may appear on the input patterns; additional hidden layers may be present as well. The final level is the output, which is presented as the output of the entire network; this network is known as a multi-layered or neural network, (see Equation (3)).

$$f\left(x\right)=f^n\left(f^{n-1}\dots .\left(f^{1\left(x\right)}\right)\right)$$

(3)

where n = number of hidden layers.

The output or forecasting values must be validated by comparing the target to the output values to learn the neural network. The training objective is to validate the forecasting and reduce errors using the loss function. The forecasting is more accurate when the error is lower. Gradient descent is a common improvement approach for improving the loss function.

As presented in Figure 7b, the neural network consists of three primary components (i.e., input layer, hidden layers, and output layer). Firstly, the input layer is used to apply the input data to the network to compute the required results. With one layer, the input layer could have multiple inputs. Secondly, the hidden layers could have one or more layers, depending on the complexity of the network, and they are used to compute the required results as well as train the neural network. Finally, the third layer, which consists of one layer identical to the input layer, is used to generate the computed results. The loss function could be computed to evaluate the network’s performance. The forecasting results are compared to the defined target results using the loss function. An example is the mean square error (MSE), which is utilized to compute the difference between the computed and target results. A small difference between the computed and target results refers to the highest accuracy and optimum forecast results, while a large difference refers to the poverty of the technique to produce optimum results that are closely defined to the target. Therefore, the relation between the MSE output and the accuracy is inverse, which means that a small MSE value refers to the optimum forecasting network results. (See Equation (4)).

$$l(y-y^{\prime })=\frac{1}{n}\sum _{t=1}^n(y-y^{\prime })$$

(4)

where y = actual value (target), y′ = forecasting value, and n = number of targets.

Neural networks have an advantage in forecasting due to their adaptive power, but neural networks have limitations in time series because they cannot capture the temporal dependency that could be observed. Thus, RNNs are created.

C.

Recurrent neural network (RNN)

RNN was first introduced in the 1980s and is composed of three main layers: an input layer, one or more hidden layers, and an output layer. RNNs have a series structure as repeating units, with the idea of storing and managing information from previous processing steps in these units. In time series, RNN can learn the sequence and solve the dependency problem. Figure 7c shows a simple RNN with an input unit, an output unit, and a recurring hidden unit spread throughout the network, where xt is the input at t time and ht is the output at once.

During the training process, RNN utilizes the reaction propagation algorithm that is commonly used in the gradient calculation and modification of the matrices that will be modified after adjusting the feedback process. (See Equations (5) and (6)).

$$h^t=f(U{x}^t,W{h}^{\left(t-1\right)})$$

(5)

$$o^t=softmax\left(V{h}^t\right)$$

(6)

For this reason, this algorithm is also commonly called Back-Propagation Through Time (BPTT) [73].

D.

Long short-term memory (LSTM)

LSTM is a revolution in RNN. Hochreiter and Schmidhuber (1997) proposed the process to address the discomforts of RNN when additional cells are added, known as the ability to learn long-term dependencies and recall of information for extended periods of time [72,74].

In

Table 3. Classification based on input, model, parameters, types, and other factors.

Ref.	Year	Model	Parameters	Type	Input	Out Put	Average Accuracy	Challenges
[75]	2020	CNN	Six hidden layers,	Face recognition algorithm	GoogLeNet’s, 5328 images	Estimation	85%Age est., 98% gender esti. 98%	more difficult to classify the more detailed classes according to age.
[76]	2022	neural activity	Five categories, 300 dimensional semantic	Speech	speak 119 words in the MRI, FMRI raw data	Estimation	52%	similarity & language perception and language production in semantic representations
[77]	2018	hypotheses 1a & 1b	fundamental frequency (F0), Speech rate on age estimates.	Speech	50 participants, Arabic, Korean and Mandarin	Estimation	71%	non-native speakers
[78]	2019	Deep neural networks	DNN-GIF, QCP, AMR, CP, IAIF and CCD.	telephone speech	windowing, interpolate and overlap add	Estimation	___	glottal flow estimation
[79]	2020	PROSE	PESQ, SNR, SSNR, SDR, STOI, MSE, WE, IS, COSH, IS-II	Speech	clean speech NOIZEUS and NOISEX-92 database	Estimation	__	Risk Estimation, F16 noise, White noise, Street noise, Train noise.
[80]	2022	stabilized wavelet transform (SWT), Auditory mode, Cross-correlation, Transform SWM,	Spectral, just noticeable difference (JND, point of subjective equality (PSE),	speech	Japanese words (FW03), TANDEMS-TRAIGHT dataset	Estimation	___	weighting function,
[81]	2022	DNN, LSTM	Hesitations, Paranormal, filler words word fragments, word repetition	speech	text augmentation approaches by adult data, Web data, text generated by RNN, 33 children, LENA recording	Estimation, Speech Recognition	--	degree of spontaneity, include incomplete sentences. speech may be ill-formed, vocal effort
[82]	2021	ANN, niHEGP	artificial neural networks	speech	3 h, 2023 newspaper style sentences, 300 sentences [83,84]	Estimation	81%	noise
[85]	2018	MF VTLN, SRN, MFCC	Speaking-rate normalization, Zero-frequency filtering	speech	TANDEM STRAIGHT	Speech Recognition		spectrogram, Did speaking-rate normalization (SRN) highly effective?
[86]	2020	GMM, MFCC, short-time, STFT	Feature extraction, Mel filter bank features, UBM, RMSE, MAE	Speech, video	The TIMIT dataset, 630 speakers Audio Forensic Dataset (AFDS)	Estimation		extraction of information, speaker characteristics
[87]	2018	GMMs, MFCC	LPCCs	Speech, video	9 age groups	Classification		Textually neutral Hindi words have been used to construct text prompts
[88]	2020	HMM, Soft Computing	(i) Rules inference engine (ii) Fuzzifier (iii) Defuzzifier	Speech	Speech signal	Speech Recognition, Classification		how know ideas fuzzy HMM strategy, how know speech signal handling territory
[89]	2020	support vector machines (SVMs), time-domain, frequency-domain parameters	glottal parameters: time-domain, frequency-domain parameters, PCA	Speech	dysarthric speech database, 765 isolated word utterances (B1, B2 and B3) 255 words, 155 common words	Classification	72.01% Test data, 73.53% Validation data	classification, intelligibility estimation tasks compared,

, the classification represents studies according to the factors that were mentioned earlier by accreditation, models, and variables to display the main elements of output, accuracy, and challenges within the field of study.

Face-based age estimation algorithms are frequently used in biometric applications and other industries such as forensics and healthcare. To validate or estimate an individual’s age for security purposes, facial characteristics can be used to restrict access to physical or logical resources for that person. It will be indicated, clarified, and displayed in Figure 8 in our study.

In addition, in 2022, a study introducing random auditory stimulation was presented for the final time. Individual perception (RaS-DeeP) used Deep Expectation (DEX CNN) to achieve an EER of 3.3% for age estimation using Deep-learning-based age estimation [90].

On one hand, this aspect will be reviewed and explained by a display of models, dataset entered, versions, and sizes arrived in the results (see Table 3). On the other hand, a study used k-nearest neighbor (KNN) by gait energy images as to age estimation [91] and gait analysis [92].

5. Evaluation Metrics

They are standard criteria that are measured by the actual performance of the work. It is the level that must be the performance and arises usually after observation, monitoring, experiments, research, and testing. These criteria are determined in advance, so, there are two types of accuracy evaluation metrics, Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE) are used. In this study, the standards were tested on the news classification of the speaker (female/male) and will be described in detail [1,93,94,95].

A.

Mean Absolute Error (MAE)

Error, also known as MAE, is a metric that measures the number of errors in a forecasting set without taking the direction of the error into account (error greater or less than the true value). This is shown in Figure 9 (See Equation (7)).

$$MAE=\frac{1}{n}\sum _{t=1}^n|y_t-{y^{\prime }}_t|$$

(7)

B.

Root Mean Square Error (RMSE)

RMSE is a commonly utilized method for calculating the error of a model in forecasting quantitative data (See Equation (8)). It is formally defined as follows:

$$RMSE=\sqrt{\sum _{t=1}^n\frac{({y^{\prime }}_t-y_t)}{n}}$$

(8)

RMSE or MAE is one of the most used metrics for measuring the mean error between predicted and real values. The RMSE is more sensitive to outliers in the data because it calculates the mean of the squared errors.

6. Challenges and Limitations

Multimodal is a mode of communication that includes Language, Writing, Typing, Body Language, Sign Language, Speech and Vocalizations, Gestures, and Facial Expressions; High Tech AAC systems, Light Tech Devices, and Low-Tech boards.

In Figure 10, there are many challenges in the field of the study of the exact specialization represented by many difficulties which are in the role of finding solutions to them. It can be shortened by capturing the five as shown in Figure 8, which are devices, spatial challenges, time challenges, methods used, and other factors. The first devices contain type of devices, modernity number, and the number of speakers per second and scan image. The second spatial challenges contain the place, nature, delivery, and work environment. The third challenge contains the age of the speaker, their health, and psychology. The fourth method used contain RNN, CNN, DT, and another (see

Table 4. The summary of the related articles, their methodology, used dataset and their details, methodology and MAS (Female, Male and Mixed).

Ref.	Year	Dataset	Size	Type of Data	Methodology	MAE (Years)
						Female	Male	Mixed
[26]	2022	ADNI, ADNI, DLBS, 1000FCP, IXI, NITRC, OASIS, PPMI, SALD	2251	images	multi-feature-based networks			3.73
[2]	2021	VoxCeleb1	1211	speech	Statistical Functional, LDA, G- RNN	9.25	10.33	10.96
[96]	2016	aGender	659	Telephone Speech	i-vector, LS-SVR 3	9.77	10.63
[31]	2018	NIST SRE	220	speech	i-vector, Neural Networks LSTM RNN	9.85 9.56 7.44 6.97	10.82 10.69 8.29 7.79
[97]	2019	Morph-II, FG-NET	55,134 1002	Face images	DAG-VGG16 DAG-GoogLeNet			2.81 2.83
[98]	2021	MORPH II	55,134	Face images	Hierarchical Attention-based Age Estimation			2.53
[99]	2018	MORPH (Album2)	55,134	Face images	M-LSDML			3.31
[100]	2017	FG-Net MORPH Year-labeled.	1002 55,134 154,294	Face images	CNN			2.78
[101]	2016	Asian Face Age Dataset (AFAD)	160 K	facial images	OR-CNN/CNN			3.27
[102]	2022	MORPH II	55,134	facial images	MWR			2.00
[103]	2016	IMDB-WIKI	523,051	facial images	DEX(IMDB-WIKI)/CNN 2.68			2.68
[104]	2015	MORPH, FG-NET	55,134 1002	Face images	DLA CNN			4.77 4.26
[105]	2018	Morph-II	55,134	facial images	CNN + ELM			3.44
[106]	2017	FGNET, MORPH, FERET, PAL	1002, 2000 2366, 576	facial images	CNN			5.39, 3.98 3.00, 3.43
[107]	2016	IMDB-Wiki	500K	Face images	CNN			2.99

, Methodology column). Finally, other factors include the used dataset, variables, sample size, and techniques.

7. Dataset

In this section, the data were reviewed. We were able to divide the data into two versions, A and B; the first contains the TIMIT Dataset, the HKUST Dataset, the SRE08/10 Dataset, the TCDSA Dataset, the JNAS Dataset, the AgeVoxCeleb dataset, and the AgeVoxCeleb2 dataset. As for the second group, it contained the utch corpus dataset, the UF-VAD dataset, the aGender dataset, and the NIST SRE dataset. Each of the arguments was reviewed separately, the form of the components of each category, its parts, numbers, types, sizes, and studies that used this category data groups. Mono-media and multi-means.

All data contain different forms and types, including speaking, sound, and others including videos and pictures. All of these are aimed at reaching the assessment of Age and Gender Estimation. It represents real data.

A.

Group A Dataset

1. TIMIT DATASET

The purpose of the TIMIT read speech corpus is to develop speech data for acoustic-phonetic research as well as for the development and evaluation of automatic speech recognition systems. Broadband recordings of 630 speakers reading ten phonetically dense phrases in eight significant American English dialects are included in TIMIT. Each utterance in the TIMIT corpus has a 16-bit, 16-kHz speech waveform file, and time-aligned orthographic, phonetic, and word transcriptions. Texas Instruments recorded the lecture, MIT transcribed it, and the National Institute of Standards and Technology inspected and prepared it for CD-ROM production (NIST) [1,2,4,108,109,110].

2. HKUST DATASET

The Hong Kong University of Science and Technology (HKUST) created HKUST Mandarin Telephone Speech, which comprises roughly 149 h of Mandarin conversational telephone speech (CTS). Although Standard Mandarin is not a natural dialect in the majority of China, it is considered the official language of instruction. Mandarin speakers can have regional accents or not. All calls were audited and classified as standard or accented without further distinction, and subjects’ birthplaces were categorized into Mandarin-dominant and non-Mandarin-dominant regions. This dataset was used in several cities across China. Most of the participants had never met before. For this purpose, topics similar to Fisher English were developed to engage in a smooth conversation. All calls were initiated by an automated operator dialing two participants at the scheduled time of contact. Participants were asked age, gender, native language/dialect, education, employment, phone type education, and other demographic questions (refer to

Table 5. Type And Number.

Grope	Number of Calls	Number of Hours	Females	Males
Training	873	144.7	797	948
Development	24	3.9	24	24
Total	867	148.6	821	972

). It denotes the type and number [111,112,113].

3. SRE08/10 DATASET

There are two types of SRE Datasets, namely the SRE08 Dataset and the SRE10 Dataset. From 2008 to 2010, revisions and modifications were made. The 2008 NIST Speaker Recognition Evaluation Test Set was published in cooperation with the Linguistic Data Consortium (LDC) and NIST (National Institute of Standards and Technology). It consists of 942 h of multilingual telephone conversation, English interview speech, transcripts, and other materials that were utilized as test data in the 2008 NIST Speaker Recognition Evaluation (SRE). The NIST SRE dataset is a sequence of NIST evaluations that are essential in determining the focus of research and the apex of technological development. They are tailored toward academics with a general interest in text-independent speaker detection. The interview is conducted in English the entire time. About 368 h of the dataset were accounted for by telephone speech, and the remaining 574 h were accounted for by microphone speech [31,114,115,116].

4. TCDSA DATASET

The Trinity College Dublin Speaker Ageing (TCDSA) Database was created with the aim of learning more about the influence of age-related voice changes on speaker verification. A major part of the collection comprises speech recordings from 26 individuals (15 males and 11 females) whose range of age starts from 25 to 58 years. A collection of 120 developing speakers is also offered, with a mix of ages, genders, and accents. [117,118].

5. JNAS DATASET

The Acoustical Society of Japan’s Project the Speech Database Committees. The following are the contents of newspaper article sentences: 155 text sets (each with roughly 100 sentences), for a total of 16176 sentences. The 503 phonetically balanced phrases in ATR are as follows; there are a total of 503 sentences in ten text sets (each with roughly 50 sentences). There are 306 speakers in all (153 males and 153 females). The environment for recording is a head-set microphone, and a desk-top microphone is included. WAV is a digital audio format (16 kHz, 16-bit, Mono) and is formatted for audio files [119,120,121].

6. AGEVOXCELEB DATASET

The VoxCeleb dataset is an audio-visual collection of short snippets of human speech collected from YouTube video interviews. It has over 7000 clips. Speakers VoxCeleb features almost 2000 h of speech from speakers of all races, accents, professions, and ages. VoxCeleb is a combination of voice and visual apparatus. Each section lasts at least three seconds. There is a total of 1,000,000 utterances. All speaking face tracks, as well as laughing, with background chatter, laughter, different lighting conditions, overlapping speech, and pose variation. Gender was distributed as follows: 61% male and 39% female. The countries of the speakers include the following, the United States, the United Kingdom Germany, India, France, and Unknown, 29%, 10%, 6%, 6%, 6%, 7%, respectively [3,122,123,124,125].

7. AGEVOXCELEB2 DATASET

This dataset consists of 5994 speakers, 145569 videos, and 1092009 utterances. It is a giant database of more than a million utterances from 6112 celebrities collected from YouTube videos for VoxCeleb2. The IDs in the VoxCeleb2 development set do not overlap with those in the VoxCeleb1 or SITW datasets [124,125,126].

B.

GROUP B DATASET

1. DUTCH CORPUS DATASET

A huge dual set includes 425 speakers from the N-best training corpus’s Flemish section which includes news, interviews, live interviews, read commentary, and reports from Belgium. The corpus contains all of them. Figure 11 displays the age histograms for both male and female speakers [127,128].

2. UF-VAD DATASET

The University of Florida Vocal Aging Database (UF-VAD), a corpus of American English that was taped between 2003 and 2007, provided the information for the tests. The collection includes 1350 read spoken utterances from 150 different speakers that are based on well-known sources like the Rainbow Passage, Grandfather Passage, and SPIN phrases. A corpus with a duration of 5 h consists of a participant who reads 2 min of the same material using the same microphone and recording settings. The contributing speakers are similarly evenly distributed among the three main age groups and genders. In this instance, there are 25 male and female speakers in the young (18–29), middle-aged (40–55), and elderly (over 55) age brackets (62–92). There are 50 delegates for each age group and 75 delegates for each gender. The average age for each age group is 21, 48, and 79, respectively [129,130].

3. AGENDER DATASET

Felix Burkhardt et al. presented the aGender corpus in A Database of Age and Gender Annotated Telephone Speech, which consists of audio recordings of free speech and predetermined utterances made by people of various ages and genders. As shown in Table 5, each utterance is categorized into one of three gender categories—male, female, or child—as well as one of four age categories—children (C), youth (Y), middle (M), or senior [96,131,132,133,134] (Refer to

Table 6. Agender dataset (age and gender).

Class No.	Age	Gender	Age Group
1	7–14	Male + Female	Children
2	15–24	Female	Young
3	15–24	Male	Young
4	25–54	Female	Male
5	25–54	Male	Male
6	55–80	Female	Seniors
7	55–80	Male	Seniors

). The dataset is summarized as 659 speakers (291 males and 368 females).

4. NIST SRE DATASET

The most enormous and comprehensive dataset for telephone speaker recognition currently available is the NIST SRE CTS Superset, which is used to compile earlier SRE datasets (SRE1996-2012). Overall, there are 605,760 segments with 6867 voices (2885 male and 3992 female). As some speakers appear in multiple source corpora, the total number of speakers in the table exceeds 6867. Each speaker has at least three sessions or calls, with each section containing somewhere between 10 and 60 s of speaking. The CTS Superset contains more than 50 languages, even though English is spoken in the vast majority of segments (both native and accented English) [31,135].

8. Conclusions

This work suggested a comprehensive framework literature review where the methods and techniques of several studies were clarified during the study period and show the types of data used in this field. It also identified some studies that used automatic learning and interaction between humans and robots, and the study showed the study environment and displayed MAE. In a systematic study of Age and Gender Estimation, the approach was presented to determine the main challenges and restrictions in the implementation of the double-stage research. We studied accurately many studies to show the benefits, challenges, and recommendations using accurate data, Age and Gender Estimation, and we identified several gaps.

This research provides a broader base for modern literature directions by data analysis, determining their sizes and knowing their type, up to the results of the disposal in research papers in Age and Gender Estimation Studies to obtain the deepest visions in the investigation area. The goal is to encourage academics and practitioners to use multimedia analysis that includes speech and images and rely on them to reach a lifetime and select sex for a precise image. Moreover, it determines the inputs on which the review operations relied upon for the most accurate results and avoid the amount of error.