AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications

Abstract

This article provides a comprehensive overview of the fairness issues in artificial intelligence (AI) systems, delving into its background, definition, and development process. The article explores the fairness problem in AI through practical applications and current advances and focuses on bias analysis and fairness training as key research directions. The paper explains in detail the concept, implementation, characteristics, and use cases of each method. The paper explores strategies to reduce bias and improve fairness in AI systems, reviews challenges and solutions to real-world AI fairness applications, and proposes future research directions. In addition, this study provides an in-depth comparative analysis of the various approaches, utilizing cutting-edge research information to elucidate their different characteristics, strengths, and weaknesses. The results of the comparison provide guidance for future research. The paper concludes with an overview of existing challenges in practical applications and suggests priorities and solutions for future research. The conclusions provide insights for promoting fairness in AI systems. The information reviewed in this paper is drawn from reputable sources, including leading academic journals, prominent conference proceedings, and well-established online repositories dedicated to AI fairness. However, it is important to recognize that research nuances, sample sizes, and contextual factors may create limitations that affect the generalizability of the findings.

1. Introduction

1.1. Background

With recent advances in artificial intelligence (AI), decision making has gradually shifted from rule-based systems to machine learning-based developments (e.g., [1,2,3]), learning patterns from data and performing pattern recognition, inference, or prediction. Although such a new methodological trend is derived from the bias brought by human rules, this bias and unfairness are gradually permeating artificial intelligence in another form, as humans are still involved in collecting the datasets used to train machine learning in the new system [4,5].

Artificial intelligence fairness (AI Fairness) is an issue proposed in response to this status quo, which is intended to prevent different harms (or benefits) to different subgroups, thereby providing a system that both quantifies prejudice and mitigates discrimination against subgroups [6,7]. Questions about AI Fairness are practical and affect the lives around us in many ways. Some decision support systems for credit applications tend to favor certain sociodemographic groups, resulting in people living in certain areas, and people of certain ethnic backgrounds or genders having a certain selection preference for loan approval, which is difficult to make completely objective and fair [8,9,10,11]. Meanwhile, disability information is highly sensitive and cannot be shared, thus exacerbating this unfairness due to the opaqueness of the information [12]. Companies may miss out on many potential talents due to an AI-based recruiting engine that is biased against region, gender, and ethnicity, and even cause the company’s team composition to gradually become homogenized in biased elements, thereby losing the advantages of diversity [13,14].

It can be seen that the study of this issue has broad social, political, and economic significance. Once the AI misunderstands the intended task, the problem of value misalignment often ensues, and many social issues and responsibilities will arise.

1.2. Directions

Based on the works in recent years, the conceptual development of AI Fairness has focused on the following directions:

• Fairness and bias [15,16,17]: Introduction of widely used fairness metrics, such as disparate impact, equal opportunity, and statistical parity. Evaluation of their applicability and limitations in different contexts, contributing to a nuanced understanding of group fairness assessment.
• Algorithmic bias mitigation [18,19,20]: Exploration of techniques, like pre-processing, in-processing, and post-processing, to mitigate algorithmic biases. Critical analysis of their effectiveness in different scenarios, offering insights into the trade-offs between bias reduction and model performance.
• Fair representation learning [21,22,23]: Introduction of techniques for learning fair representations, including adversarial debiasing and adversarial learning frameworks. Investigation into their potential for producing fair and informative representations, fostering a deeper comprehension of their role in mitigating biases, understanding the true sources of disparities, aiding in the design of more targeted interventions.

Based on the above conceptual directions, we condense and analyze the methodology and technical analysis involved, and focus on the major key elements in this paper, including definition and problem formulation, bias analysis, fair training, and corresponding applications and practices.

This article undertakes a comprehensive exploration of the critical subject of fairness issues within artificial intelligence (AI) systems. The overarching scope of this survey is to provide an in-depth analysis of the multifaceted landscape of AI fairness, covering its foundational aspects, developmental trajectory, practical applications, and emerging research directions. By delving into these dimensions, the survey aims to shed light on the complex challenges linked to fairness in AI, while offering insights into potential remedies and avenues for future exploration.

1.3. Scope

This article mainly collects the research with the details of the corresponding research plan and methodological route, providing a comprehensive survey of the advancements in the domain of AI Fairness. The scope of this survey is expansive and encompasses diverse facets of AI fairness. It begins by elucidating the foundational background and definition of fairness within the realm of AI systems. Subsequently, the survey ventures into the dynamic landscape of fairness concerns, exploring practical applications and recent advancements. Of particular significance are the domains of bias analysis and fairness training, which are delved into as crucial research directions aimed at ameliorating biases and fostering equitable AI outcomes. The survey encompasses meticulous explanations of the concepts, implementations, characteristics, and practical use cases of each method, thereby providing a comprehensive understanding of their nuances.

Encompassing a retrospective spectrum, the covered literature spans from the most recent contributions (e.g., [24,25,26]) to the initial inception of pertinent theories, extending as far back as 1993 (e.g., [27,28,29]). A meticulous curation process led to the inclusion of 310 papers from an extensive pool of 1083 pieces of materials. The selection criteria entailed a thorough assessment of factors, such as the intrinsic significance, perceptible impact, novelty, ingenuity, and citation prevalence of the respective works. These works were methodically categorized and subjected to thorough examination within the manuscript. The ensuing textual discourse encompasses not only analysis but also the deliberation and the derivation of insightful perspectives.

1.4. Contributions

This article offers an extensive and comparative analysis of diverse approaches, leveraging contemporary research to expound upon their distinct attributes, strengths, and limitations. This comparative exploration not only guides researchers but also informs practitioners, providing them with a nuanced understanding of available methods and aiding their decision making.

Additionally, the survey enriches the discourse on AI fairness by contextualizing its practical implications. By exploring strategies to mitigate bias and enhance fairness in AI systems, it bridges the gap between theoretical foundations and real-world challenges. The survey further discusses the critical subject of challenges and solutions in real-world AI fairness applications, offering insights into the current limitations and potential remedies.

Furthermore, this survey contributes by acknowledging the dynamic nature of the AI fairness landscape and the evolving nature of research and advancements. It underscores the evolving nature of the field and the limitations associated with the evidence derived from reputable sources. While these limitations stem from factors such as research nuances, sample sizes, and contextual intricacies, the survey remains committed to fostering the continuous exploration and understanding of AI fairness.

In essence, this survey encompasses a wide-ranging scope, delving into the genesis, evolution, applications, and challenges of AI fairness. Its multifaceted contributions aim to advance the understanding of fairness in AI, providing valuable insights for both academia and industry in their pursuit of equitable and unbiased AI systems.

1.5. Organization of This Article

The organization of the article is organized as follows. Section 2 introduces the background and definition of AI fairness, and Section 3 formulates the definitions and problems of fairness in AI systems. The main directions of the research of addressing AI fairness, bias analysis, and fair training are reviewed in Section 4 and Section 5 with details of corresponding methodologies. Section 6 discusses the measures of migrating the bias and improving fairness in the AI system. Section 7 reviews the related issues and solutions in the practical applications of AI fairness, and corresponding future works are discussed and given. Section 8 concludes the paper.

2. Preliminary

2.1. Status Quo

Although the study of fairness in machine learning is a relatively new topic, it has attracted extensive attention. IBM launched AI Fairness 360 [30,31,32], which can help detect and mitigate unwanted bias in machine learning models and datasets. It provides around 70 fairness metrics to test for bias and 11 algorithms to reduce bias in datasets and models, thereby reducing software bias and improving its fairness (e.g., [33]).

Facebook has also developed the Fairness Flow tool to detect bias in AI, which works by detecting forms of statistical bias in some of Facebook’s commonly used models and data labels, enabling the analysis of how certain types of AI models perform across different groups [34,35,36]. It defines “bias” as the systematic application of different standards to different groups of people. Given a dataset containing predictions, labels, group membership (for example, gender or age), and other information, Fairness Flow can divide the data used by the model into subsets and estimate its performance.

In 2019, Google also embedded the Fairness Indicators component in a series of AI tools it developed, resulting in tools built on top of TensorFlow model analysis that can regularly calculate and visualize fairness indicators for binary and multi-class classification [37,38,39].

Although the above work provides tools and theoretical analysis, due to the short research time of this problem and still in the preliminary exploratory stage, there is no mature standard for how to quantify the risk of AI fairness and little insights in how to make decisions in the case of consensus and controversy with the commonly accepted solutions to the risks.

2.2. Review Methodology

2.2.1. Materials

Aiming at the topic along with the issues above, we collected a variety of current academic and technical materials on AI equity and conducted a synthesis study. The information synthesized in this study comes from a variety of reliable sources. These sources include recent publications in prestigious academic journals, distinguished conference proceedings, and well-established online repositories dedicated to the fairness of AI.

Our comprehensive search strategy includes systematic searches of respected academic databases, including IEEE Xplore, ScienceDirect, ACM Digital Library, Springer, Wiley Library, etc. In addition, we carefully reviewed relevant conference proceedings and authoritative organization websites to ensure research inclusiveness. In a robust and diverse collection, we conducted a meticulous review process that encompassed a wide range of sources. We extensively searched through various databases, culminating in the compilation of 1083 materials, comprising Proceedings, Miscellaneous, Articles, Tech Reports, Books, Ph.D. theses, and Collections. To filter the literature and complete the review, the review process was multifaceted and involved several key criteria.

2.2.2. Criteria

The inclusion and exclusion criteria we use in selecting sources of information are carefully thought out to ensure the relevance and quality of the studies included in our analysis. Our criteria included peer-reviewed academic articles, conference papers, and authoritative reports that explicitly address the fairness of AI in data management and analytics. During the synthesis process, we thoughtfully grouped studies based on thematic affinity, methodology, and the nature of the fairness challenges to be addressed. This grouping strategy facilitates a coherent and well-organized synthesis of the different perspectives in the literature. Our criteria for selecting materials are carefully considered and include the following factors, including the following:

• Duplication: We strive to offer diverse and original content to our audience. To avoid redundancy, we review submissions to ensure that the material we publish does not duplicate the existing content in our collection.
• Ineligible content: Our selection process also involves evaluating whether the submitted content meets our eligibility criteria, including adhering to our guidelines and standards.
• Publishing time: We value timeliness and relevance. We prioritize materials that are current and align with the most recent developments and trends in the respective field.
• Quality of publication: Ensuring the quality of the content we publish is of utmost importance. We assess the accuracy, credibility, and overall value of the material to ensure it meets our quality standards.
• Accessibility: Our goal is to make information accessible to a wide range of readers. We select materials that are well structured, clear, and easily understandable, catering to readers with varying levels of expertise.
• Similarity of content: While covering a broad spectrum of topics, we also strive for variety and distinctiveness in our content selection. We aim to present diverse perspectives and insights to enrich the reader experience.

We adopted PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) in our review, which meticulously outlines the systematic progression of the study identification, screening, eligibility, and inclusion phases, thereby increasing the reproducibility and rigor of the review process. The PRISMA procedure is shown in Figure 1 as a flowchart. After the procedure, reference lists of selected articles in the field are reviewed intensively while identifying potential studies.

2.3. Limitations

It is important to recognize that while these sources have contributed significantly to our understanding, there are limitations to the evidence they provide. These limitations stem primarily from nuances in research methodology, sample size, and context, which may affect the generalizability of the conclusions drawn from individual studies. The landscape of AI fairness is dynamic, with research and advancements continually shaping our understanding of its complexities. While our current coverage might have limitations due to the rapid pace of change and ongoing research, please know that we are committed to further studying and exploring this crucial subject. Despite these inherent limitations, our review endeavors to provide a comprehensive and balanced overview of the current state of research related to the fairness of AI. We are dedicated to providing accurate and comprehensive information to readers with the notice of the need to stay engaged with emerging topics like AI fairness.

3. Definition and Problems

3.1. Definition

As AI technologies continue to permeate all aspects of society, ensuring fairness in their decision-making processes is crucial to avoid perpetuating bias and inequality. However, defining what constitutes fairness in AI is a complex and multifaceted task. So far, there are mainly seven types of definitions, including individual fairness [40,41], group fairness [42], equality of opportunity [11], disparate treatment [43], fairness through unawareness [44,45], disparate impact [46], and subgroup fairness [47].

Fairness in AI can be approached through different conceptual lenses. Individual fairness emphasizes equitable treatment for similar individuals, while group fairness aims to avoid disparate treatment based on demographic attributes. Equality of opportunity focuses on consistent predictive accuracy and error rates across various groups, regardless of outcomes.

An alternative approach is fairness through unawareness, achieved by ignoring sensitive attributes in decision making. Despite its intentions, this method might indirectly perpetuate bias present in data. Disparate impact examines whether AI systems disproportionately harm certain groups, irrespective of intent. It aims to uncover biases in outcomes, intentional or not.

To address complex interactions, subgroup fairness evaluates fairness at the intersection of multiple protected attributes, ensuring equitable experiences for diverse subgroups. These conceptions contribute to a comprehensive understanding of fairness in AI and underscore the multifaceted nature of achieving equitable outcomes.

Table 1. Fairness definition.

Data Bias	Definition	Main Cause	References
Individual Fairness	Similarity at the individual level	Treat similar individuals similarly	[40,41,48,49]
Group Fairness	Equitable outcomes for demographic groups	Avoid disparities among groups	[42,50,51]
Fairness through Unawareness	Ignoring sensitive attributes	Treat individuals as if attributes are unknown	[44,45,52]
Equality of Opportunity	Equal chances for similar qualifications	Ensure equal chances for outcomes	[11,53,54]
Disparate Impact	Disproportionate negative effects	Evaluate disparities in outcomes	[6,46,55]
Disparate Treatment	Explicit unequal treatment	Detect explicit biases in treatment	[43,56,57]
Subgroup Fairness	Fairness at the intersection of multiple attributes	Consider fairness for multiple groups	[47,58]

summarizes these approaches to fairness in AI. Individual fairness prioritizes personalized treatment, while group fairness targets demographic equity. Fairness through unawareness and equality of opportunity tackle fairness differently—ignoring attributes vs. ensuring equal chances based on qualifications. Disparate impact assesses negative effects, disparate treatment detects unequal treatment. Subgroup fairness navigates complex attribute interactions for equitable outcomes. These concepts collectively enrich the understanding of fairness in AI systems.

3.2. Problems

The risks with AI systems mainly come from data accountability [24,59] and algorithm accountability [60,61,62]. The connotation of data accountability mainly includes data ownership, storage, use, and sharing, while algorithm accountability emphasizes determining who is responsible for the output of AI algorithms. The interplay of these two risks also raises the question of mission inclusivity [63,64,65], which mainly focuses on whether the AI system is effective for diverse user populations. Bias effects in machine learning are shown in Figure 2.

Biases in AI models can arise from data collection, labeling, and partitioning, affecting data integrity. Human factors during training, including optimization objectives and parameter configurations also contribute to bias. Inclusivity applications involve tuning and updates guided by user feedback, which holds substantial influence. Techniques like K-fold mitigate dataset bias, but original bias persists, highlighting the importance of robust data accountability. Subjective optimization objective design exacerbates bias effects. Addressing these issues requires comprehensive strategies for data and model development.

Feedback bias in AI model tuning can arise if participants providing feedback are disproportionately represented in specific communities or feature sets. This concentration can lead to model adjustments aligning more closely with the preferences of that group. Grouping can be based on experimental settings or attributes, aiding the analysis of cognitive and labeling differences’ impact on bias patterns. Statistical features synthesized from group evaluations can influence machine learning model outcomes.

The inclusivity of intelligent computing services relies on data and algorithm accountability to mitigate bias and ensure fairness in machine learning processes. While industry practices often exclude sensitive attributes to address fairness concerns, this approach overlooks the potential influence of non-sensitive attributes in reflecting bias. Additionally, evaluating fairness using static test sets poses challenges, including potential incompleteness and inherent bias carried over from existing systems.

Moreover, the feedback loop between machine learning system outputs and inputs can perpetuate and reinforce biases, necessitating the analysis of algorithms in dynamic systems. Label noise further complicates the picture, as large datasets essential for deep network training can inadvertently incorporate incorrect labels, undermining model accuracy and performance.

Addressing these challenges, ongoing research focuses on detecting and mitigating bias while designing fair machine learning mechanisms and intelligent systems. Bias analysis and fair training emerge as critical areas, aiming to enhance the technical understanding and current status of each direction.

4. Bias Analysis

4.1. Data Bias

Data bias is a critical concern in artificial intelligence (AI) systems, as biased data can lead to unfair and discriminatory outcomes. It arises when the training data used to develop AI models are skewed, leading to biased predictions or decisions [66,67]. Biased data can perpetuate historical prejudices and result in discriminatory outcomes. There are five main types bias, including selection bias [68,69,70,71], sampling bias [25,72,73], labeling bias [26,74,75,76,77], temporal bias [78,79,80,81], aggregation bias [82,83,84,85,86], historical bias [52,87,88,89], measurement bias [4,90,91,92], confirmation bias, proxy bias, cultural bias, under-representation bias [93,94,95], and homophily bias [96,97,98].

Table 2. Comparison on data biases.

Data Bias	Definition	Main Cause	Impact on AI	References
Selection Bias	Certain groups are over/under-represented	Biased data collection process	AI models may not be representative, leading to biased decisions	[68,69,70,71]
Sampling Bias	Data are not a random sample	Incomplete or biased sampling	Poor generalization to new data, biased predictions	[25,72,73]
Labeling Bias	Errors in data labeling	Annotators’ biases or societal stereotypes	AI models learn and perpetuate biased labels	[26,74,75,76]
Temporal Bias	Historical societal biases	Outdated data reflecting past biases	AI models may reinforce outdated biases	[78,79,80,81]
Aggregation Bias	Data combined from multiple sources	Differing biases in individual sources	AI models may produce skewed outcomes due to biased data	[82,83,84,85]
Historical Bias	Training data reflect past societal biases	Biases inherited from historical societal discrimination	Model may perpetuate historical biases and reinforce inequalities	[52,87,88,89]
Measurement Bias	Errors or inaccuracies in data collection	Data collection process introduces measurement errors	Model learns from flawed data, leading to inaccurate predictions	[4,90,91,92]
Confirmation Bias	Focus on specific patterns or attributes	Data collection or algorithmic bias towards specific features	Model may overlook relevant information and reinforce existing biases	[27,99,100,101,102]
Proxy Bias	Indirect reliance on sensitive attributes	Use of correlated proxy variables instead of sensitive attributes	Model indirectly relies on sensitive information, leading to biased outcomes	[42,103,104,105]
Cultural Bias	Data reflect cultural norms and values	Cultural influences in data collection or annotation	Model predictions may be biased for individuals from different cultural backgrounds	[72,106,107]
Under-representation Bias	Certain groups are significantly underrepresented	Low representation of certain groups in the training data	Model performance is poorer for underrepresented groups	[93,94,95]
Homophily Bias	Predictions based on similarity between instances	Tendency of models to make predictions based on similarity	Model may reinforce existing patterns and exacerbate biases	[96,97,98]

shows the comparison of the different types of data biases.

Bias in AI models can stem from various data-related sources. Selection bias arises from skewed data representation due to biased collection or incomplete sampling. Aggregation bias results when data from different sources with varying biases are combined without proper consideration. Sampling bias emerges when training data fail to represent the target population adequately. Labeling bias occurs due to errors in annotation, introducing bias into training. Measurement bias originates from inaccuracies during data collection, impacting the model’s ability to learn accurately. Temporal and historical biases arise from reflecting outdated societal biases. Unconscious biases, such as cultural bias, lead to biased decisions for diverse groups. Proxy bias uses correlated proxy variables, indirectly introducing bias. Homophily bias reinforces existing patterns in prediction, potentially intensifying bias. Understanding and mitigating these biases are crucial for equitable AI systems.

4.2. Algorithmic Bias

Algorithmic bias refers to biases inherent in the design and structure of AI models [108,109]. These biases may be unintentionally introduced during the development process, leading to unequal treatment of different groups. The main algorithmic biases include prejudice bias, sampling bias, feedback loop bias, lack of diversity bias, and automation bias.

Prejudice bias arises from biased training data, perpetuating societal stereotypes. Sampling bias stems from data misrepresentation, causing poor generalization. Feedback loop bias is a self-reinforcing cycle, where biased AI predictions lead to biased feedback. Lack of diversity bias emerges from inadequate dataset representation, affecting underrepresented groups. Automation bias involves over-reliance on AI decisions without scrutiny, potentially amplifying underlying biases.

Table 3. Algorithmic bias comparison.

Algorithmic Bias	Definition	Main Cause	Impact on AI	References
Prejudice Bias	AI models trained on biased data	Biased training data and societal prejudices	Reinforces biases, leads to discriminatory outcomes	[76,110,111,112]
Sampling Bias	Data do not represent the target population	Incomplete or skewed sampling methods	Poor generalization, biased predictions	[85,113,114,115]
Feedback Loop Bias	Self-reinforcing bias cycle in AI predictions	Biased predictions influencing biased feedback	Amplifies biases, perpetuates discrimination	[116,117,118,119,120]
Lack of Diversity Bias	Training on limited or homogeneous datasets	Insufficient representation of diverse groups	Performs poorly for underrepresented groups	[40,121,122,123,124,125]
Automation Bias	Human over-reliance on AI decisions	Blind trust in AI without critical evaluation	Perpetuates biases without human intervention	[126,127,128,129,130,131]

summarizes the comparison of different types of algorithmic bias in AI systems, highlighting their definitions and main implications. Prejudice bias originates from biased data collection, reinforcing discrimination. Sampling bias results from non-representative data, causing biased predictions. Feedback loop bias is a self-reinforcing cycle driven by biased predictions and feedback. Lack of diversity bias emerges from homogeneous training datasets, affecting underrepresented groups. Automation bias is the uncritical reliance on AI decisions, amplifying underlying biases.

4.3. User Interaction Bias

User interaction bias occurs when AI systems adapt their behavior based on user feedback, potentially reinforcing and amplifying existing biases [67,132]. It manifests in various forms, each contributing to biased decision making and unequal outcomes in AI systems.

Table 4. User interaction bias comparison.

Bias Type	Definition	Main Cause	Impact on AI	Reference
User Feedback Bias	User Feedback Bias occurs when biased user feedback or responses influence the behavior of AI systems.	Biased user feedback or responses can be influenced by users’ subjective preferences, opinions, or prejudices. The AI system learns from this feedback and incorporates it into its decision-making process.	AI models may generate biased predictions and decisions based on the biased feedback, potentially leading to unequal treatment of certain user groups. User satisfaction and trust in the AI system can be affected by biased outputs.	[116,117,118]
Biases from Underrepresented or Biased User Data	This bias arises when the data collected from users lack diversity or contain inherent biases, which can lead to biased model predictions and decisions that disproportionately affect certain user groups.	Lack of diversity or inherent biases in user data can result from biased data collection practices, data preprocessing, or historical biases reflected in the data.	AI systems trained on biased user data may produce unfair outcomes, disproportionately impacting specific user groups. Biases in data can lead to the perpetuation and amplification of existing inequalities.	[133,134,135]
Automation Bias in Human–AI Interaction	Automation bias refers to biased decision making by users when utilizing AI systems, potentially influencing the AI system’s outcomes and recommendations.	Automation bias can occur when users over-rely on AI recommendations without critically evaluating or verifying the results. Human trust in AI systems and the perceived authority of the AI can contribute to automation bias.	Automation bias can lead to the uncritical acceptance of AI-generated outputs, even when they are biased or inaccurate. It may result in erroneous or unfair decisions based on AI recommendations. Awareness of automation bias is crucial to avoid blindly accepting AI decisions without human oversight.	[126,128,129]

summarizes the typical user interaction biases, including user feedback bias, underrepresented or biased user data bias, and automation bias.

User feedback bias and bias from underrepresented or biased user data contribute to user interaction biases, influencing AI system behavior and predictions. The interaction between humans and AI, rooted in automation bias, further affects these biases. Notably, user interaction bias and algorithmic bias overlap, as biases from human–computer interaction data impact industrial intelligence models, highlighting their interconvertibility.

5. Fair Training

5.1. Fair Training Methods

In response to the above bias analysis, we hope to be able to develop an AI system without bias by conducting fair training so that we can avoid perpetuating inequalities due to discriminatory appearances caused by biases. Fairness training aims to reduce these biases and promote fair decision making. There are several fair training methods that are currently in common use, including pre-processing fairness [136], in-processing fairness [137], post-processing fairness [46], regularization-based fairness [43], counterfactual fairness [41,45,138].

Pre-processing, in-processing, and post-processing fairness techniques address bias in AI systems from different angles. Pre-processing involves modifying training data to balance group representation. In-processing modifies learning algorithms to integrate fairness. Post-processing adjusts model predictions to align with fairness goals. Additionally, regularization-based methods introduce fairness constraints in optimization, aiming to minimize disparities, while counterfactual fairness measures fairness by assessing outcome consistency for similar individuals across sensitive attributes.

Fair training techniques, as depicted in

Table 5. Fair training method comparison.

Fair Training Method	Definition	Implementation	Key Features	References
Pre-processing Fairness	Modifying training data before feeding into the model	Re-sampling, re-weighting, data augmentation	Addresses bias at the data level	[136,139,140]
In-processing Fairness	Modifying learning algorithms or objective functions	Adversarial training, adversarial debiasing	Simultaneously optimizes for accuracy and fairness	[137,141,142]
Post-processing Fairness	Adjusting the model’s predictions after training	Re-ranking, calibration	Does not require access to the model’s internals	[46,143,144,145]
Regularization-based Fairness	Adding fairness constraints to the optimization process	Penalty terms in the loss function	Can be combined with various learning algorithms	[43,146,147]
Counterfactual Fairness	Measuring fairness based on changes in sensitive attributes	Counterfactual reasoning	Focuses on individual-level fairness	[45,148,149]

, strive to mitigate biases in AI systems by integrating fairness considerations into the training process. Through the incorporation of sensitive attributes and fairness constraints, these methods aim to diminish the impact of such attributes on model predictions, guarding against biased outcomes that could disproportionately affect marginalized groups. The challenge lies in striking a balance between fairness and accuracy, avoiding the compromise of model performance while enhancing fairness.

5.2. Pre-Processing Fairness

Preprocessing fairness applications involve modifying training data before feeding them into an AI model to reduce bias and promote fairness. These techniques focus on addressing biases in the data themselves, which can lead to fairer model results. Common methods include resampling, reweighting, data augmentation, fairness-aware clustering, and synthetic oversampling techniques.

Resampling techniques, such as oversampling and undersampling, adjust data distribution to alleviate bias by equalizing group representation. Reweighting assigns higher weights to underrepresented instances during model training, reducing bias against marginalized groups. Data augmentation generates synthetic data to bolster underrepresented groups, enhancing fairness. Fairness-aware clustering ensures equitable grouping, while the Synthetic Minority Oversampling Technique (SMOTE) generates synthetic samples to balance class distribution, promoting fairness in classification tasks. These methods collectively counteract bias and enhance fairness in AI models.

In

Table 6. Pre-processing fairness comparison.

Pre-Processing Fairness Method	Features	Pros	Cons	References
Re-sampling Techniques	Balance representation of different groups	Simple and easy to implement	May lead to loss of information and increased computation	[150,151,152,153]
Re-weighting Techniques	Assign higher weights to underrepresented groups	Does not alter the original dataset	Requires careful selection of appropriate weights	[154,155,156,157,158,159]
Data Augmentation	Generate synthetic data to increase representation	Increases the diversity of the training dataset	Synthetic data may not fully represent real-world samples	[160,161,162,163]
Fairness-aware Clustering	Cluster data points while maintaining fairness	Incorporates fairness constraints during clustering	May not guarantee perfect fairness in all clusters	[164,165,166,167]
Synthetic Minority Over-sampling Technique (SMOTE)	Generate synthetic samples for the minority class	Addresses class imbalance	May result in overfitting or noisy samples	[168,169,170,171]

, it can be seen that re-sampling techniques handle class imbalance, re-weighting adjusts data importance, data augmentation enhances diversity, attribute swapping equalizes sensitive attributes, fairness-aware clustering ensures equitable grouping, and SMOTE addresses class imbalance. By selecting and applying the appropriate pre-processing fairness method based on the specific dataset and fairness goals, AI practitioners can develop models that prioritize fairness and equitable outcomes. Continued research and experimentation with these techniques will advance the pursuit of unbiased AI applications across various domains.

5.3. In Processing Fairness

In-processing fairness refers to modifying the learning algorithm or objective function during model training to explicitly incorporate fairness constraints. These techniques aim to simultaneously optimize accuracy while reducing bias and promoting fairness. Their main approaches include adversarial training [172,173,174,175,176], adversarial debiasing [137,177,178,179], equalized odds post-processing [11,144,177,180], fair causal learning [45,181,182,183,184], and meta-fairness [163,185,186].

Table 7. In-processing fairness comparison.

In-Processing Fairness Method	Features	Pros	Cons	References
Adversarial Training	Adversarial component to minimize bias impact	Enhances model’s fairness while maintaining accuracy	Sensitive to adversarial attacks, requires additional computational resources	[172,173,174,175,176]
Adversarial Debiasing	Adversarial network to remove sensitive attributes	Simultaneously reduces bias and improves model’s fairness	Adversarial training challenges, potential loss of predictive performance	[137,177,178,179]
Equalized Odds Post-processing	Adjust model predictions to ensure equalized odds	Guarantees fairness in binary classification tasks	May lead to suboptimal trade-offs between fairness and model performance	[11,144,177,180]
Causal Learning for Fairness	Focus on causal relationships to adjust for bias	Addresses confounding factors to achieve fairer predictions	Requires causal assumptions, may be limited by data availability	[45,181,182,183,184]
Meta Fairness	Learns fairness-aware optimization algorithm	Adapts fairness-accuracy trade-off to changing requirements	Complexity in learning the optimization algorithm, potential increased complexity	[163,185,186]

summarizes the comparison of the methods above. Adversarial training and adversarial debiasing introduce an adversarial component to the learning process, aiming to minimize the influence of sensitive attributes on model predictions. These methods have been applied across tasks like natural language processing, computer vision, and recommendation systems to enhance fairness and reduce bias. Causal learning methods focus on understanding causal relationships within data and addressing confounding factors that lead to biased predictions. This approach has been implemented in domains such as healthcare and criminal justice to ensure fairer and more interpretable outcomes. Meta Fairness involves learning a fairness-aware optimization algorithm that dynamically adjusts the balance between fairness and accuracy during model training. It is particularly valuable when fairness requirements vary across user groups or over time.

Adapting to different biases and trade-offs between fairness and performance, these methods provide valuable tools for equitable AI. Choosing the appropriate method hinges on factors such as the application context and bias type.

5.4. Post-Processing Fairness

Post-processing fairness methods focus on adjusting or post-processing the output of trained AI models to ensure fairness and reduce bias after the model has made its predictions. These techniques are applied after the model has made a decision to align the results with the fairness goal and mitigate any potential bias present in the predictions. Some common post-processing fairness methods include equalized odds post-processing [11,144,177,180], calibration post-processing [187,188,189,190], rejected options classification (ROC) post-processing [144,191,192,193], priority sampling post-processing [194,195,196], threshold optimization post-processing [197,198,199,200], and regularization post-processing [201,202,203,204].

Table 8. Post-processing fairness comparison.

Post-Processing Fairness Method	Features	Pros	Cons	References
Equalized Odds Post-processing	Adjust model predictions to ensure equalized odds	Ensures equalized false positive and true positive rates across groups	May lead to suboptimal trade-offs between fairness and model performance	[11,144,177,180]
Calibration Post-processing	Calibrates model’s predicted probabilities	Improves fairness by aligning confidence scores with true likelihood	Calibration may not entirely remove bias from the model	[187,188,189,190]
Reject Option Classification (ROC) Post-processing	Introduces a “reject” option in classification decisions	Allows the model to abstain from predictions in high fairness concern cases	May lead to lower accuracy due to abstaining from predictions	[144,191,192,193]
Preferential Sampling Post-processing	Modifies the training data distribution by resampling instances	Improves fairness by adjusting the representation of different groups	May not address the root causes of bias in the model	[194,195,196]
Threshold Optimization Post-processing	Adjusts decision thresholds for fairness and accuracy trade-off	Allows fine-tuning of fairness and performance balance	May not fully eliminate all biases in the model	[197,198,199,200]
Regularization Post-processing	Applies fairness constraints during model training	Encourages fairness during the optimization process	Fairness constraints might impact model performance	[201,202,203,204]

summarizes the features, pros and cons with the comparison of these methods.

Equalized odds post-processing is employed post-model training to align predictions, ensuring equal false alarm and omission rates across different groups. Calibration refines predicted probabilities to accurately reflect event likelihood. Reject option classification introduces a “reject” option to avoid biased predictions in sensitive situations. Preferential sampling post-processing reshapes training data distribution for enhanced fairness. Threshold optimization post-processing adjusts decision thresholds for a balanced fairness–accuracy trade-off. Regularization post-processing employs regularization techniques to encourage fairness during model optimization. These techniques offer ways to enhance fairness in AI predictions and are particularly useful in contexts like credit scoring and hiring decisions.

The effectiveness and suitability of post-processing fairness methods vary with the AI application and fairness goals. While valuable in certain contexts, these techniques might not entirely resolve root biases. A holistic AI fairness approach should combine pre-processing, in-processing, and post-processing methods, alongside continuous monitoring and evaluation, to ensure equitable outcomes.

5.5. Regularization Based Fairness

Regularization-based fairness methods are emerging as a promising approach to mitigate biases in machine learning models. Regularization techniques aim to enforce fairness constraints during the model training process, ensuring that the model’s predictions are less influenced by sensitive attributes and promote equitable outcomes.

Table 9. Regularization-based fairness comparison.

Regularization-Based Fairness Method	Features	Pros	Cons	References
Adversarial Regularization	Introduces adversarial component	Encourages disentanglement of sensitive attributes	Computationally expensive	[205,206,207,208]
Demographic Parity Regularization	Enforces similar distributions across groups	Addresses group fairness	May lead to accuracy trade-offs	[201,204,209,210,211]
Equalized Odds Regularization	Ensures similar false/true positive rates	Emphasizes fairness in both rates	May lead to accuracy trade-offs	[201,212,213]
Covariate Shift Regularization	Reduces impact of biased/underrepresented subgroups	Addresses bias due to distributional differences	Sensitive to noise in the data	[214,215]
Mixture Density Network Regularization	Models uncertainty in predictions	Provides probabilistic approach to fairness regularization	Requires larger amount of data to estimate probability distributions	[216,217,218]

summarizes and compares different regularization methodologies for AI fairness, including adversarial regularization [205,206,207,208], demographic parity regularization [201,204209,210,211], equalized odds regularization [201,212,213], covariate leveling regularization [214,215], and mixture density network regularization [216,217,218].

Regularization-based fairness methods introduce additional components to the model training process to mitigate bias in AI predictions. Adversarial regularization aims to minimize model dependence on sensitive attributes by introducing an adversarial component. Demographic parity regularization enforces similar prediction distributions across sensitive attribute groups. Equalized odds regularization maintains consistent false alarm and true alarm rates among these groups. Covariate leveling regularization adapts predictions to diverse data distributions. Mixture density network regularization models prediction uncertainty through probability density functions. Each approach offers distinct benefits and trade-offs in addressing bias.

5.6. Counterfactual Fairness

Counterfactual fairness is an approach that seeks to address bias in AI models by considering counterfactual scenarios, where sensitive attributes are altered while keeping other features fixed. The idea is to evaluate fairness by examining how the model’s predictions would change if an individual belonged to a different demographic group, allowing for a more nuanced understanding of biases.

Table 10. Counterfactual fairness methods comparison.

Counterfactual Fairness Method	Features	Pros	Cons	References
Individual Fairness	Focuses on treating similar individuals similarly based on their features	Considers fairness at the individual level, promoting personalized fairness	Defining similarity metrics and enforcing individual fairness can be challenging	[40,196,219]
Equal Opportunity Fairness	Minimizes disparate impact on true positive rates across sensitive attribute groups	Targets fairness in favor of historically disadvantaged groups	May neglect other fairness concerns, such as false positive rates or overall accuracy	[220,221,222]
Equalized Odds Fairness	Aims for similar false positive and true positive rates across sensitive attribute groups	Addresses fairness in both false positives and false negatives	May lead to accuracy trade-offs between groups	[229,230,231]
Reweighted Counterfactual Fairness	Assigns different weights to instances based on similarity to counterfactual scenarios	Provides better fairness control by adjusting instance weights	Determining appropriate weights and balancing fairness and accuracy can be challenging	[223,224,225]
Oblivious Training	Trains the model to be ignorant of certain sensitive attributes during learning	Offers a simple and effective way to mitigate the impact of sensitive attributes	May result in lower model performance when sensitive attributes are relevant to the task	[226,227,228]

summarizes and compares different regularization methodologies for AI fairness, including individual fairness [40,168,196,219], equal opportunity fairness [220,221,222], reweighted counterfactual fairness [223,224,225], and oblivious training [226,227,228].

Individual fairness focuses on treating similar individuals equally despite their protected attributes. It promotes personalized fairness at the individual level, emphasizing fine-grained treatment. Equal opportunity fairness ensures similar true positive rates across different groups to prevent disparate impact in binary classification. Reweighted counterfactual fairness adjusts data weights during training to mitigate bias and can be combined with fairness-aware algorithms. Oblivious training trains models on both original and counterfactual data to promote fairness without explicit labels.

These methods address different fairness concerns, considering both individual and group fairness aspects, each with their computational and implementation considerations. Each fairness method has strengths and limitations, potentially impacting areas like model performance and interpretability. The method chosen should align with specific fairness criteria and application contexts, as certain methods may be better suited for particular domains than others.

In medical data collection, informed consent methods are employed to clarify data usage and potential risks. Privacy techniques like anonymization protect individuals’ identities. Data minimization reduces privacy risks by collecting only necessary information, though this may limit insights. Transparency communicates data collection processes, building trust while potentially raising privacy concerns. Data security measures include encryption and access controls to prevent unauthorized access. Accuracy and accountability methods involve auditing for reliable data and research outcomes. These approaches enhance data quality and accountability but may require resource allocation. Balancing these strategies is essential for ethical and effective data collection in scientific research.

6. Discussion

6.1. Fair Data Collection

To guarantee the fairness in data collection, we summarize the different methods with the comparison in

Table 11. Fair data collection fairness methods comparison.

Method Category	Features	Pros	Cons	References
Informed Consent	Obtain explicit consent from participants	Respects individual autonomy	May lead to selection bias	[232,233,234]
Informed Consent	Clear explanation of data collection purpose	Builds trust with participants	Consent may not always be fully informed	[235,236]
Informed Consent	Informed of potential risks		Difficulties with complex research studies	[237,238,239]
Privacy and Anonymity	Data anonymization, aggregation, de-identification	Protects participant privacy	Reduced utility of anonymized data	[240,241]
Privacy and Anonymity	Prevents re-identification of individuals	Minimizes risk of data breaches	Challenges in preserving data utility	[242,243,244]
Data Minimization	Collect only necessary data	Reduces data collection and storage costs	Limited data for certain analyses	[28,245,246]
Data Minimization	Avoid gathering excessive/inappropriate data	Mitigates privacy risks	Potential loss of insights	[247,248]
Transparency	Clear communication of data collection process	Builds trust with data subjects	May lead to privacy concerns	[249,250,251]
Transparency	Information on methods and data use	Increases data sharing and collaboration	Difficulties in balancing transparency	[249,250,251]
Data Security	Encryption, access controls, security audits	Protects data from unauthorized access	Implementation costs	[252,253,254]
Data Security	Safeguards data from breaches	Prevents data manipulation and tampering	Potential usability impact	[252,253,254]
Accuracy and Accountability	Processes for data accuracy and accountability	Ensures reliability of data	Requires resource allocation for auditing	[24,255,256]

between informed consent, privacy and anonymity, privacy and anonymization, accuracy and accountability, data security, data minimization, and transparency approach.

6.2. Regular Auditing and Monitoring

The continuous monitoring and auditing of AI systems are crucial to identify and address emerging biases throughout the AI lifecycle. Regular auditing and monitoring are crucial aspects of ensuring AI fairness in real-world applications.

Table 12. Regular auditing and monitoring comparison.

Method	Features	Pros	Cons	References
Disparate Impact Analysis	Measures disparate impact ratios	Easy to implement and interpret	Only captures one aspect of fairness (impact ratios)	[6,257,258]
Fairness-aware Performance Metrics	Simultaneously evaluates accuracy and fairness	Provides a holistic view of model performance and fairness	Choice of fairness metric may not fully capture desired notions of fairness	[259,260,261]
Bias Detection Techniques	Identifies biases in data or model predictions	Alerts to potential fairness issues early	May require domain expertise for interpreting and addressing identified biases	[71,262,263]
Algorithmic Fairness Dashboards	Real-time visualizations and metrics for monitoring	Enables continuous fairness monitoring	Complexity in designing comprehensive dashboards	[264,265,266]
Model Explanation and Interpretability	Provides insights into decision-making	Facilitates understanding of model behavior and potential biases	May not fully capture complex interactions in the model, leading to limited interpretability	[267,268,269,270]
Continual Bias Monitoring	Ongoing and regular assessment	Detects and addresses emerging fairness issues over time	May require significant resources for continuous monitoring	[47,271,272]

summarizes different methods for auditing and monitoring AI fairness, including disparate impact analysis, fairness-aware performance metrics, bias detection techniques, algorithmic fairness dashboards, model explanation and interpretability, and continual bias monitoring.

7. AI Fairness in Practice

AI fairness has a large number of real-world applications in a variety of fields, where it is critical to ensure that machine learning models do not perpetuate or amplify bias and discrimination. The areas where the current research and application work are more focused are AI-based social infrastructure and management and business applications.

Table 13. AI fairness in social administration practices.

Application	Issues	Mechanism	Opportunities	Challenges
Health Care	Racial and gender biases in diagnosis and treatment. Unequal healthcare due to socioeconomic factors.	diversifying representative datasets. Personalized treatment plans based on individual characteristics.	Enhancing healthcare access and outcomes for all individuals. Reducing healthcare disparities.	Ensuring patient privacy and data security. Addressing biases in data collection and data sources.
Education	Bias in admissions and resource allocation. Unequal access to quality education.	Fair criteria for admissions and resource allocation. Personalized learning for individual needs. Identifying and assisting at-risk students.	Reducing educational disparities. Enhancing learning outcomes for all students.	ethical considerations regarding data privacy in educational settings. avoiding undue focus on standardized testing.
Criminal Justice and Sentencing	Racial Bias in predictive policing and sentencing. Unfair allocation of resources for crime prevention.	focus on rehabilitation with regular auditing and updating the models with transparency in decision-making.	Reducing biased arrests and sentencing. Allocating resources more efficiently.	The ethical implications of using AI in criminal justice. Ensuring model accountability and avoiding “tech-washing”.

and

Table 14. AI fairness in business practices.

Application	Issues	Mechanism	Opportunities	Challenges
Recruiting	Bias in job ads and candidate selection. Lack of diversity in hiring.	Debiasing job descriptions, candidate screening and removing identifiable information, diversifying training data.	Increasing workforce diversity. Reducing hiring discrimination.	Balancing fairness and competence. Ensuring fairness across different demographics.
Lending and Credit Decisions	Discrimination in loan approvals. Lack of transparency in decision making.	Implementing fairness-aware algorithms, explaining model decisions, alternative data to creditworthiness.	Expanding access to credit for marginalized groups. Improving overall lending practices.	Striking a balance between fairness and risk assessment. Handling potential adversarial attacks on models.
Online Advertising	Targeting ads based on sensitive attributes. Reinforcing stereotypes through ad delivery.	Differential privacy to protect privacy, biased message screening, providing users preference controls.	Improving user experience and privacy protection. Fostering a positive brand image.	The balance between targeted ads and user privacy. Identifying and Addressing hidden biases in ad delivery.
Customer Service and Chatbots	biased responses and inappropriate interactions. Lack of understanding diverse linguistic expressions.	Training chatbots on inclusive and diverse datasets with reinforcement learning to improve interactions with feedback on bot behavior.	Enhancing user experience and customer satisfaction. Scaling customer support efficiently.	Minimizing harmful or offensive responses. Dealing with novel inputs and out-of-distribution data.

summarize common applications and case studies with a comparison of the approaches and challenges, respectively, including education [273,274,275,276,277] health care [52,278,279,280] criminal justice and sentencing [88,281,282,283], hiring and recruiting [284,285,286], lending and credit decisions [287,288,289,290,291,292], online advertising [8,293,294,295], customer service and chatbots [296,297,298,299,300,301,302,303].

7.1. Social Administration

Artificial intelligence (AI) has become an integral part of all industries, changing the way decisions and processes are managed. In recent years, the concept of AI fairness has gained prominence, especially in the field of social management. AI systems are increasingly being used in areas such as criminal justice, healthcare, and education. Table 13 summarizes the typical applications, including issues, mechanisms, opportunities and challenges.

7.1.1. Health Care

AI fairness can also be applied to healthcare diagnosis and treatment recommendation systems to reduce bias and ensure fairness in healthcare delivery, as some healthcare AI systems have been found to differ in the diagnosis of certain diseases among different racial groups [52,278]. The use of a fairness-aware algorithm improves the performance of the model and provides a fairer diagnosis for all patients [279,280].

AI applications in healthcare face challenges of interpretability, trust, data privacy, and security. Privacy concerns hinder data sharing, while complex AI models lack interpretability, impacting trust. Transparent and responsible data management is needed for data sharing while protecting privacy. Explainable AI can enhance trust by making AI recommendations understandable [304]. Aligning with healthcare governance measures can ensure trustworthy AI use. Addressing these issues can revolutionize medical decision making, improve outcomes, and foster equitable and patient-centered healthcare.

7.1.2. Education

Artificial intelligence fairness is applied to education technology to ensure equal opportunity for students regardless of their background [273]. This is because in reality, AI-powered tutoring systems exhibit biases when assigning tasks to students [274,275]. Therefore with the incorporation of fairness awareness, the system can adjust the recommendations to treat all students fairly [276,277].

AI algorithms in college admissions and resource allocation may unintentionally perpetuate biases, impacting opportunities and diversity. Personalized learning platforms might worsen educational disparities, particularly for marginalized students. Testing and assessment bias can lead to unfair evaluations, affecting self-esteem and prospects. Future work should design fairness-conscious admissions models and AI systems optimizing fairness to mitigate bias. Transparency and accountability measures should guide AI-based educational decisions. Incorporating equity in personalized learning algorithms and diverse educational content can promote equitable support. Culturally responsive education and diverse resources can also aid in reducing bias in assessments.

7.1.3. Criminal Justice and Sentencing

Utilizing AI fairness to address bias in risk assessment tools ensures that criminal sentencing decisions are fair [88,281]. In the criminal justice system, some AI-based risk assessment tools have been found to be racially biased, resulting in harsher sentences for some minorities [282]. Implementation of fairness awareness training resulted in a significant reduction in system bias [283].

Current research predominantly addresses racial, socioeconomic, and recidivism prediction biases. Data-driven disparities may lead to biased arrests, bail decisions, and sentencing. Biased recidivism prediction algorithms misclassify groups as high risk, perpetuating unfair treatment and higher incarceration rates for marginalized groups.

Future efforts should gather diverse, representative training data to mitigate bias. Fairness-aware algorithms and AI models optimizing fairness should be developed to prevent differential treatment based on race or socioeconomic status. Transparent risk assessment models can enhance interpretability, aiding defendants and legal professionals. Regular model audits are essential to identify and rectify potential biases in model deployment.

7.2. Business

Another important application area for AI Fairness is business. More and more AI technologies are also being continuously introduced into commercial applications, and Table 14 summarizes and compares several trending widely used scenarios.

7.2.1. Hiring and Recruiting

The integration of artificial intelligence (AI) in human resource management (HRM) introduces transformative enhancements. Figure 3 shows a recruitment process supported by AI throughout. AI-driven algorithms streamline CV screening and candidate profiling, while proctored assessments ensure secure remote testing. AI optimizes interview scheduling and personalizes HR training. Behavior tracking and personality analysis provide insights into candidate dynamics, and AI aids in appraisal monitoring through performance metric analysis. These applications collectively reshape HRM practices, enhancing efficiency and informed decision making.

In the applications of AI system in HRM, artificial intelligence fairness is applied to mitigate bias in automated hiring systems, ensuring equitable and non-discriminatory candidate selection [305,306]. Many AI-driven recruitment tools have exhibited biases, favoring specific candidates and overlooking job requisites [307,308]. Utilizing AI fairness techniques rectifies these model biases, fostering impartial hiring decisions independent of attributes unrelated to job proficiency [309]. Tackling these challenges and future endeavors in AI fairness is pivotal to harnessing the potential of AI for equitable recruitment, fostering diversity and inclusivity in workforce dynamics.

7.2.2. Loan and Credit Decisions

AI applications in loan and credit decision making aim to improve decision accuracy, speed, and fairness while maintaining prudent risk management. As shown in Figure 4, AI applications in loan and credit decision making involve leveraging artificial intelligence techniques to assess creditworthiness, streamline lending processes, and enhance risk management. These applications use AI algorithms to analyze various data sources, such as financial records, transaction histories, and alternative data, to make more accurate and efficient lending decisions. This aids in automating and optimizing the loan approval process, reducing human bias, and increasing access to credit for underserved populations. AI assists in fraud detection, predicting default risks, and personalizing loan terms based on individual borrower profiles.

To counteract bias in credit-scoring models and ensure equitable access to loans and credit opportunities for all individuals, AI fairness is applied with bias migration strategies [287,288]. Certain AI-driven credit scoring models have exhibited potential bias towards specific demographic groups [288,289]. Implementing bias mitigation techniques enhances model fairness, leading to more impartial lending determinations [290,291,292]. Looking ahead, future efforts should focus on integrating diverse data sources like rental histories or utility payments into credit assessments while maintaining fairness. Designing AI models capable of adapting to various data distributions, including non-traditional data, can also sustain fairness and accuracy. Incorporating fairness-aware explanations into AI models offers insight into achieving equitable credit decisions with transparency. Through these applications and ongoing research, the aim is to foster inclusive and just lending practices by minimizing bias and promoting unbiased access to credit [287,288,290,291,292].

7.2.3. Online Advertising

AI systems have a substantial role in online advertising, including targeted ad delivery, content censorship and related design. Figure 5 shows an example of AI applications according to the hierarchical taxonomy of online advertising. It can be seen that the AI support can enhance ad relevance and user experience, while also mitigating inappropriate content. However, challenges related to biases in ad targeting and content moderation necessitate the development of fairness-aware approaches to ensure equitable outcomes. In this context, AI technologies both facilitate and necessitate ongoing efforts to maintain fairness and effectiveness in online advertising practices.

To address the issue above, AI fairness is used for ad targeting to avoid promoting discriminatory or biased content to users based on their attributes [293]. Some online advertising platforms have experienced problems with certain ads being disproportionately shown to users from specific demographic groups [8,294]. By incorporating fairness constraints, the platform achieved more balanced ad targeting across all users [295]. AI algorithms in advertising may unintentionally yield biased ad targeting and content due to biased training data and content generation. Future work should center on fairness-aware targeting, bias audits, diverse data, and inclusive content to ensure fairness and inclusivity in ad delivery.

7.2.4. Customer Service

Artificial intelligence is also introduced into customer service, or the customer relationship management (CRM) system [296], which enhances customer interactions and support. As shown in Figure 6, the chatbots utilize natural language processing and sentiment analysis to understand customer queries and provide accurate, timely responses. They enable automated, efficient, and personalized customer interactions, improving user experience. AI-driven chatbots handle routine inquiries, offer real-time support, and gather insights for businesses to enhance their services. This technology aims to streamline customer service operations while ensuring effective and satisfactory customer interactions.

However, some customer service chatbots show biased responses to users who speak certain language dialects [297,298,299]. Aiming at this, AI fairness is used in chatbot design to avoid biased responses or inappropriate behavior towards users [300,301]. After implementing fairness checks, the chatbot provides culturally sensitive and fair interactions [302,303].

The current challenge in the customer service bot domain pertains to mitigating uncertain biased and inappropriate responses. Chatbots often unknowingly offer biased or offensive replies due to training data exposure, leading to customer dissatisfaction and reputational harm. Additionally, limitations in comprehending diverse linguistic expressions hinder accurate responses to various language forms, including slang. Further, inadequacies in addressing sensitive topics and emotional responses lead to inappropriate interactions in some customer service bots.

In prospective research, countering the aforementioned concerns requires embedding bias detection and mitigation mechanisms to identify and address biased language and responses in chatbot interactions. Mitigating biased replies can be achieved by adopting inclusive training data representing diverse user demographics and employing natural language processing techniques to enhance language comprehension. Continuous learning is essential for customer service bots to adapt and comprehend various language styles through user interactions.

8. Conclusions

This article introduces the study of fairness in artificial intelligence, detailing the background and definition of this concept. The article introduces the development process of the fairness problem in AI systems from the perspectives of practical applications and the current state of development, and reviews and discusses the main research directions for solving the fairness problem in AI—bias analysis and fairness training—respectively. In the course of the review, the ideas and implementations of each method are explained in detail, and their respective characteristics and occasions of use are compared. The article also explores measures to reduce bias and improve fairness in AI systems, reviews relevant problems and solutions in practical applications of AI fairness, and discusses possible future research directions. On the basis of the theoretical foundations and methodology of AI fairness, the paper also explores scenarios and application examples in practical applications, thus contributing to the current discussion on fair and unbiased AI systems.

This paper also provides an in-depth comparison of the characteristics, advantages, and disadvantages of each of the different methods, based on the collation of the state-of-the-art research. The results of the comparison will provide advisory support for future research and development. At the same time, this paper also summarizes some of the existing problems in existing applications and proposes some focuses and solution ideas for future research work. These summaries will provide ideas for the further development of fairness in future AI systems.

The information synthesized in this study comes from a variety of reliable sources. These sources include recent publications in prestigious academic journals, distinguished conference proceedings, and well-established online repositories dedicated to the fairness of AI. It is important to recognize that while these sources have contributed significantly to our understanding, there are limitations to the evidence they provide. These limitations stem primarily from nuances in the research methodology, sample size, and context, which may affect the generalizability of conclusions drawn from individual studies. The landscape of AI fairness is dynamic, with research and advancements continually shaping our understanding of its complexities. While our current coverage might have limitations due to the rapid pace of change and ongoing research, please know that we are committed to further studying and exploring this crucial subject.