Fair-CMNB: Advancing Fairness-Aware Stream Learning with Naïve Bayes and Multi-Objective Optimization

1. The authors should refer more to relevant studies from the last three years. The current manuscript is under-researched for the most recent results.

The literature review section has been updated to include the state-of-the-art methods presented in the domain of fairness aware learning.

2.The paper should provide a detailed explanation of the enhancements made to Naïve Bayes. How does the proposed method handle class imbalance and mitigate discrimination while maintaining efficiency and model interpretability? Clarification on the specific modifications made to the Naïve Bayes algorithm would enhance understanding.

Model interpretability is not part of our research problem. The key contributions of our works are:

1.     We challenge the Deep learning dogma by presenting a novel adaptation of Naïve Bayes (Fair-CMNB) to address fairness concerns in streaming environments where computational efficiency, model interpretability, and active learning are important.

2.     We mitigate discrimination as well as reverse discrimination (discrimination towards the privileged group) over the stream while simultaneously improving the predictive performance through multi-objective optimization.

3.     Fair-CMNB is also capable of dynamically handling concept drifts and class imbalance.

4.     Fair-CMNB is agnostic to the employed fairness notion (including the causal fairness notion FACE).

Section 4.2 in the manuscript explains how we handle class imbalance for fairness aware learning.

3.The paper mentions the use of multi-objective optimization. It would be beneficial to elaborate on how the multiple objectives are defined in this context and how the optimization process is conducted. What are the objectives, and how do they contribute to the overall fairness-aware stream learning?

Our multiple objectives are:

1.     Discrimination mitigation

2.     Improvement in Predictive performance (balanced accuracy)

The hyper-parameter λ directly influences discrimination score and balanced accuracy. We use multi-objective optimization to actively tune λ. Further details can be found in Section 4.4.1 of manuscript.

4.The paper mentions tuning hyperparameters, such as α and ϵ. Could you elaborate on the sensitivity of the model's performance to these hyperparameters? How robust is the model to changes in these values?

The hyper-parameter λ directly influences discrimination score and balanced accuracy. We use multi-objective optimization to actively tune λ. Section of 7.5 has been added to the manuscript along with Figure 3 which elaborates on hyperparameter sensitivity. IT is also detailed below for your reference:

The most important hyper-parameter in reducing discrimination is λ from Algorithm 2. We examined the effect of changing λ on the ability of our proposed model to reduce discrimination, as shown in a specific figure. We use the Adult Census dataset as a reference for this analysis. As can be seen in one figure, when the value of λ is 0.01, the discrimination value immediately drops to zero, indicating that this value is too large. With this value of λ, we achieve a balanced accuracy of 75.13%. If we decrease λ to a value of 0.001, the discrimination score decreases to a smaller and stable value after about 20,000 instances. The balanced accuracy is also not much affected with a value of 78.61%. If we further decrease the value of λ to 0.0001, the discrimination score does not reach a stable value until the end of the stream, although it decreases. This value of λ leads to a balanced accuracy of 79.93%. If we leave λ at 0.00001, the discrimination score does not decrease throughout the data stream, and the achieved balanced accuracy is 80.37%. Therefore, we chose the value 0.001 for λ, which provides a good trade-off between the balanced accuracy and the attenuation of the discrimination score.
 

5.The paper briefly mentions concept drift detection using Hinkley's method. How does the model adapt to different types and intensities of concept drift? Are there specific challenges or scenarios where the proposed model may struggle with concept drift?

We have used Page Hinkley’s drift detection method. Further deeper analysis is not performed in this work.  The aspects mentioned in the paper are not studied in our paper.

6.While Naïve Bayes is known for its scalability, how does the proposed model scale with increasing data volume and streaming speed? Are there any scalability challenges or limitations observed during experimentation?

Section 7.2 has been added to the manuscript to discuss the scalability of Fair-CMNB. The detail is also added here for your reference:
 Fair-CMNB adapts well to large data volumes. Law School is the smallest dataset with approximately 18,000 instances, while KDD and NYPD are much larger, each containing around 300,000 instances. As evident from the referenced tables, Fair-CMNB's performance remains consistent across both small (Law School) and large (KDD, NYPD) datasets. This demonstrates Fair-CMNB's efficient scalability with increasing data volume.

7.In the context of multi-objective optimization, how does the model strike a balance between fairness and predictive performance? Are there instances where maximizing fairness negatively impacts accuracy or vice versa?

Badar et al. Prove that a trade-off exists between fairness and predictive performance. Also, in our work we have found that a trade-off exists between fairness and balanced accuracy.

[1]   Badar, M., Nejdl, W., & Fisichella, M. (2023). FAC-fed: Federated adaptation for fairness and concept drift aware stream classification. Machine Learning, 112(8), 2761-2786.

1.  The authors have many statement with no justification or references. For example in "Deep learning algorithms have achieved significant success in various domains, including image and speech recognition, natural language processing, and many others. However, in fairness-aware stream learning, deep learning may not always be the best choice due to its high computational complexity. In contrast, traditional machine learning algorithms such as Naïve Bayes are often more efficient and require less computational resources than deep learning algorithms. This makes them more suitable for processing large volumes of data in real-time, which is essential for stream learning applications. Naïve Bayes requires less training data compared to deep learning models, which makes it appropriate for small datasets where the number of instances is limited."

These kind of statements must be either justified or reference(s) must be given.

The required references are added in the manuscript on page 2.

2. On page 15: "From the figure, it is evident that while both Fair-CMNB and FABBOO achieve". Which figure?

The cross-reference has been added.

3. Equations: Make sure that each Equation is explained in the text.

Our multiple objectives are:

1.     Discrimination mitigation

2.     Improvement in Predictive performance (balanced accuracy)

The hyper-parameter λ directly influences discrimination score and balanced accuracy. We use multi-objective optimization to actively tune λ. Further details can be found in Section 4.4.1 of manuscript.

4.I suggest to show percentage by how how much is there approach better than other methods

 The percentages are added in the mansucript in section 7.

5. References:  I recommend that the author check some recent references. They have only 1 paper from 2023, 2 paper from 2022 and 2 papers from 2021.

The literature review section has been updated to include the state-of-the-art methods presented in the domain of fairness aware learning.

1.   Some parts in the manuscript require citations, like: "Naïve Bayes are often more efficient and require less computational resources than deep learning algorithms. This makes them more suitable for processing large volumes of data in real-time."

The required references are added in the manuscript on page 2.

2.  Fair-CMNB -> What is CMNB? A long form is required at first.

Fair-CMNB means Fairness and Class Imbalance-aware Mixed Naïve Bayes.

3.  Section 3, protected and non-protected concepts, can be better presented (even maybe with a figure). Also, more real-life examples must be presented in terms of applicability. I highly recommend some support by references. Is this "protected" group concept important in data science; the reader should understand that.

The following example has been added to the manuscript:

In a loan approval scenario, a financial institution uses a machine learning model to automate decision-making, with gender as the sensitive attribute (classifying female applicants as the protected group and male as non-protected group), to address historical gender biases.

Sensitive attribute, protected group and non-protected group are well established terms in the domain of fairness aware learning [1][2][3].

[1]  Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., & Ntoutsi, E. (2022). A survey on datasets for fairness‐aware machine learning. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , 12 (3), e1452.

[2]  Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM computing surveys (CSUR) , 54 (6), 1-35.

[3]  Iosifidis, V., & Ntoutsi, E. (2020, October). -Online Fairness-Aware Learning Under Class Imbalance. In International Conference on Discovery Science (pp. 159-174). Cham: Springer International Publishing.

4. Are Equations 1 & 2 proposed by the authors?

These are well established notions of fairness. The required references are added in the manuscript on page 5.

5.  Regarding Figure 1, what are the state-of-the-art works having cascade these kinds of concepts such as imbalance monitoring, instance weighting, concept drift, etc?

The setting is inspired by [1]. While handling fairness in a streaming environment, class imbalance and concept drift are inherent problems. Therefore, we need to handle them along with fairness in streaming environments.

[1]  Iosifidis, V., & Ntoutsi, E. (2020, October). -Online Fairness-Aware Learning Under Class Imbalance. In International Conference on Discovery Science (pp. 159-174). Cham: Springer International Publishing.

6.  Algorithm 3 should be placed on the next page.

Thank you for pointing out. The algorithm is now on next page.

7.  In the datasets, do the authors assume the protected feature to be gender or any other assumption?

Table 1 mentions the sensitive attribute in each dataset. For example, ‘gender’ for Adult, KDD, Default, Law School, NYPD, and Loan datasets, ‘Race’ for Compas dataset, and ‘Marital status’ for Bank Marketing dataset.

8.  My concern for the results is about the previous work comparison in addition to the model comparisons as "Proposal versus Other Method". I understand that a fair comparison is not easy in terms of the different datasets and cascading modules for the proposed architecture, but at least I recommend having an explanation and overview related to the prior art results.

Table 2, Table 3, and Figure 2 presents comparison between our work and five baseline methods namely CSMOTE, OSBOOST, MS, FAHT, FABBOO.
All the baselines are trained using the same settings as Fair-CMNB to ensure fair comparison.

9. As the proposed model contains many cascading modules, could the authors shed light on the computational complexity of the model?

Section 5 has been added to the manuscript which explains the time complexity analysis of Fair-CMNB.