Review Reports - Classifying with the Fine Structure of Distributions: Leveraging Distributional Information for Robust and Plausible Naïve Bayes

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

I only have minor comments for the authors

It will be better to use legend for Figures 1B to 1G.
Remove the citations in the abstract.
What are the limitations of this study?
Are the authors thinking of developing a library package in R or Python for this method?

Author Response

We revised our manuscript accordingly and addressed the comments as detailed out in the following with our point-by point responses below the reviewers' comments, marked in red.

The authors presented a method to classify with the fine structure of distributions by leveraging distributional information for robust and plausible naïve Bayes. The method developed by the authors is interesting and rigorous. The authors validated their results by presenting some results and visualizations.
I only have minor comments for the authors
It will be better to use legend for Figures 1B to 1G.
We are grateful to the reviewer for his valid remarks that allows us to improve our work markedly. We included the legends in the Figure 1 for the Posteriori plots B-G.
Remove the citations in the abstract.
We removed the citations.
What are the limitations of this study?
The limitations are stated as its own paragraph close to the end of the discussion (lines 756-758):
“The PDENB is applicable to numeric tabular data as shown in the benchmark study. Theoretically, it is limited to independent features, however in practice it showed high performance despite significant dependency measures within the data.”

Are the authors thinking of developing a library package in R or Python for this method?
There is an R package developed which is published on CRAN along with this article which will be referenced in the final draft (right now there is only a GitHub link). There is no plan for any Python package. We clarify the situation now at the end of the instruction after the highlights (lines 121):
“We developed an open-access R package on CRAN (https://CRAN.R-project.org/package=PDEnaiveBayes).”

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

The paper proposes Bayes classification that leverages on Pareto Density Estimation. While not surpassing the best performing classifiers, the method is stable, scalable, and applicable for a variety of data classification problems.
The topic is interesting, I find the method applicable and of possible practical interest. Overall, I consider the paper deserves to be published after fixing a few issues.

One major issue with the paper is the vast bibliography cited, some references not providing enough information to be identified (like publisher), and many not exactly focusing on the topic the citation was used for. The authors use many self citations, with some of such references discussing only tangentially the problems they were cited for. Take, for instance, [9]: there is no publisher info, pages are definitely incorrect, and the paper discusses clustering. The paper discusses clustering and I do not consider it a good reference for complexity of choosing an appropriate distance measure.

Suggestions to improve the overall quality of the paper:
1. There are many references with incomplete bibliographic information, the whole reference list must be revisited.
2. When a reference is cited, please provide a brief description of the results in the reference and how the results actually apply to current work.
3. Tables and figures are out of range across the paper.

Other issues, typos:
- lines 23 and 26, abstract: citation in abstract is not, in general, a good practice. I don't see a problem if that is removed. Regardless, the citation format is different from the rest of the text and does not matches labels in References.
- line 121, citation: not properly formatted. What exactly is cited there? A problem formulation?
Also, I believe that the statement that $D$ contains $N=|G|$ data points is incorrect. $G$ appears to be a set of sets and the cardinality is definitely $k$. You may want to take the union of $C_i$'s to get the number of data points.
- line 127: close quotes for "knowledge
- line 134, fonts in (2) appear smaller for $p(C_i)$.
- line 139, formula (3): you clearly missed $\vec{x}_j$ argument from $h_i(\vec{x}_j)$; Also, it appears that you classify data point $x_j$. Please clarify.
- line 149, formula (4): what is $x^l$?
- line 429: "CRAN [42]" is not what you indicate.
- lines 443-452: please put the acronyms of all algorithms testes (like for PDENB, GNB and NPNB). They appear in Table 2, but it's not clear which one is which.
- section 3.1: a citation for datasets would be helpful here; where are they from? perhaps UCI datasets?

Author Response

We revised our manuscript accordingly and addressed the comments as detailed out in the following word file with our point-by point responses below the reviewers' comments, marked in red.

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

This paper introduces PDENB (Pareto Density Estimation Naïve Bayes), a parameter-free, non-parametric extension of the naïve Bayes classifier designed to be both robust and interpretable. Instead of assuming Gaussian or kernel densities for each feature, PDENB employs Pareto Density Estimation (PDE) to learn class-conditional likelihoods directly from the data. PDE chooses an adaptive, information-theoretic neighborhood radius for every point, capturing fine distributional structure (multimodality, skewness, tails) without manual bandwidth tuning. To suppress sampling noise, the raw PDE histogram is smoothed by FFT-based convolution with a Gaussian kernel whose bandwidth equals the Pareto radius; a monotone Hermite spline then yields a continuous likelihood function.

A second novelty is a “plausible-Bayes” safeguard against the low-evidence pitfall of standard Bayes’ rule: when the joint likelihood of an observation across all classes drops below an ABC-analysis-derived threshold ε, the algorithm checks whether a different class has a significantly closer mode. If so, the likelihoods of the two classes are conservatively adjusted, preventing counter-intuitive assignments in the distribution tails.

The authors benchmark PDENB on 14 diverse datasets (UCI plus a high-dimensional flow-cytometry set) against seven naïve-Bayes and k-NN variants. Using repeated 80/20 splits and Matthews Correlation Coefficient (MCC), PDENB achieves the lowest average rank (2.46) and statistically significant wins on most datasets, even when features are strongly correlated. Interpretability is delivered through mirrored-density plots of every feature’s class-conditional likelihood and 2-D Voronoi posterior maps that reveal compact high-confidence regions.

A dedicated biological application shows PDENB distinguishing bone-marrow from peripheral-blood samples with 99.3 % accuracy (98.8 MCC), outperforming a previous explainable-AI pipeline (96.8 %). The authors conclude that PDENB offers a scalable, assumption-free baseline whose visual diagnostics help domain experts discover feature-class relationships and validate decisions.

Questions
1. How sensitive is PDENB to the choice of the plausibility threshold ε, and could data-driven optimization of ε further improve accuracy?
2. The smoothing step uses a Gaussian kernel whose bandwidth equals the Pareto radius; have the authors experimented with alternative kernels or adaptive bandwidths that might better preserve sharp multimodal peaks?
3. PDENB currently estimates one Pareto radius per feature, independent of class. Could estimating class-specific radii capture finer structure without introducing excessive variance or computational cost?
4. The plausible-Bayes correction relies on mode separation; how does performance degrade when class modes overlap substantially or when distributions are strongly skewed?
5. The flow-cytometry application uses population frequencies as features. How would PDENB scale to raw, millions-of-cells event-level data, and are there plans to integrate automated gating or feature extraction within the same framework?

Author Response

We revised our manuscript accordingly and addressed the comments as detailed out in the following word document with our point-by point responses below the reviewers' comments, marked in red.

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors scrupulously followed the suggestions of the reviewer.
There are minor issues with the presentation: (i) most figures appear to be appropriate in size, but not aligned with the general text content; (ii) tables are still out of range (maybe put them in landscape format would solve the issue?).
I hope these issues can be easily taken care of during the camera ready editing.

Author Response

We thank the reviewer for their time and effort. We aligned tables and figures appropriately and used a landscape layout in some cases, to solve the issue. While taking a closer look at the tables, we also used the opportunity to fix a typo in our program for computing the statistical tests and thus updated some values in the tables of the statistical tables, which does not influence the end result in any way.

Reviewer 3 Report

Comments and Suggestions for Authors

This paper proposes the Plausible Pareto Density Estimation-based Naive Bayes classifier (PDENB), an innovative baseline classifier addressing key limitations of traditional Naive Bayes. Unlike conventional methods relying on Gaussian mixtures or kernel density estimation with rigid distributional assumptions, PDENB leverages nonparametric Pareto Density Estimation (PDE) to model feature distributions without prior assumptions. It incorporates smoothing via Gaussian convolution and plausibility correction to resolve misclassifications of low-evidence observations, a critical pitfall in Bayesian theory. Evaluated on 14 datasets (including UCI repositories and a cell population dataset), PDENB achieved the top average rank (2.46) among 8 classifiers, with MCC ≥ 0.95 on multiple datasets. In a practical application distinguishing blood from bone marrow via flow cytometry, it reached 99.3% accuracy, outperforming existing methods. PDENB also supports interpretability through mirrored-density plots and 2D Voronoi tessellations, enabling insight into class-conditional distributions and decision boundaries. Its strength lies in robust, scalable performance across diverse data scenarios while maintaining transparency. 1. How does the choice of the δ parameter in PDENB’s plausibility correction influence classification stability and conservatism? 2. Compared with state-of-the-art non-baseline classifiers (e.g., random forests), what are PDENB’s performance gaps on highly imbalanced datasets? 3. Why is a class-independent Pareto radius preferred over class-dependent estimation, and what empirical evidence supports this design choice? 4. Can PDENB be extended to handle categorical features, and if so, what modifications to the PDE-based density estimation are required? 5. How does PDENB’s computational efficiency scale with datasets containing millions of samples and hundreds of features? 6. What measures could further enhance PDENB’s robustness when mode estimation fails in overlapping or noisy feature distributions?

Author Response

We again thank the reviewer for the time it took to review the paper a second time. The responses are marked in red below in the word file.

Author Response File: Author Response.docx