Review Reports - A Machine-Readable Value-Added Catalog for Uncertain-Type Blazars in the Fifth Roma-BZCAT

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The authors construct a machine-readable value-added catalog for the 227 uncertain-type (BZU) blazars in the fifth Roma-BZCAT. Supervised classifiers are trained on the labeled BZB, BZQ, and BZG sources using a compact set of multiwavelength catalog features, with two missing-data strategies compared under a common cross-validation protocol. Combining a gradient-boosted tree model and a neural tabular model in a stacking ensemble, they assign three-way class probabilities to every BZU source, flag a small subset as ambiguous candidates for follow-up, and use feature-attribution methods to identify the observables driving the classification. However, the manuscript in its current form lacks sufficient clarity. In particular, it is not established that the classifier is learning genuine multiwavelength discrimination rather than reproducing the catalog's own labeling criterion, and the reliability of the tabulated probabilities is not demonstrated. I have provided detailed comments below.

"missing_Z" and "Z" together carry more SHAP weight than the other 10 features combined (Table 9 and Table 10). But redshift measurability is not an independent observable. It is a direct consequence of the same optical spectroscopy property that defines BZB vs. BZQ in 5BZCAT. Likewise BZG is defined by host-galaxy dominance, which is intrinsically a low-redshift condition, so Z encodes that label too. The classifier is therefore substantially recovering the catalog's own labeling rule rather than learning independent multiwavelength discrimination. The authors should clarify the consequence of this for the results obtained.
\alpha_ro, \alpha_rx, \alpha_ox as estimated in the imputation branch are exact algebraic combinations of F_5GHz, R_mag, and F_X. But all of the latter are already features 2, 4, 5 in Equatoin 1. For every source where the \alpha values are imputed, the "physically motivated estimate" is not adding any information. Could the authors clarify if it is redundant? Is it possible to state what fraction of each \alpha is catalog supplied vs. computed?
In the ten sources listed in Table 8, 5BZUJ0058−5659 and 5BZUJ0100+0745 have identical probability vectors to three decimals. The probability vector of 5BZUJ0040+4050 is also the same (within 0.001). If I understand correctly, identical outputs imply identical post-imputation feature vectors. This means these sources have most features missing and were filled with the same fold medians, and the "classification" is the model's response to a missingness pattern, not to any measurement of that source. Finding such degenerate clusters in a sample of 10 implies many more in the full 227 list. The authors should add a column per source mentioning the number of genuinely observed features, report the distribution of that count across 227, and quantify how many BZU sources share a degenerate feature vector. This is essential for anyone using the catalog.
Section 3.6 lists log loss, multiclass Brier score, and expected calibration error among the evaluation metrics, but none of these appears in the manuscript; Table 4 reports only accuracy, macro-F1, balanced accuracy, ROC-AUC, PR-AUC, and MCC, and the calibration metrics are deferred to an accompanying machine-readable file (Table 4 note). Every metric that does appear is insensitive to calibration, namely accuracy, macro-F1, and balanced accuracy depend only on the arg max, and ROC-AUC and PR-AUC only on the ranking. The manuscript therefore provides no evidence bearing on whether the tabulated posteriors are quantitatively meaningful.
All performance figures in Tables 3, 4 and 5 are reported as single-point estimates without uncertainties. Since the evaluation protocol is stratified five-fold cross-validation (Section 3.5), per-fold values are already available, and reporting mean +/- standard deviation across folds would require no additional computation. In its present form, the manuscript gives the reader no basis for judging which of the tabulated differences are meaningful. This matters directly for the central model-selection claim. Line 289 states that the stacking ensemble achieves the best macro-F1 and balanced accuracy, but the margin over TabNet in macro-F1 is 0.890 - 0.883 = 0.007. Macro-averaging weights the BZG class equally with BZB and BZQ despite its much smaller support (274 sources, i.e., roughly 55 per fold), so a small number of BZG sources crossing a decision boundary can shift macro-F1 by an amount comparable to or larger than this margin. The reported difference may therefore be within fold-to-fold scatter. I ask the authors to report per-fold means and standard deviations for all metrics in Tables 3 and 4, and to support the pairwise model comparisons with a test that accounts for the shared evaluation sample.
All performance figures come from cross-validation on the 3334 labeled sources, i.e., objects that 5BZCAT classified confidently. The model is then applied to the 227 BZU sources, which are uncertain precisely because their classification is difficult. No performance estimate on sources of this kind is presented, so it is difficult to judge whether an accuracy of 0.930 carries over to the sample the catalog is about. This is of particular concern given the model's reliance on redshift availability, which is informative for the labeled sample but least reliable for BZU objects. The authors should either provide evidence that the reported metrics are indicative of performance on uncertain-type sources, or state clearly that they are not.
The abstract and conclusions call the 13 flagged objects "potentially transitional cases." Given item 4, the more likely driver is poor data coverage. I recommend the authors plot maximum posterior probability against the number of genuinely observed (non-imputed) features, across all 227. If the flagged sources cluster at low observed-feature counts, low confidence is tracking data coverage. If flagged sources appear across the full range of coverage, some well-observed sources also landing near the boundary, then some of the ambiguity is real, and the physical interpretation survives for those.

Author Response

We thank the reviewer for the careful and technically important assessment. We have re-run the model comparison, revised the manuscript, and regenerated the publication-facing BZU catalog. The central revision is a change in scope: the catalog now reports LightGBM probabilities as conditional similarity scores to the existing 5BZCAT labels, rather than presenting the stacking outputs as independently validated physical posteriors.

Comments 1: “"missing_Z" and "Z" together carry more SHAP weight than the other 10 features combined (Table 9 and Table 10). But redshift measurability is not an independent observable. It is a direct consequence of the same optical spectroscopy property that defines BZB vs. BZQ in 5BZCAT. Likewise BZG is defined by host-galaxy dominance, which is intrinsically a low-redshift condition, so Z encodes that label too. The classifier is therefore substantially recovering the catalog's own labeling rule rather than learning independent multiwavelength discrimination. The authors should clarify the consequence of this for the results obtained.”

Response 1: Thank you for pointing this out. We agree that redshift and redshift availability are not independent astrophysical observables in the same sense as radio, X-ray, or gamma-ray fluxes. We have therefore revised the manuscript to make this limitation explicit. In the revised version, redshift and its missingness indicator are treated as optical-spectroscopy-related catalog-label proxies. We also added controlled ablation experiments in which redshift is removed, and then both redshift and redshift availability are removed. These changes can be found in the Abstract, Page 1; Section 2.1, Page 3, Paragraphs 1-2; Section 2.2, Page 3, Paragraph 2; Section 4.2, Page 9, Paragraph 3 and Table 5; Discussion, Page 15, Paragraphs 1-2; and Conclusions, Page 16, Paragraph 2.

Comments 2: “\alpha_ro, \alpha_rx, \alpha_ox as estimated in the imputation branch are exact algebraic combinations of F_5GHz, R_mag, and F_X. But all of the latter are already features 2, 4, 5 in Equatoin 1. For every source where the \alpha values are imputed, the "physically motivated estimate" is not adding any information. Could the authors clarify if it is redundant? Is it possible to state what fraction of each \alpha is catalog supplied vs. computed?”

Response 2: Thank you for pointing this out. We agree that formula-derived indices are algebraic combinations of their parent flux/magnitude features and add no independent information. They are now described only as transparent fallbacks for missing catalog fields, with source-level provenance retained. For BZU sources, the catalog-supplied / formula-calculated / fold-median-imputed counts are 213/3/11 for $\alpha_{ro}$, 148/0/79 for $\alpha_{rx}$, and 145/0/82 for $\alpha_{ox}$. The changes are in Sections 2.2, 2.3, 5.1, 5.4, and 6.

Comments 3: “In the ten sources listed in Table 8, 5BZUJ0058−5659 and 5BZUJ0100+0745 have identical probability vectors to three decimals. The probability vector of 5BZUJ0040+4050 is also the same (within 0.001). If I understand correctly, identical outputs imply identical post-imputation feature vectors. This means these sources have most features missing and were filled with the same fold medians, and the "classification" is the model's response to a missingness pattern, not to any measurement of that source. Finding such degenerate clusters in a sample of 10 implies many more in the full 227 list. The authors should add a column per source mentioning the number of genuinely observed features, report the distribution of that count across 227, and quantify how many BZU sources share a degenerate feature vector. This is essential for anyone using the catalog.”

Response 3: Thank you for this suggestion. We added the per-source column `N_Genuinely_Observed_Features`; missingness indicators and formula-derived indices are excluded from this count. The BZU distribution is 2, 5, 12, 48, 19, 13, 88, and 40 sources with 2 through 9 genuinely observed features, respectively. We also assign a post-imputation feature-vector identifier and group size. No two of the 227 BZU sources have the same post-imputation 12-dimensional feature vector. The apparent agreement of some previously displayed rounded probabilities therefore did not indicate duplicate feature vectors. The changes are in Section 4.2 and the accompanying machine-readable tables.

Comments 4: “Section 3.6 lists log loss, multiclass Brier score, and expected calibration error among the evaluation metrics, but none of these appears in the manuscript; Table 4 reports only accuracy, macro-F1, balanced accuracy, ROC-AUC, PR-AUC, and MCC, and the calibration metrics are deferred to an accompanying machine-readable file (Table 4 note). Every metric that does appear is insensitive to calibration, namely accuracy, macro-F1, and balanced accuracy depend only on the arg max, and ROC-AUC and PR-AUC only on the ranking. The manuscript therefore provides no evidence bearing on whether the tabulated posteriors are quantitatively meaningful.”

Response 4: Thank you for this suggestion. We now report log loss, multiclass Brier score, ECE, fold summaries, and a reliability diagram. LightGBM has out-of-fold log loss 0.217, Brier score 0.116, and ECE 0.029. Stacking has higher balanced accuracy (0.907 versus 0.888) but worse log loss (0.258) and Brier score (0.120). We therefore changed the publication-facing probability columns from stacking to LightGBM. The manuscript also states that internal calibration metrics do not validate calibration for the BZU population without an external uncertain-type test set. The changes are in the Abstract, Sections 3.5, 3.6, 4.2, 5.4, and 6.

Comments 5: “All performance figures in Tables 3, 4 and 5 are reported as single-point estimates without uncertainties. Since the evaluation protocol is stratified five-fold cross-validation (Section 3.5), per-fold values are already available, and reporting mean +/- standard deviation across folds would require no additional computation. In its present form, the manuscript gives the reader no basis for judging which of the tabulated differences are meaningful. This matters directly for the central model-selection claim. Line 289 states that the stacking ensemble achieves the best macro-F1 and balanced accuracy, but the margin over TabNet in macro-F1 is 0.890 - 0.883 = 0.007. Macro-averaging weights the BZG class equally with BZB and BZQ despite its much smaller support (274 sources, i.e., roughly 55 per fold), so a small number of BZG sources crossing a decision boundary can shift macro-F1 by an amount comparable to or larger than this margin. The reported difference may therefore be within fold-to-fold scatter. I ask the authors to report per-fold means and standard deviations for all metrics in Tables 3 and 4, and to support the pairwise model comparisons with a test that accounts for the shared evaluation sample.”

Response 5: Thank you for this suggestion. The electronic tables now provide the five-fold means and standard deviations for every metric, and paired stratified bootstrap comparisons on the shared out-of-fold sources. The stacking--LightGBM macro-$F_1$ difference is 0.007 (95% CI [-0.001, 0.016], $p=0.087$), so we no longer claim that stacking is meaningfully superior in macro-$F_1$. In contrast, stacking has significantly worse log loss (+0.041; 95% CI [0.030, 0.052]) and Brier score (+0.0043; 95% CI [0.0001, 0.0085]). The changes are in Sections 3.6 and 4.2.

Comments 6: “All performance figures come from cross-validation on the 3334 labeled sources, i.e., objects that 5BZCAT classified confidently. The model is then applied to the 227 BZU sources, which are uncertain precisely because their classification is difficult. No performance estimate on sources of this kind is presented, so it is difficult to judge whether an accuracy of 0.930 carries over to the sample the catalog is about. This is of particular concern given the model's reliance on redshift availability, which is informative for the labeled sample but least reliable for BZU objects. The authors should either provide evidence that the reported metrics are indicative of performance on uncertain-type sources, or state clearly that they are not.”

Response 6: Thank you for this suggestion. We now state this limitation explicitly in the Abstract, Results, Discussion, and Conclusions. Cross-validation measures performance only within the confidently labeled BZB/BZQ/BZG reference sample; it does not estimate accuracy or calibration on BZU objects. We identify uniform follow-up spectroscopy and external validation as necessary for that purpose. The changes are in the Abstract, Sections 4.2, 5.4, and 6.

Comments 7: “The abstract and conclusions call the 13 flagged objects "potentially transitional cases." Given item 4, the more likely driver is poor data coverage. I recommend the authors plot maximum posterior probability against the number of genuinely observed (non-imputed) features, across all 227. If the flagged sources cluster at low observed-feature counts, low confidence is tracking data coverage. If flagged sources appear across the full range of coverage, some well-observed sources also landing near the boundary, then some of the ambiguity is real, and the physical interpretation survives for those.”

Response 7: Thank you for this suggestion. We added a figure of maximum LightGBM probability versus genuinely observed feature count. The 17 flagged sources are not concentrated only at low coverage; all 7 sources with three or fewer genuinely observed features are unflagged by this criterion. We therefore removed the phrase “potentially transitional cases.” The flags are now described as probability-separation / model-uncertainty flags that prioritize follow-up observations, not as physical evidence for a transition. The changes are in Sections 4.2, 5.4, and 6.

Comments - Editorial check: “During our routine checks, we noted that the self-citation rate is 43.14% (22 out of 51 references), which is above the journal's standard threshold. The specific references contributing to this rate are Refs. 7, 8, 10, 11, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 30, 32, 34, 40, 43, and 44. Please consider replacing some of these self-citations with other appropriate references to reduce the overall self-citation rate. If you believe specific self-citations are essential and cannot be reasonably reduced, please provide a brief justification in your response.”

Response - Editorial check: Thank you for this editorial check. We removed the self-citations identified in the editorial check where they were not essential to the argument, removed the corresponding unused bibliography entries, and rewrote the surrounding context using foundational or independent references. We then expanded the bibliography with independent references directly relevant to blazar optical classification, spectroscopic redshift incompleteness, probability calibration, proper scoring rules, uncertainty under dataset shift, statistical comparison after cross-validation, and external catalog completeness.

Reviewer 2 Report

Comments and Suggestions for Authors

Review of "A Machine-Readable Value-Added Catalog for Uncertain-Type Blazars in the Fifth Roma-BZCAT" by Jingtian Zhu et al.

This manuscript presents a machine-readable value-added catalog for uncertain-type blazars in the Fifth Roma-BZCAT. The paper is well written, detailed, and thorough. The authors clearly present the limitations of their analysis and provide a thoughtful discussion of the physical interpretation of their results, rather than focusing solely on the machine learning aspects. Overall, I find this to be a valuable contribution that will be useful to the community. I have only a few minor comments that I believe could further improve the manuscript.

1. Abstract: The abstract contains more technical details than necessary. In particular, I suggest removing lines 13–16 to make the abstract more concise and focused on the main results.

2. References to journal names: In lines 68 and 70, I do not think it is necessary to mention the journal names in the text. Similarly, for line 392, I suggest simply writing:

"This work is also complementary to our related fine-grained 4FGL-DR4 classification study [44]."

The reference list already indicates that the cited work has been submitted, so there is no need to mention this explicitly in the main text.

3. Imputation method: The description of the imputation procedure (lines 120–123) is rather brief. I recommend providing a more detailed explanation of the adopted method so that readers can better understand how missing values were handled and assess the robustness of the approach.

4. Future work in relation to Sec.5: The feature importance analysis highlights redshift and R-band magnitude as particularly informative features. This suggests a natural direction for future work: improving the completeness of the input catalog by incorporating data from external resources. At present, the BZCAT entries do not include all available information from more recent catalogs. For example:

* 5BZUJ0040+4050 has no redshift listed in BZCAT, whereas a redshift is available in OCARS (which references Milliquas).
* 5BZUJ2025+3343 has no R-band magnitude in BZCAT, while an r-band magnitude is available in Pan-STARRS DR2.

Incorporating such complementary information could further improve the completeness of the catalog and potentially enhance the performance of future classification efforts.

Overall, I recommend publication of the manuscript after these minor revisions.

Author Response

We sincerely thank the reviewer for the careful and technically important assessment of our manuscript. We have revised the manuscript substantially in response to the comments. The central revision is a clarification of scope: the reported probabilities are now presented as similarity scores to existing 5BZCAT labels conditional on available catalog measurements, rather than as independently validated physical posterior probabilities for BZU sources. We have also added redshift ablations, missing-data provenance, feature-vector diagnostics, probability-calibration metrics, a reliability diagram, fold-level uncertainty summaries, paired stratified bootstrap comparisons, and additional independent references. All revised text in the manuscript is marked in red.

Comments 1: “The abstract contains more technical details than necessary. In particular, I suggest removing lines 13-16 to make the abstract more concise and focused on the main results.”

Response 1: Thank you for pointing this out. We agree. We shortened the Abstract by removing several technical validation details and retained only the essential information: the catalog task, the compact input set, the role of redshift and redshift availability, the reason for adopting LightGBM probabilities, and the final BZU class counts. This makes the Abstract more focused on the catalog product and its interpretation. The change is in the Abstract.

Comments 2: “In lines 68 and 70, I do not think it is necessary to mention the journal names in the text. Similarly, for line 392, I suggest simply writing: “This work is also complementary to our related fine-grained 4FGL-DR4 classification study [44].” The reference list already indicates that the cited work has been submitted, so there is no need to mention this explicitly in the main text.”

Response 2: Thank you for pointing this out. We agree. We checked the revised manuscript and removed unnecessary journal-name or submission-status statements from the main text. The text now relies on the reference list for bibliographic status and keeps the main manuscript focused on the scientific comparison. The relevant changes are in the Introduction and in the discussion of previous machine-learning classifications.

Comments 3: “The description of the imputation procedure is rather brief. I recommend providing a more detailed explanation of the adopted method so that readers can better understand how missing values were handled and assess the robustness of the approach.”

Response 3: Thank you for pointing this out. We agree. We expanded the missing-data subsection to state the sequence of operations explicitly. Missingness indicators are set before imputation; medians are fitted only on the training subset of each cross-validation fold and then applied to the validation subset; formula-derived spectral slopes are used only when the catalog index is absent and all parent measurements are available; and any remaining missing values are fold-median imputed. We also state that this design avoids validation-set leakage and keeps the provenance of the broad-band index substitutions traceable. The change is in Section 2.3.

Comments 4: “The feature importance analysis highlights redshift and R-band magnitude as particularly informative features. This suggests a natural direction for future work: improving the completeness of the input catalog by incorporating data from external resources. Examples include redshift information from OCARS/Milliquas and optical photometry from Pan-STARRS DR2.”

Response 4: Thank you for pointing this out. We agree. We added a paragraph to the limitations and follow-up section explaining that future versions could cross-match BZU sources with complementary resources such as OCARS, Milliquas, and Pan-STARRS DR2 to update redshifts and optical photometry when reliable matches are available. We also clarify that the present study intentionally uses the 5BZCAT entries without external augmentation, so that the current results quantify similarity conditional on the original catalog measurements. The change is in Section 5.4.

Reviewer 3 Report

Comments and Suggestions for Authors

This manuscript describes a study to use machine-learning techniques attempting to identify blazars classified as uncertain type in the 5th edition of the RomaBZCAT catalog. The authors provide their results in detailed machine-readable tables where the probability of the different classifications are listed. Besides these results, they also list which observational features are more important for the classification.

I found this study worth to publish and important in studying the different type of blazars. However I have a few suggestions before I can recommend it to publication.

I think the most important aspect to improve in the manuscript is the description of the various machine-learning approaches employed. The authors must be experts of this field, however to reach a wider audience I suggest to explain several methods (and abbreviations) in more details. For example, I only have a vague guess what SHAP means. Or what do ROC-AUC and PR-AUC stand for. I also call the authors attention to the fact that as far as I know there is an abbreviations' listings in the journal template. Nevertheless a simple explanation and possible references (where needed) of the abbreviations used would be essential.

Abstract:
I suggest to include less details in the abstract, e.g. the stacking accuracies and several other numbers, especially the PR-AUC, TabNet, etc. which as I mentioned above may not be common knowledge.

Also instead of listing "broad-band spectral indices ...", I would rephrase something like spectral slopes between optical, X-ray, and radio bands

Introductions:

I am not familiar with the expression "few-shot learning". I suggest the authors to elaborate on that a bit (or rephrase it).

last paragraph: Please include here a reference to the 5th Roma-BZCAT catalog.

Line 82, "In the imputation branch ..." - I think this is not needed here, as a more detailed description of both the imputation branch and the complete-case analysis are given later.

Sect. 2.

I wonder if the inclusion of BL Lac and BL Lac candidates in the BZB group impacts the results. Does it not weaken the BL Lac identification? What is the difference between the BL Lac and BL Lac candidates in the original RomaBZCAT? What are the percentages of these two groups, and would not be better (more prudent) to only include firmly classified BL Lac objects in the training set?

Below eq. 1: two radio flux density measures are listed in equation (1), at 1.4 GHz and 5 GHz. Both of them were used? Then rephrase the sentence below the equation.

The radio-to-optical and X-ray spectral indices were calculated with which radio flux density value? Are there sources with radio measurements made at only a frequency below 5 GHz, how they were treated?

Sect. 3.2

What is a softmax function? It has the form as given in equation 15? If yes, I suggest rephrase this like: "through the softmax function as follows:"

Please explain the meaning of SHAP values.

Sect. 3.4

Line 194: "elementwise multiplication" - means a vector multiplication? (or a tensor vector multiplication?)

In equation 21, the Wd multiplication is a vector multiplication? Otherwise how to add a scalar to a vector?

Sect. 3.6

What are TP, FN?

Table 4: Please use the same number of digits within each column (PR-AUC)

Minor comments:
Table 6 is referenced in the text as Table 4.3
Sect. 6. line 430: "requiring follow-up" I suggest to add a word like observations/investigations/studies.

Author Response

Comments 1: “The manuscript should describe the machine-learning approaches and abbreviations in more detail for a wider audience. Examples include SHAP, ROC-AUC, and PR-AUC. The journal template also includes an abbreviations list.”

Response 1: Thank you for pointing this out. We agree. We expanded the methods text to define SHAP as SHapley Additive exPlanations and to explain that SHAP approximates Shapley-value feature contributions. We also define ROC-AUC as the area under the receiver operating characteristic curve and PR-AUC as the area under the precision-recall curve. In addition, we added an Abbreviations section listing the main astronomical and machine-learning abbreviations used in the manuscript. The changes are in the Introduction, Sections 3.2, 3.6, 3.7, and the new Abbreviations section.

Comments 2: “The abstract contains too many technical details and should avoid several machine-learning quantities that may not be common knowledge. The phrase “broad-band spectral indices” could also be rephrased as spectral slopes between optical, X-ray, and radio bands.”

Response 2: Thank you for pointing this out. We agree. We shortened the Abstract and removed most detailed metric values. We also replaced the more technical phrase “broad-band indices” with “spectral slopes connecting the radio, optical, and X-ray bands.” The change is in the Abstract.

Comments 3: “The expression “few-shot learning” may not be familiar and should be elaborated or rephrased.”

Response 3: Thank you for pointing this out. We agree with the concern. In the revised manuscript, this expression is not used in the Introduction. The related-work discussion now uses broader wording about statistical and machine-learning classifications of Fermi-LAT sources. No additional manuscript text using “few-shot learning” remains.

Comments 4: “The last paragraph of the Introduction should include a reference to the fifth Roma-BZCAT catalog.”

Response 4: Thank you for pointing this out. We agree. We added the 5BZCAT reference to the final paragraph of the Introduction when introducing the present BZU value-added catalog. The change is in Section 1.

Comments 5: “The discussion of the imputation branch is not needed in the Introduction because the detailed description is given later.”

Response 5: Thank you for pointing this out. We agree. The Introduction no longer contains this methodological detail. The imputation and complete-case procedures are described in the methods and sample-construction sections instead. The relevant text is in Sections 2.3 and 3.5.

Comments 6: “I wonder if the inclusion of BL Lac and BL Lac candidates in the BZB group impacts the results. Does it not weaken the BL Lac identification? What is the difference between the BL Lac and BL Lac candidates in the original Roma-BZCAT? What are the percentages of these two groups, and would not be better, or more prudent, to only include firmly classified BL Lac objects in the training set?”

Response 6: Thank you for pointing this out. We agree that this point should be explicit. We added the counts and percentages: within the BZB training class, 1059 of 1151 sources (92.0%) are listed as BL Lac objects and 92 sources (8.0%) as BL Lac candidates. We retain both groups because both are part of the 5BZCAT BZB convention and because the aim is to estimate similarity to the existing 5BZCAT classes, not to build a pure confirmed-BL-Lac classifier. We also added a caution that the resulting probabilities should be interpreted as similarity to the 5BZCAT BZB class. The change is in Section 2.1.

Comments 7: “Below Equation (1), two radio flux-density measures are listed, at 1.4 GHz and 5 GHz. Were both of them used? Please rephrase the sentence below the equation.”

Response 7: Thank you for pointing this out. We agree. We clarified that both radio flux-density entries are used as separate input features. The 1.4/0.843 GHz measurement preserves low-frequency catalog information, whereas the 5 GHz measurement is the radio anchor used for estimating radio-optical and radio-X-ray spectral slopes when a catalog index is absent. The change is in Section 2.2.

Comments 8: “The radio-to-optical and radio-to-X-ray spectral indices were calculated with which radio flux-density value? Are there sources with radio measurements made only at a frequency below 5 GHz, and how were they treated?”

Response 8: Thank you for pointing this out. We agree that this needed clarification. We now state that the 5 GHz flux density is used for the radio-to-optical and radio-to-X-ray index estimates. If a source lacks the 5 GHz measurement, we do not substitute the lower-frequency 1.4/0.843 GHz flux into these formulas; the corresponding index remains missing until the fold-median imputation step. The change is in Section 2.3.

Comments 9: “What is a softmax function? It has the form as given in Equation 15? If yes, I suggest rephrasing this as: ‘through the softmax function as follows:’”

Response 9: Thank you for pointing this out. We agree. We revised the LightGBM and TabNet descriptions to introduce the probability equations as softmax transformations of the class-score vector. The changes are in Sections 3.2 and 3.3.

Comments 10: “Please explain the meaning of SHAP values.”

Response 10: Thank you for pointing this out. We agree. We added a definition of SHAP and explain that SHAP assigns additive feature contributions to the model output by approximating the Shapley-value concept from cooperative game theory. The change is in Section 3.7.

Comments 11: “Line 194: ‘elementwise multiplication’ - does this mean vector multiplication, or tensor-vector multiplication?”

Response 11: Thank you for pointing this out. We agree that the phrase could be clearer. We now state that $\odot$ denotes element-wise, or Hadamard, multiplication between two vectors of the same length. The change is in Section 3.3.

Comments 12: “In Equation 21, is the Wd multiplication a vector multiplication? Otherwise, how can a scalar be added to a vector?”

Response 12: Thank you for pointing this out. We clarified the notation by stating that $\mathbf{W}$ is a weight matrix and $\mathbf{b}$ is a bias vector. Therefore, $\mathbf{W}\mathbf{d}+\mathbf{b}$ gives a class-score vector, not a scalar plus a vector. The change is in Section 3.3.

Comments 13: “What are TP and FN?”

Response 13: Thank you for pointing this out. We agree. We now define TP, FP, FN, and TN as true positives, false positives, false negatives, and true negatives in the evaluation-metrics subsection. The change is in Section 3.6.

Comments 14: “Table 4: Please use the same number of digits within each column, including PR-AUC.”

Response 14: Thank you for this suggestion. We checked the table formatting and kept the numeric columns, including PR-AUC, to a consistent three-decimal presentation in the compact main-text tables. The change is reflected in Table 4.

Comments 15: “Table 6 is referenced in the text as Table 4.3.”

Response 15: Thank you for this suggestion. We checked and corrected the table cross-references so that LaTeX labels are used consistently. The revised manuscript no longer contains the erroneous “Table 4.3” reference.

Comments 16: “Section 6, line 430: after ‘requiring follow-up’, please add a word such as observations, investigations, or studies.”

Response 16: Thank you for pointing this out. We agree. We revised the wording to refer to “follow-up observations” where appropriate. The change is in Section 3.1 and is consistent with the discussion of follow-up priorities.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have taken into account and improved the manuscript. I recommend the manuscript for publication in its current form.

Author Response

Comments: “The authors have taken into account and improved the manuscript. I recommend the manuscript for publication in its current form.”

Response: We sincerely thank you for your careful evaluation of the revised manuscript and for recommending it for publication in its current form. We greatly appreciate your constructive comments during the review process, which helped us improve the clarity and quality of the manuscript.

Reviewer 3 Report

Comments and Suggestions for Authors

I thank the authors for responding my questions and making the suggested changes. I have a few, mostly minor comments, before I can recommend the manuscript for publication.

Abstract:
Please provide the full name of LightGBM.

Introduction:
Please provide references for LightGBM, TabNet, out-of-fold stacking.
The meaning of "macro-F1" also requires a brief explanation.

Sect. 2.
Please be more specific of the radio flux density measurements. According to equation 1, it seems, there are flux density measurements at most at two frequencies, 1.4 and 5 GHz. Then after the equation, a third frequency, 0.843 GHz is being mentioned. Please clarify, what really is given in the catalog, and whether you used 0.843 GHz flux densities or not.

After equation 1, please give separately the meaning of f_radio,1.4GHz and f_radio,5GHz. E.g. "..., f_radio,1.4GHz and f_radio,5GHz gives the radio flux density at 1.4 GHz and 5 GHz respectively, ..."

Equation 5: Please give which magnitude-to-Jansky scale is used, with a reference, and if possible explain why that one is used.

Sect. 3.2
Line 198: You already gave the description for the SHAP abbreviation in the Introduction, so there is no need for that here.

Sect. 3.6
Line 274, please give the abbreviations after the descriptions i.e., "... namely true positives, TP, false positives, FP, etc. ...."

Throughout the text: please be consistent and use the (correct) wording flux density (instead of flux).

Author Response

Comments 1: “Abstract: Please provide the full name of LightGBM.”
Response 1: Thank you for pointing this out. We agree. At its first occurrence in the Abstract, LightGBM is now written as “Light Gradient Boosting Machine (LightGBM).” This change can be found in the Abstract, Page 1, Paragraph 1.
Comments 2: “Introduction: Please provide references for LightGBM, TabNet, out-of-fold stacking.”
Response 2: Thank you for pointing this out. We agree. We added the foundational references for LightGBM, TabNet, and stacked generalization at their first occurrence in the final paragraph of the Introduction. We also added the original stacked-generalization paper to the reference list. These changes can be found in Section 1, Page 2, final paragraph; and in the References, Page 20, References 24--26.
Comments 3: “The meaning of ‘macro-F1’ also requires a brief explanation.”
Response 3: Thank you for pointing this out. We agree. We now explain that macro-$F_1$ is the arithmetic mean of the class-specific $F_1$ scores and therefore gives equal weight to the BZB, BZQ, and BZG classes despite their unequal sample sizes. This change can be found in Section 1, Page 2, final paragraph.
Comments 4: “Section 2: Please be more specific about the radio flux density measurements. According to Equation (1), it seems that there are flux density measurements at most at two frequencies, 1.4 and 5 GHz. Then, after the equation, a third frequency, 0.843 GHz is mentioned. Please clarify what is given in the catalog and whether 0.843 GHz flux densities were used.”
Response 4: Thank you for pointing this out. We agree that the previous notation obscured the catalog structure. We now state that 5BZCAT provides one combined low-frequency field: it contains a 1.4 GHz flux density from NVSS or FIRST when available, and otherwise a 0.843 GHz flux density from SUMSS. The input CSV does not include a per-source frequency or survey flag. The pipeline uses every non-missing value in this combined field without frequency rescaling, so the 0.843 GHz substitutions are included in the analysis. We changed the feature notation from $f_{\rm radio,1.4GHz}$ to $f_{\rm radio,1.4/0.843GHz}$ and updated Table 2 accordingly. We also clarify that the nominal 5 GHz field is mainly based on 4.85 GHz measurements from GB6 or PMN. These changes can be found in Section 2.2, Page 3, Paragraph 1 and Equation (1); and in Table 2, Page 4.
Comments 5: “After Equation (1), please give separately the meaning of $f_{\rm radio,1.4GHz}$ and $f_{\rm radio,5GHz}$. For example, ‘$f_{\rm radio,1.4GHz}$ and $f_{\rm radio,5GHz}$ give the radio flux density at 1.4 GHz and 5 GHz, respectively.’”
Response 5: Thank you for pointing this out. We agree. Immediately after Equation (1), we now define $f_{\rm radio,1.4/0.843GHz}$ as the radio flux density stored in the combined low-frequency catalog column and $f_{\rm radio,5GHz}$ as the nominal 5 GHz radio flux density. The following sentences describe the survey origin and use of each field separately. These changes can be found in Section 2.2, Page 3, Paragraph 1, immediately after Equation (1).
Comments 6: “Equation (5): Please give which magnitude-to-Jansky scale is used, with a reference, and if possible explain why that one is used.”
Response 6: Thank you for pointing this out. We agree. We now identify the adopted conversion as the AB magnitude-to-flux-density scale with zero point $F_{\nu,0}=3631\,\mathrm{Jy}$ and cite Oke and Gunn (1983). We use this scale as a single transparent and reproducible conversion because the input CSV combines heterogeneous 5BZCAT optical magnitudes into one Rmag field without a per-source filter and zero-point flag. We also state explicitly that 5BZCAT values may originate from USNO-B1 $R$, SDSS $r$, or another available band; therefore, the resulting optical flux density and formula-derived spectral indices are approximate fallback quantities rather than precision-calibrated photometry. These changes can be found in Section 2.3, Page 4, Paragraph 2 and Equation (5); Section 2.3, Page 5, Paragraph 1; and in the References, Page 20, Reference 28.
Comments 7: “Section 3.2, line 198: You already gave the description for the SHAP abbreviation in the Introduction, so there is no need for that here.”
Response 7: Thank you for pointing this out. We agree. We removed the repeated expansion of SHAP from Section 3.2 while retaining the methodological statement about tree-based SHAP feature attributions and its supporting reference. This change can be found in Section 3.2, Page 6, final paragraph.
Comments 8: “Section 3.6, line 274: Please give the abbreviations after the descriptions, i.e., ‘true positives, TP, false positives, FP,’ etc.”
Response 8: Thank you for pointing this out. We agree. The contingency terms are now introduced as true positives ($\mathrm{TP}$), false positives ($\mathrm{FP}$), false negatives ($\mathrm{FN}$), and true negatives ($\mathrm{TN}$), with the descriptions preceding the abbreviations. This change can be found in Section 3.6, Page 8, Paragraph 1.
Comments 9: “Throughout the text: Please be consistent and use the correct wording ‘flux density’ instead of ‘flux.’”
Response 9: Thank you for pointing this out. We agree. We checked the manuscript throughout and replaced “radio flux” with “radio flux density” wherever a monochromatic radio measurement is intended. We retain “X-ray flux” for the integrated 0.1--2.4 keV energy flux and “gamma-ray photon flux” for the 1--100 GeV quantity because these are integrated band measurements rather than spectral flux densities. The corresponding feature labels in Figures 5--6 were also updated. These changes can be found in Section 1, Page 2, Paragraph 2; Section 2.2, Page 3, Paragraph 1; Table 2 and Section 2.3, Page 4; Section 3.2, Page 6, final paragraph; Section 4.4, Page 13, Paragraph 1; Tables 9--10 and Figures 5--6, Pages 14--15; and Section 6, Page 16, Paragraph 1.

Author Response File: Author Response.pdf