Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript addresses an important and timely topic, namely the need for greater methodological standardization in agent-based modeling (ABM), particularly in the context of social sciences. The paper provides a comprehensive synthesis of the literature through a scoping review and presents a structured framework to support decision-making in the definition of model parameters, probability distributions, and sensitivity analysis strategies. The use of multiple analytical approaches (heatmaps, network analysis, and Sankey diagrams) is a strong aspect of the study and contributes to the clarity of the presented findings.
Overall, the manuscript is well structured, clearly written, and offers a valuable contribution by making implicit methodological patterns in the literature more explicit. However, several aspects could be further strengthened to improve the analytical depth and practical applicability of the study.
1. The proposed decision-oriented framework represents a central contribution of the paper, but its validation remains limited. It would significantly strengthen the manuscript to include a more explicit validation or demonstration of the framework.
2. While the scoping review is well designed and follows established guidelines, additional clarification regarding the study selection and data extraction process would improve transparency and reproducibility. In particular, more details on inclusion/exclusion decisions and potential sources of bias would be beneficial.
3. The manuscript identifies a structural misalignment between model objectives and methodological rigor, which is an important insight. However, this argument could be further deepened by providing more concrete examples or quantitative evidence illustrating how this misalignment affects model outcomes or interpretation.
4. The discussion of sensitivity analysis methods is relevant, but it could be strengthened by a more explicit comparison between local, global, and advanced approaches, including clearer guidance on when each method is most appropriate in practice.
5. The internal logic linking model objectives, parameterization strategies, and sensitivity analysis is well presented; however, the framework could benefit from a more formalized or operational description to facilitate its direct application by researchers.
6. Some figures (e.g., network visualizations and Sankey diagrams) are informative but relatively dense. Improving readability through clearer labeling and simplified presentation would enhance accessibility for a broader audience.
7. Although the manuscript is conceptually strong, it would benefit from a clearer positioning with respect to existing guidelines and protocols. Explicitly highlighting similarities, differences, and added value would strengthen the novelty of the contribution.
8. Minor editorial improvements could be considered to further streamline the text, particularly in sections where the discussion becomes descriptive.
In summary, the manuscript presents a relevant and well-structured contribution to the field of agent-based modeling. Addressing the points above would enhance its methodological rigor and practical impact.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The paper addresses a timely and worthwhile question — how the ABM community in the social sciences handles parameterization and sensitivity analysis — and the analytical pipeline from frequency distributions through cross-tabulated heatmaps, a Sankey flow, and a methodological network is a sensible structure for a scoping review. However, several reporting and methodological gaps raise my concerns.

1. The authors cite PRISMA-ScR, yet the manuscript provides neither a PRISMA flow diagram nor per-stage record counts (identified / deduplicated / title–abstract screened / full-text assessed / included / excluded with reasons). The pivotal figure of "304 articles" appears for the first time only in §3.4, with no audit trail leading up to it. For a scoping review that explicitly follows the PRISMA-ScR methodology, this is a non-negotiable requirement that should be added.
2. The authors describe two reviewers screening independently with a third arbitrator, but no reliability statistic appears anywhere in the paper — no Cohen's κ, not even a simple percent agreement — at either the title–abstract screening stage or the MO/DS/SA coding stage. Because every downstream result of the paper rests on these category counts, the absence of any reliability evidence makes it difficult to take the figures at face value.
3. Scoping reviews conventionally do not demand a formal risk-of-bias assessment, but the explicit ambition of this paper is to propose standards — a normative claim. When a paper sets out to prescribe norms for a field, it should offer at least some baseline judgment about the reliability of the evidence it builds on; otherwise the normative weight of the recommendation will be open to challenge.
4. The illustrative case in §5 — reference [55] — is a paper by the corresponding author's group, and is in fact the same item as reference [4] in the introduction (Rossi et al., Complexities, 2026, urban agriculture). I recommend the authors (a) explicitly disclose the self-citation; (b) avoid grounding the demonstration of the framework solely in their own prior work, since doing so makes the "framework works in practice" argument circular; and (c) merge [4] and [55] in the bibliography.
5. Network analysis methodology needs more detail. Two points in particular:

• The modularity score Q = 0.1755 is presented as evidence of "weak community structure", but the paper offers no comparison baseline and no significance test against a null model (e.g., configuration-model rewiring). A single Q value in isolation cannot sustain a qualitative "weakly modular" claim.
• The keyword analysis is based on "authors' keywords", and the paper mentions "1,133 harmonized keywords" only once before moving directly to a filtered 83-node network. The specific harmonization rules — singular/plural unification, hyphenation, abbreviation-vs-full-form merging (e.g., "sensitivity analysis" vs "SA"), the synonym list — should be reported, otherwise the network is not reproducible.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Summary
The manuscript addresses the lack of standardization in agent-based modeling (ABM), focusing on parameter definition and sensitivity analysis. Based on a scoping review and quantitative analyses, the authors identify recurring methodological patterns and propose a decision-oriented framework linking model objectives, data sources, and sensitivity analysis methods.

General assessment
The topic is relevant and the manuscript is clearly written. The literature synthesis is useful and highlights important issues, particularly the mismatch between model objectives and methodological rigor. In my view, the overall contribution remains largely descriptive, and the proposed framework is not sufficiently operational to constitute a robust methodological guideline.

Major comments

1. Descriptive rather than operational contribution
   The proposed framework is mainly conceptual and lacks explicit decision rules or practical criteria that would make it directly applicable.
2. Insufficient treatment of sensitivity analysis
   While limitations of current practices are correctly identified, the manuscript does not provide clear guidance on the use of more advanced methods (e.g., global sensitivity analysis or surrogate modeling), nor does it discuss their applicability or trade-offs.
3. Limited methodological positioning
   The manuscript would benefit from engaging with related modeling traditions in other domains, where issues such as parameterization, sensitivity, and validation have been addressed in a more systematic and theoretically grounded manner. While direct transfer may not be possible, these approaches could provide useful methodological benchmarks.
4. Parameters treated as independent
   The discussion overlooks potential structural constraints among parameters (e.g., consistency relations or underlying model structure), which are relevant in many complex systems.
5. Weak notion of validation
   Validation is mainly framed in terms of robustness, without considering reproduction of known behaviors or theoretical consistency.
6. Lack of practical validation
   The framework is not illustrated through a concrete case study.
7. Supplementary materials and reproducibility
   The availability of computational notebooks is appreciated and supports reproducibility of the analyses. However, it would be helpful to clarify to what extent these materials validate the proposed framework, beyond reproducing the descriptive results presented in the manuscript. In particular, a notebook illustrating how the framework guides methodological decisions in a concrete modeling example would significantly strengthen the contribution.

Recommendation
Major Revision

Suggestions for improvement

• Make the framework operational (explicit rules or criteria).
• Strengthen the discussion of sensitivity analysis methods.
• Improve methodological positioning by connecting with broader modeling traditions.
• Clarify parameter constraints and validation criteria.
• Include an illustrative case study.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

It is an interesting quite well-written manuscript that examines methodologies the scholars adopt in their ABM research. I have two major concerns regarding this manuscript.

First, I suppose the authors should prove the accuracy of their classification methodology for model objectives, data parametrization, and sensitivity analysis.

Next, the paper examines a long time span of 2015 and 2025. I believe the simple averaging employed in this research is not an accurate representation of the real world.

I guess, there should be substantial changes in methodologies during this period. For instance, a trivial guess is that the last several years witness a sharp trend of using AI-based calibration approaches, though this was not the case 10 years ago.

As a minor concern, I suppose the authors should be slightly more careful in statistical reporting of their results (indicate pvalues, etc.). 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The revision addresses most Round-1 points. Three issues should still be resolved.

1. The illustrative case grounded in the authors' own prior work was removed, which is appropriate. However, ref [4] (Rossi, Carvalho & Sarti 2026, Complexities) is by the corresponding author's group and is still cited in the introduction at [3–5] without disclosure. Please either flag the self-citation in-text or replace it with an independent reference.
   
   
2. §3.3 reports 1,147 raw → 1,133 harmonized keywords (1.2% reduction). §4.4 then states "1,203 harmonized keywords and 3,839 edges in the full network." The difference 1,133 vs 1,203 is not reconciled. Please state explicitly what each figure refers to (e.g. unique vocabulary vs total tokens vs pre-filter network nodes).
   
   
3. v1 reported Q=0.1755 and described the network as showing "weak community structure." v2 now reports Q=0.2208 and, on the basis of the configuration-model null (z=-8.22), interprets the same result as a "highly integrated conceptual landscape." Two concerns: (i) the Q value moved from 0.1755 to 0.2208 between versions with no stated reason — is this due to the new harmonization protocol, a different filtering threshold, or a different community-detection resolution? (ii) "Integrated conceptual landscape" overclaims relative to what the z-score actually supports. A more conservative reading such as "hub-dominated, low-modularity topology" would be preferable and avoid tension with the paper's parallel argument about structural misalignment.
   
   

Author Response

Please see attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

The authors have addressed all of my comments. The paper is in better shape.

Author Response

Please see attachment.

Author Response File: Author Response.pdf