Preliminary Psychometric Evaluation of the Inventory of Statements About Self-Injury (ISAS) in a Greek Adolescent Inpatient Psychiatric Sample
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsI am grateful for the opportunity to review the manuscript “Psychometric Validation of the Inventory of Statements About Self-Injury (ISAS) in a Greek Adolescent Inpatient Psychiatric Sample”, submitted to the Primary and Hospital Care journal.
Below is my review, in which I assess the scientific quality, methodology, results, and the manuscript’s relevance to the journal’s scope.
The manuscript presents a preliminary psychometric validation of the Inventory of Statements About Self-Injury (ISAS) in a sample of Greek adolescent psychiatric inpatients, contributing as the first study in this cultural and population context characterized by a high prevalence of non-suicidal self-injury (NSSI). The findings support a bifactorial structure (intrapersonal functions are more strongly endorsed and correlated with suicidal ideation via the YSR), with satisfactory internal consistency at the higher level (adequate α values after excluding weak subscales) and moderate factor/convergent validity. Tables are clear, statistical methods are appropriate (EFA/CFA/MICE), and conclusions are cautious, aligned with the literature on the functional distinction of NSSI.
However, there are some points that require revision to enhance the manuscript’s robustness.
1) The sample was predominantly female and small (as inferred from the low descriptive statistics in Table 1), with weak subscales excluded (“Autonomy” and “Interpersonal Boundaries” showed α < 0.70 and were excluded from the preferred model). I recommend reporting the exact N, statistical power (post-hoc for EFA/CFA), and tests of invariance by gender/age.
2) The medical record data (“retrospective data collection”) underestimate unreported NSSI. I recommend discussing bias clearly and directly (e.g., recall bias in inpatients), proposing a future prospective study, and including 95% CI in all estimates.
3) The bifactorial model (CFI=0.89, RMSEA=0.07) is acceptable but not optimal (“fit indices… only moderate”). I recommend testing the bifactorial vs. unifactorial model using modification indices (MI); report SRMR and χ²/df. Justify the exclusion of items without reported EFA loadings.
4) Positive correlations with YSR (intrapersonal r > 0.30 with self-harm/ideation), but no construct discrimination (e.g., absence of divergent validity). I recommend adding convergent/divergent validity with NSSI-specific scales (e.g., SIQ for suicidal ideation) and a complete correlation table.
5) In the discussion, the authors could expand on the clinical implications.
Author Response
We sincerely thank the reviewer for the careful and constructive evaluation of our manuscript and for recognizing the clinical relevance of examining the ISAS in a Greek adolescent inpatient psychiatric sample. We are grateful for the positive comments regarding the study’s contribution, the use of EFA/CFA and multiple imputation, the clarity of the tables, and the cautious alignment of the conclusions with the international literature. In response to the reviewer’s helpful suggestions, we have revised the manuscript to strengthen reporting of sample size and power considerations, clarify the limitations of retrospective clinical-record data, expand confidence-interval reporting, provide fuller CFA and modification-index information, add a complete validity correlation table, and expand the clinical implications.
Comment 1: The sample was predominantly female and small; weak subscales were excluded. Report exact N, power, and invariance by gender/age.
Response: We thank the reviewer for this important comment. We have clarified the exact analytic sample size and subgroup composition throughout the manuscript. The final analytic sample consisted of N = 95 adolescents, including 82 females and 13 males. We have also added sample-size and power considerations for the factor-analytic analyses. Specifically, for the EFA, we report the participant-to-indicator ratio and an approximate sensitivity analysis showing that N = 95 provides 80% power to detect correlations of approximately |r| = 0.28 at α = 0.05, indicating adequate power for moderate but not small factor loadings. For the CFA, we have revised the interpretation substantially. Because of the small sample and because EFA and CFA were conducted in the same dataset, the CFA results are now presented as exploratory and illustrative rather than confirmatory. Measurement invariance by gender and age was considered but not tested because the available subgroups were too small and uneven for reliable multi-group CFA. In particular, the male subgroup included only 13 participants, and age-based subdivision would have produced small groups within a narrow adolescent age range. Accordingly, no claims are made regarding measurement equivalence of the ISAS across gender or age groups.
Comment 2: Retrospective medical record data underestimate unreported NSSI. Discuss bias, propose prospective study, include 95% CI in all estimates.
Response: We thank the reviewer for this helpful comment. We have revised the manuscript to address the limitations of retrospective clinical-record data more directly. Specifically, we now state that the study could only capture NSSI that was disclosed by adolescents and recorded during routine inpatient assessment, and that undisclosed or undocumented NSSI may have led to under-ascertainment. We also expanded the limitations section to discuss recall bias in acutely admitted psychiatric inpatients, including possible effects of distress, shame, fear of consequences, dissociation, depressive symptoms, limited insight, and variable clinical documentation. We further clarified that the descriptive NSSI characteristics should be interpreted as clinical-record-based estimates rather than prevalence estimates. In addition, we added a future research recommendation for a prospective validation study using standardized ISAS administration, structured clinical assessment of NSSI, repeated assessment during hospitalization or follow-up, and systematic recording of missing or non-disclosed data. Finally, we revised the reporting of results so that 95% confidence intervals are included for the primary descriptive, reliability, factor-analytic, CFA, and validity estimates.
Comment 3: The bifactorial model is acceptable but not optimal. Test bifactorial vs. unifactorial model using modification indices; report SRMR and χ²/df. Justify exclusion of items without reported EFA loadings.
Response: We thank the reviewer for this useful suggestion. We have revised the CFA reporting to compare the unifactorial and two-factor models more explicitly. To avoid ambiguity, we now refer to the model as a “two-factor” or “correlated two-factor” model rather than a true bifactor model, because no general factor plus specific factors was estimated. We also inspected modification indices for the two-factor model. Modification indices primarily suggested residual covariances among conceptually related interpersonal ISAS functions. We therefore tested a modification-index-informed two-factor model, allowing only theoretically defensible residual covariances between Peer Bonding and Marking Distress and between Interpersonal Influence and Peer Bonding. This improved model fit, but the model is now interpreted as exploratory and illustrative rather than confirmatory. We have also revised the CFA table to include SRMR and χ²/df for all CFA models. Finally, we added the EFA loadings for all ISAS functional subscales, including Autonomy and Interpersonal Boundaries. We clarified that these two subscales were not removed from the ISAS instrument. Rather, they were excluded only from a post-hoc exploratory CFA sensitivity model because of their weak internal consistency, unstable contribution to the CFA solution, and limited developmental/clinical interpretability in this adolescent inpatient sample.
Comment 4: Positive correlations with YSR, but no construct discrimination/divergent validity. Add convergent/divergent validity with NSSI-specific scales and complete correlation table.
Response: We thank the reviewer for this important comment. We agree that the previous version emphasized convergent validity but did not sufficiently address construct discrimination or divergent validity. We have therefore revised the manuscript to distinguish more clearly between convergent validity and discriminant-pattern evidence. Because the study was retrospective and based on routinely collected clinical records, no additional NSSI- or suicidality-specific validation measure, such as the Suicidal Ideation Questionnaire–Junior, was available. We have stated this explicitly as a limitation and have avoided claiming formal divergent validity. To address the reviewer’s request as fully as possible with the available data, we added a complete correlation table including the ISAS total functional score based on all 13 original functions, intrapersonal factor, and interpersonal factor in relation to all YSR syndrome scales, YSR broadband scales, and the YSR self-harm and suicidal ideation items. The revised Results section now notes that, among the a priori clinical criterion items, the intrapersonal factor was associated with self-harm behavior and suicidal ideation, and that it also showed associations with internalizing-related YSR dimensions and YSR Total Problems. In contrast, the original broad ISAS interpersonal score showed generally weaker associations, with only small associations observed for selected broader YSR dimensions. We interpret this as modest preliminary evidence of convergent validity and limited discriminant-pattern evidence, not as definitive divergent validity. We also added a recommendation that future prospective validation studies include NSSI- and suicidality-specific measures such as the SIQ, DSHI, SITBI, or NSSI-AT.
Comment 5: Expand clinical implications.
Response: We thank the reviewer for this suggestion. We have expanded the Discussion to clarify the clinical implications of using the ISAS in a Greek adolescent inpatient psychiatric sample. Specifically, we now discuss how the ISAS may support routine clinical assessment by helping clinicians move beyond the presence or frequency of NSSI toward understanding the function of self-injury. We also clarify the potential treatment implications of differentiating intrapersonal and interpersonal motives, including implications for emotion regulation work, suicide-risk assessment, interpersonal formulation, family work, safety planning, and discharge planning. We have also emphasized that the ISAS should not be used as a stand-alone diagnostic or risk-assessment tool, but rather as part of a broader multimethod clinical assessment.
Reviewer 2 Report
Comments and Suggestions for AuthorsThis is a timely and clinically relevant psychometric validation study of the Inventory of Statements About Self-Injury (ISAS) in an under-studied population: Greek adolescents in inpatient psychiatric care. The study addresses a genuine gap—no prior Greek-language validation exists for this high-risk clinical group—and uses appropriate methods (EFA + CFA, internal consistency, convergent validity with YSR). Strengths include real-world retrospective clinical data, transparent reporting of limitations, and pragmatic handling of missing data via multiple imputation. The core finding (broad support for a two-factor intrapersonal–interpersonal structure, with caveats for specific subscales) aligns with the international literature while highlighting developmental and clinical nuances in adolescents. However, the manuscript has several major shortcomings that currently limit its scientific rigor and publishability. The small sample (N=95), data-driven post-hoc modifications, only moderate CFA fit weaken the claims. The paper reads more like an exploratory clinical report than a definitive validation study. Below I detail the issues by priority.
1.The sample size (N = 95) is substantially below recommended minimums for factor analysis, particularly for CFA. Comrey and Lee (1992) suggest N > 300 as good, N = 100 as poor. While the authors acknowledge this limitation, the conduct of both EFA and CFA on the same small dataset is problematic and likely contributed to the mixed and modest fit indices. The authors should present the analysis as purely exploratory in this sample, with CFA results interpreted as illustrative rather than confirmatory. Alternatively, a split-sample approach (if feasible) or cross-validation in an independent sample is needed.
2. Using the same dataset for both EFA and CFA violates fundamental principles of cross-validation. The "revised two-factor model" was derived from EFA and then tested with CFA on the same data, inflating capitalisation on chance. Reframe the results such that the EFA-driven model is treated as a preliminary solution requiring independent replication. Remove language suggesting confirmatory support (e.g., "retained as the preferred working solution").
3. The final model fit indices are poor by conventional standards (CFI = 0.85, TLI = 0.79, RMSEA = 0.11). Standard psychometric thresholds typically require a CFI > 0.90 and an RMSEA < 0.08. given these subpar indicesA RMSEA > 0.10 indicates poor fit, not "moderate." The authors should more explicitly justify why this model is acceptable for clinical use .
4.EFA supported 2 factors (good).
Original 2-factor CFA fit was poor.
You then tested a modified 3-factor and a revised 2-factor excluding Autonomy and Interpersonal Boundaries (post-hoc, based on low α).
This is circular: subscales were dropped because of poor reliability, then the model was re-tested without them.
5. Generalizability Concerns. 86.3% female – severe gender imbalance limits generalisability. NSSI functions may differ by gender, yet no gender-based analyses are possible.
Single-site inpatient setting – results may not apply to outpatient or community adolescents.
6 Convergent Validity Evidence Is Weak. Correlations with YSR scales are small (r = 0.12–0.18 for most associations). While some are statistically significant, the magnitude suggests limited convergent validity.
7.Weak Reliability of Key Subscales. Autonomy (α=0.33) and Interpersonal Boundaries (α=0.42) are unacceptably low. You correctly exclude them, but this means the Greek ISAS no longer fully assesses the original 13 functions. Self-Care also shows a weak loading (0.28);
Potential recall bias, non-standardized administration, and selection bias (only records with complete questionnaires) are acknowledged but could be expanded. No information on inter-rater reliability of chart extraction.
7.Power and Generalizability. Heavily female (86%), single site, inpatient only. No power analysis for correlations or CFA.
Minor concerns
1.Retrospective Data from Clinical Records
Data were extracted from routine clinical files, not collected under standardised research protocols. This introduces variability in administration conditions, potential missing data, and possible recording biases.
2.Handling of Low-Performing Subscales (Autonomy, Interpersonal Boundaries)
Excluding these subscales post-hoc without theoretical justification is concerning. The authors suggest developmental reasons, but this remains speculative.
3.Mean age at onset is reported as 11.20 years (SD = 2.77), with a range from 1 to 15 years. An onset age of 1 year is implausible and suggests either data entry error or misunderstanding of the question.
Author Response
We thank the reviewer for the careful and constructive evaluation of our manuscript. We agree that, given the small sample size, the use of both EFA and CFA in the same dataset, and the post-hoc nature of some model modifications, the study should not be presented as a definitive validation study. We have therefore substantially revised the manuscript to frame the work as a preliminary and exploratory psychometric evaluation of the Greek ISAS in an adolescent inpatient psychiatric sample. We have toned down claims of confirmation, clarified the limitations of the CFA, added 95% confidence intervals, expanded the discussion of retrospective-data bias and generalizability, and emphasized the need for prospective replication in larger and independent samples.
Major comments
Comment 1: N = 95 is substantially below recommended minimums for factor analysis. EFA and CFA in same small dataset are problematic. Present as exploratory.
Response: We agree with the reviewer. We have revised the manuscript to state explicitly that the sample size is small for factor analysis, particularly CFA, and that the findings should be considered exploratory. We also added a sample-size and power paragraph indicating that the analyses were better powered to detect moderate than small associations. The CFA results are now described as illustrative model-based sensitivity analyses rather than confirmatory validation. We have also added Comrey and Lee’s sample-size recommendation to contextualize the limitation and state clearly that independent cross-validation is required.
Comment 2: Using same dataset for EFA and CFA violates cross-validation principles. Remove language suggesting confirmatory support, including “retained as preferred working solution.”
Response: We agree. We have removed language suggesting definitive confirmatory support, including the phrase “retained as the preferred working solution.” The EFA-derived and post-hoc modified CFA models are now described as provisional exploratory solutions requiring independent replication. We also explicitly state that conducting EFA and CFA in the same dataset increases the risk of capitalization on chance and prevents true cross-validation.
Comment 3: Final model fit indices are poor by conventional standards. RMSEA > 0.10 indicates poor fit, not moderate. Justify acceptability for clinical use.
Response: We thank the reviewer for this clarification. We have revised the interpretation of the CFA findings. The post-hoc 11-function two-factor sensitivity model with RMSEA = 0.11 is now described as showing inadequate or poor fit by conventional criteria. The MI-informed two-factor model showed improved fit on some indices, but we now describe it as exploratory and not definitive. We no longer claim that the model is established or sufficient for clinical use. Instead, we state that the ISAS may be useful only as an adjunctive clinical assessment tool, interpreted alongside clinical interview, suicide-risk assessment, family information, and longitudinal observation.
Comment 4: Dropping Autonomy and Interpersonal Boundaries and retesting is circular.
Response: We agree that excluding subscales post-hoc and retesting the model can introduce circularity and capitalize on chance. We have therefore reframed the 11-function model as a post-hoc exploratory sensitivity analysis, not as a revised validated version of the Greek ISAS. We now report all 13 original ISAS functions descriptively and clarify that Autonomy and Interpersonal Boundaries were not removed from the instrument. Their exclusion was limited to the exploratory CFA sensitivity model because they showed weak internal consistency and unstable model behavior. We also added EFA loadings for all functions so that the decision is transparent.
Comment 5: Generalizability concerns: 86.3% female, single-site inpatient setting, no gender analyses possible.
Response: We agree. We have expanded the limitations section to emphasize that the sample was predominantly female, single-site, and limited to adolescents receiving inpatient psychiatric care. We now state explicitly that the findings cannot be generalized to male adolescents, outpatient adolescents, community samples, or non-clinical populations. We also state that measurement invariance and gender-based analyses could not be conducted because the male subgroup included only 13 participants.
Comment 6: Convergent validity evidence is weak; correlations with YSR are small.
Response: We agree that the convergent validity evidence should be interpreted cautiously. We have revised the Results and Discussion to describe the validity evidence as modest and preliminary. Among the a priori clinical criterion items, the strongest associations were observed between the original broad ISAS intrapersonal score and YSR self-harm behavior and suicidal ideation. The intrapersonal factor also showed a moderate association with YSR Total Problems. Associations with most other broader YSR scales were small to moderate.
Comment 7a: Weak reliability of key subscales; Self-Care weak loading; Greek ISAS no longer fully assesses original 13 functions.
Response: We thank the reviewer for this important point. We have clarified that the Greek ISAS was not shortened or redefined. All 13 original ISAS functions are still reported and discussed. Autonomy and Interpersonal Boundaries were excluded only from a post-hoc exploratory CFA sensitivity model, not from the instrument itself. We also discuss the less distinct cross-loading pattern of Self-Care and state that its placement should be interpreted cautiously. The revised manuscript emphasizes that these findings indicate psychometric instability of some functions in this sample rather than evidence that these functions are clinically irrelevant.
Comment 7b: Potential recall bias, non-standardized administration, selection bias, and no inter-rater reliability for chart extraction.
Response: We agree. We have expanded the limitations section to discuss recall bias, non-standardized questionnaire administration, clinical-record documentation bias, and selection bias introduced by including only records with available questionnaire data. We also added that no formal inter-rater reliability assessment was conducted for chart extraction, which limits confidence in the consistency of retrospective data abstraction.
Comment 8: Power and Generalizability. Heavily female (86%), single site, inpatient only. No power analysis for correlations or CFA.
Response: We thank the reviewer for this important comment. We have added post-hoc/sensitivity power information for both the correlation analyses and the CFA. For the correlation analyses, with N = 95 and two-sided α = 0.05, the study had approximately 80% power to detect correlations of about |r| = 0.28. Thus, the study was powered to detect approximately moderate correlations but not small associations. This is now explicitly acknowledged when interpreting the convergent validity findings. For the CFA, we added an RMSEA-based sensitivity/power statement. Using N = 95, df = 41, α = 0.05, a close-fit null hypothesis of RMSEA = 0.05, and alternative RMSEA values of 0.08 and 0.10, power was limited for detecting modest model misfit. Specifically, power was approximately 0.36 for RMSEA = 0.08 and approximately 0.73 for RMSEA = 0.10. These results reinforce our revised interpretation that the CFA findings are exploratory and illustrative rather than confirmatory. We have also expanded the limitations section to emphasize that the predominantly female, single-site inpatient sample limits generalizability and prevents reliable gender-based analyses or measurement invariance testing.
Minor concerns
Comment 1: Retrospective data from clinical records introduce variability and recording bias.
Response: We agree and have expanded the Procedure and Limitations sections accordingly. The revised manuscript now states that the data reflect disclosed and documented NSSI rather than all NSSI occurring in the inpatient population, and that unreported or undocumented NSSI may have been missed.
Comment 2: Excluding low-performing subscales post-hoc without theoretical justification is concerning.
Response: We agree that the exclusion required clearer explanation. We now clarify that Autonomy and Interpersonal Boundaries were excluded only from an exploratory CFA sensitivity model and not from the instrument. We also added a table showing their EFA loadings, reliability estimates, and rationale for their post-hoc exclusion from the sensitivity model. The discussion now emphasizes that the developmental interpretation is tentative and requires replication.
Comment 3: Age at onset range includes 1 year, which is implausible.
Response: We thank the reviewer for identifying this issue. We rechecked the data and identified implausible values of 1 year for age at NSSI onset. These values were treated as data-quality errors and recoded as missing before recalculating descriptive statistics for age at onset. The revised manuscript now reports the corrected age at NSSI onset as M = 11.56 years, SD = 2.07, range = 5–15, with 95% CI = 11.12–12.00. We have also added a statement in the statistical analysis section indicating that implausible values were reviewed and treated as missing when appropriate.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThis is the second review of the manuscript “Preliminary Psychometric Evaluation of the Inventory of Statements About Self-Injury (ISAS) in a Greek Adolescent Inpatient Psychiatric Sample”. The new version of the manuscript demonstrates that the authors have satisfactorily addressed the main recommendations from the first round of review. The revised manuscript presents significant improvements over the previous version, particularly regarding methodological clarity, limitations, and the expansion of the clinical implications section.
The terminology has been corrected to reflect a two-factor correlated model. The modification indices were reviewed, and theoretically defensible residual covariances were allowed, with the model labeled as exploratory. SRMR and χ²/df were added to the CFA table. The EFA factor loadings for the Autonomy and Interpersonal Boundaries subscales were included, and their exclusion from the CFA sensitivity analysis was justified (weak internal consistency, unstable contribution, limited interpretability in adolescents).
A comprehensive table of correlations (Table 7) was included, showing the relationships between ISAS scores (total, intrapersonal, interpersonal) and all YSR scales, including items on suicidal ideation and self-harm. The authors made a clear distinction between evidence of convergent validity and evidence of discriminant validity. The absence of specific measures for NSSI (SIQ, DSHI, SITBI, NSSI-AT) was acknowledged as a limitation and included in recommendations for future research.
The discussion of limitations was expanded to include underreporting, recall bias, shame, fear of consequences, dissociation, depressive symptoms, and variable clinical documentation. 95% CIs were added to the descriptive, reliability, and factor estimates. The recommendation for a prospective validation study with standardized administration of the ISAS and structured clinical assessment of NSSI was incorporated.
The section on clinical limitations was expanded to address the role of the ISAS in routine assessment, the differentiation between intrapersonal and interpersonal motivations for treatment, and implications for emotional regulation, suicide risk assessment, interpersonal formulation, family work, safety planning, and hospital discharge. A caveat was also included stating that the ISAS should not be used as a standalone diagnostic or risk assessment tool.
