The Development of the “Checklist for Life Skills Educational Assessment” (CLSEA)

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper presents the development and validation of a new instrument, the Checklist for Life Skills Educational Assessment (CLSEA), designed to assess life skills in students with developmental disabilities aged 7 to 18. The authors highlight the importance of life skills for students with developmental disabilities and the need for reliable assessment tools to support the development of individualized education plans. The study provides a detailed account of the checklist's development, validation, and potential applications. The paper is well-structured, with clear sections on methodology, results, and discussion, and it addresses a critical gap in inclusive education research.

Strengths:

• The study addresses a critical need for reliable and valid assessment tools in the area of life skills for students with developmental disabilities.
• The development of the CLSEA is described in a clear and comprehensive way. The methodology features the development process of the checklist, involving a systematic literature review, expert input, pilot testing, and validation through exploratory factor analysis.
• The results are presented in a detailed and organized manner, with tables summarizing the factor analysis for each section of the checklist. The reliability and validity of the instrument are well-supported by statistical evidence.
• The checklist has practical applications for teachers and professionals, enabling them to assess and monitor life skills development and design targeted interventions.
• The checklist adopts a strengths-based approach and covers important domains of life skills, including daily living skills, personal-social skills, and prevocational-vocational skills.

Weaknesses:

• The sample is limited to students with developmental disabilities in Greece, which limits the generalizability of the findings to other populations and cultural contexts.
• The study acknowledges that the CLSEA was not adapted for students with sensory and/or motor disabilities, which raises concerns about its inclusivity.
• The authors note that some items, such as those related to sex education, did not load on any factor, and they suggest further investigation is needed.
• The checklist relies solely on teacher assessments, which may introduce subjective bias. The absence of input from students, parents, or other stakeholders limits the comprehensiveness of the assessment.
• The study does not include longitudinal data to demonstrate the checklist's effectiveness in tracking life skills development over time.
• The discussion could be strengthened by providing a more in-depth exploration of the implications of the findings for educational practice and future research.

 

Scope for Improvement of the article:

Introduction and background

The introduction provides a good overview of the importance of life skills and the need for assessment tools. It also references relevant literature. However, the context could be strengthened by a more critical analysis of existing tools and a clearer articulation of the specific gap the CLSEA aims to fill. For example, the authors mention various tools like the Competency Rating Scale (CRS) and the Life Skills Inventory, but they could provide a more in-depth comparison of these tools' strengths and weaknesses in relation to the CLSEA.

Further analysis suggestions:

Within the scope of the available data, here are some ways the study could be improved:

• Deeper Exploration of Factor Analysis Results: The authors could provide a more in-depth discussion of the factor analysis results. For instance, they could offer more detailed interpretations of each factor, discuss any unexpected factor loadings, or compare their findings to previous research in this area.
• Analysis of Item Performance: While the study reports on the factors and their reliability, it could be enhanced by including an analysis of individual item performance. This could involve examining item means, standard deviations, and item-total correlations to identify any items that may not be functioning optimally.
• Examination of Subscale Correlations: The authors could explore the correlations between the different subscales of the CLSEA (i.e., daily living skills, personal-social skills, and prevocational-vocational skills). This would provide insights into the relationships between these different domains of life skills.
• Further Analysis of Missing Data: The study mentions that "don't know" and "not applicable" responses were treated as missing data. The authors could provide a more detailed analysis of the extent and pattern of missing data and discuss the potential implications of missing data for the results.
• Additional Reliability Analyses: While Cronbach's alpha is reported, other measures of reliability, such as test-retest reliability or inter-rater reliability, could be explored to provide a more comprehensive assessment of the CLSEA's reliability. However, this would likely require additional data collection.
• Detailed Demographic Analysis: The authors present a table on the demographic characteristics of the teachers and students. A more detailed analysis of how these demographics might relate to the life skills of students could be conducted. For example, is there a significant difference in life skills between primary and secondary school students?

Discussion & conclusion

The presentation of the factor analysis results is clear, and the authors provide interpretations of the identified factors. However, the discussion could be more compelling by:

• Providing a more in-depth discussion of the implications of these factors for educational practice.
• Acknowledge the limitations of the study more explicitly in the discussion section and propose strategies to overcome them in future research.
• Offering a more nuanced discussion of why certain items did not load onto any factors (e.g., sex education items).

• Provide practical recommendations for implementing the checklist in schools, such as training modules for teachers or integrating the tool into existing assessment frameworks.

• The conclusions are generally supported by the results, but some claims could be strengthened. For instance, the authors conclude that the CLSEA can be a valuable tool for program planning and individual support. While the study provides evidence for the reliability and validity of the CLSEA, further discussion on how the checklist practically translates into effective interventions would add more support to this conclusion.

Literature scope and quality:

• The paper cites a substantial body of literature to support its claims, including references to previous studies on life skills education and assessment tools. The validation analyses are conducted using established statistical methods, and the results are presented transparently. However, some claims, such as the effectiveness of life skills training in improving adult outcomes, are based on general literature and may benefit from more specific empirical evidence.

Conclusion

The paper makes a valuable contribution to the field of inclusive education by developing a tool to assess life skills in students with developmental disabilities. However, the analysis could be improved by further data analysis (outlined above) and the scope and applicability of the paper could be enhanced by discussing implementation strategies and addressing limitations explicitly. Some of the literature is a little general too and could benefit from support from empirical studies. Addressing these issues would strengthen the paper's impact and ensure the checklist's broader utility in inclusive education settings.

 

Future directions - scope for Improvement of the study/CLSEA outwith the scope of the current article

1. Expand the Participant Pool:

• Include a more diverse sample of participants, such as teachers from different countries, students with sensory and motor disabilities, and other stakeholders like parents and psychologists. For example, adapting the checklist for students with visual or hearing impairments could enhance its inclusivity. The authors should consider how the CLSEA can be adapted or expanded to be more inclusive of students with sensory and/or motor disabilities. For example, they could explore the use of alternative response formats or assessment methods that do not rely on specific sensory or motor skills.

• Cultural Adaptations: To enhance the generalizability of the CLSEA, the authors should conduct cross-cultural studies and adapt the instrument for use in different cultural contexts. This could involve translating and adapting the items, as well as considering cultural differences in the understanding and importance of life skills.

2. Address Missing Life Skills:

• Further Investigation of Specific Skills: The authors should delve deeper into the finding that sex education items did not load on any factor. They could conduct further research to explore the reasons for this and to develop more effective ways to assess these critical skills. For instance, adding questions about recognizing personal boundaries or expressing opinions could enrich the tool.
• Integration of Multiple Data Sources: The authors could emphasize the importance of using the CLSEA in conjunction with other assessment methods, such as observations and interviews, to obtain a more comprehensive picture of students' life skills.
• Longitudinal Studies: Future research could examine the predictive validity of the CLSEA by following students over time and assessing the relationship between their life skills and their outcomes in adulthood.

Comments for author File: Comments.pdf

Author Response

 "Please see the attachment."

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

The work is suitable for publication.

However, in our opinion there are two points that could be improved [a] updating the bibliographic references and

[b] clarifying more the text with some bibliographic documentation as per see. ''Moreover, this study could be expanded to include a larger population of SWDs. The de- [line] 392 velopment of CLSEA is based solely on teachers’ assessments of Greek students with de- [line]393 velopmental disabilities. In order to be further utilized, certain adaptations are needed to [line] 394 include students with hearing, visual, and/or physical impairments/disabilities as well as 395 immigrant and refugee students. [line] 396''

Author Response

Reviewer's comment: The work is suitable for publication. However, in our opinion there are two points that could be improved [a] updating the bibliographic references and [b] clarifying more the text with some bibliographic documentation as per see. 

Response: Thank you very much for your comments and your suggestions. We took into consideration your suggestion and we have updated our bibliographic references as well added some more bibliographic documentation [Added refs: Mazzotti et al. (2016), Tsakanikos et al. (2011), Powers et al. (2018), Sitlington (2008)]

We have also added in the first paragraph of our introduction two new empirical studies to validate our claims, Kim & Dymond (2020) and Yazici & Stancer (2020). Accordingly, we modified the references in the fourth paragraph of the introduction by adding a new systematic review, Mazzotti et al. (2016) along with a reference used above, Kim & Dymond (2020).