Extended Foster Care Practice and Program Reform: Perspectives of Workers and Community Partners

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Review of Social Science 4193059 

Extended Foster Care Practice and Program Reform: Perspectives of workers and community partners  

 

The paper uses qualitative focus group and interview data from child welfare workers and community organization workers who work with the Extended Foster Care program in Washington State to identify program and practice themes. The qualitative inquiry was conducted as part of a larger system assessment of the Extended Foster Care program. Main themes were identified using rapid qualitative analysis and presented with illustrative quotes. The study includes literature review to contextualize the findings and recommendations.  

Overall, the study has the potential to make a meaningful contribution to the body of research on the service needs and outcomes of youth aging out of foster care. The data and methods are appropriate. Recommendations for the authors to improve the manuscript are to clarify the guiding research questions or aims, tailor the background literature review more tightly to the present study, provide additional information about the execution and limitations of the qualitative analysis, and enhance the structure of the findings and discussion to distinguish key takeaways for the reader.  Specific major and minor comments are detailed below. 

Thank you to the authors for their work seeking to improve services for youth aging out of foster care. 

 

Major comments  

The present study, 53-65: Were there any specific research questions or research aims that guided the qualitative data collection and analysis? (In addition to the overall goal of the systems assessment).  

Background: This section describes the literature on EFC programs but is missing a description of how the program operates in Washington. The introduction references when the program started and the number of youth served and their living arrangements but not any information about the scope or scale of services offered. A brief description of the Washington EFC program operations (from the overall system needs assessment) would help contextualize the qualitative findings. 

Background, throughout: Recommend reorganizing this section for more logical flow and tighter connection to the presented study. For example, present “EFC Programs and Practice” with “Effectiveness of EFC” (currently separated by the discussion of emerging adulthood). Also, the “Current Context for Young Adults” is missing a connection back to the target population of youth aging out of foster care.  

Data analysis, 194-206. Please clarify how the coding and thematic analysis was conducted across the three staff as it relates to inter-rater reliability and member checking. For example, if the transcripts were divided among the coders, what portion where coded by more than one coder and how were instances of conflicting codes resolved by the group.  

Findings, throughout: This section would be strengthened with the addition of a summary description for each of the three main themes before the list of individual findings/subthemes.  

Limitations, 592-608: The limitations section addresses the overall systems assessment and the sample but omits mention of the specific limitations of the rapid qualitative analysis method and any steps that were taken to ameliorate those limitations. For example, rapid qualitative analysis can be limited by reduced nuance, shallow theme development, reduced opportunity for inter-rater reliability and member checking, and uncertainty if data saturation has been truly achieved. The authors mention that the rapid method was used to facilitate developing findings in time for a legislative hearing (which is an important deadline for ensuring policy relevance). Was any additional post-hearing review or analysis conducted as the study was being prepared for manuscript submission?  

 

Minor comments 

Abstract, 3-15: Consider naming the state in the abstract. Consider revising the abstract to include why/how the study makes a specific important contribution.  

Background, throughout: This section uses inconsistent subheadings (e.g., underline and italics, within paragraph and spaced above).  

Background, 79-85. The end of this paragraph makes several definitive statements about the EFC literature and findings without supporting references. The single outcome study referenced in this paragraph is the 2021 CalYOUTH study. Recommend adding additional supporting references and/or revising language.   

Findings, throughout: The purpose of the underlined text is not clear. Is it being used as a heading or to provide emphasis? Or to identify specific subthemes? More consistent formatting could help clarify the author’s intent and the structure of the findings presented under each of the three primary themes. 

Findings, 216. The paper states “The main themes spotlighted in this paper are:” Were there other emergent that themes in the analysis that are not presented and, if so, what is the rationale for the focus on these three themes? It may be that I am misinterpreting, and the “spotlighted” language does not mean to imply only a subset of themes were selected for this paper.  

Discussion: This section would benefit from additional organization, such as presenting the recommended practice and policy reforms organized by the three primary themes.  

Limitations, 600-602. Since the study doesn’t claim to be a systematic literature review, it is not necessarily a limitation that one was not conducted. That statement could be omitted.  

 

Author Response

Social Sciences Special Issue

Title: Extended Foster Care Practice and Program Reform: Perspectives of Workers and Community Partners

Reviewer 1

Major comments  

The present study, 53-65: Were there any specific research questions or research aims that guided the qualitative data collection and analysis? (In addition to the overall goal of the systems assessment).  

We appreciate the opportunity to specify the research questions which guided the qualitative analysis reported on in the present study. We have noted these on lines 94-105.

Background: This section describes the literature on EFC programs but is missing a description of how the program operates in Washington. The introduction references when the program started and the number of youth served and their living arrangements but not any information about the scope or scale of services offered. A brief description of the Washington EFC program operations (from the overall system needs assessment) would help contextualize the qualitative findings. 

We have added description of the EFC program in our state, including details regarding the services and program operations.  58-72; 80-88

Background, throughout: Recommend reorganizing this section for more logical flow and tighter connection to the presented study. For example, present “EFC Programs and Practice” with “Effectiveness of EFC” (currently separated by the discussion of emerging adulthood). Also, the “Current Context for Young Adults” is missing a connection back to the target population of youth aging out of foster care.  

Thank you for this helpful suggestion. We have grouped content more logically in the background section as recommended and we have made edits throughout the manuscript to more consistently connect content to the present study and the target population.

Data analysis, 194-206. Please clarify how the coding and thematic analysis was conducted across the three staff as it relates to inter-rater reliability and member checking. For example, if the transcripts were divided among the coders, what portion where coded by more than one coder and how were instances of conflicting codes resolved by the group.  

We have augmented discussion of our data analysis methods. Specifically, we have described our transcript review process, how we resolved conflicting codes as a research team, and we have added specifics about our member checking process.  268-293

Findings, throughout: This section would be strengthened with the addition of a summary description for each of the three main themes before the list of individual findings/subthemes.  

We have added a summary description for each of the main themes prior to listing individual findings. 353-363; 441-451; 574-581

Limitations, 592-608: The limitations section addresses the overall systems assessment and the sample but omits mention of the specific limitations of the rapid qualitative analysis method and any steps that were taken to ameliorate those limitations. For example, rapid qualitative analysis can be limited by reduced nuance, shallow theme development, reduced opportunity for inter-rater reliability and member checking, and uncertainty if data saturation has been truly achieved. The authors mention that the rapid method was used to facilitate developing findings in time for a legislative hearing (which is an important deadline for ensuring policy relevance). Was any additional post-hearing review or analysis conducted as the study was being prepared for manuscript submission?  

Thank you for pointing out this important oversight. We have revised our limitations section to specifically address limitations with regard to our rapid qualitative analysis methods, including limitations noted by Reviewer 1. Our study team did not conduct additional analysis after the legislative session, however we did have opportunities to discuss the findings and recommendations with stakeholders and re-affirm conclusions from the study. 775-783

Minor comments 

Abstract, 3-15: Consider naming the state in the abstract. Consider revising the abstract to include why/how the study makes a specific important contribution.  

We have named the state in our abstract and augmented the justification for the study while staying within the journal’s 200 word maximum. 7-10

Background, throughout: This section uses inconsistent subheadings (e.g., underline and italics, within paragraph and spaced above).  

Thank you for noting this – we have revised the manuscript to use consistent subheadings. 

Background, 79-85. The end of this paragraph makes several definitive statements about the EFC literature and findings without supporting references. The single outcome study referenced in this paragraph is the 2021 CalYOUTH study. Recommend adding additional supporting references and/or revising language.  

We have revised this section and added supporting references. 127-134 

Findings, throughout: The purpose of the underlined text is not clear. Is it being used as a heading or to provide emphasis? Or to identify specific subthemes? More consistent formatting could help clarify the author’s intent and the structure of the findings presented under each of the three primary themes. 

We have removed the underlining in the findings section. We had indeed intended to use that to provide emphasis but had not done so consistently.

Findings, 216. The paper states “The main themes spotlighted in this paper are:” Were there other emergent that themes in the analysis that are not presented and, if so, what is the rationale for the focus on these three themes? It may be that I am misinterpreting, and the “spotlighted” language does not mean to imply only a subset of themes were selected for this paper.  

We appreciate this question. Our use of the term ‘spotlight’ was not intended to imply that only a subset of themes were selected for this paper. We have revised the language to avoid that misinterpretation in the future. 

Discussion: This section would benefit from additional organization, such as presenting the recommended practice and policy reforms organized by the three primary themes.  

We have revised our discussion section with an eye to improved organization, including better linking the discussion to the key themes. 

Limitations, 600-602. Since the study doesn’t claim to be a systematic literature review, it is not necessarily a limitation that one was not conducted. That statement could be omitted.  

We have omitted our mention of that as a limitation.

Reviewer 2 Report

Comments and Suggestions for Authors

This manuscript presents findings from the qualitative component of a state-level systems assessment of an extended foster care program. The topic is important and relevant, particularly given increasing attention to supporting youth transitioning out of care. This study has the potential to contribute valuable insights into service gaps; however, several areas require further clarification to enhance the rigor and clarity. The followings provide comments by the sections.

 

Introduction/Background

While the manuscript clearly elaborates the overall significance of the topic, the introduction/background section would benefit from providing more foundational information, especially for international readers. Specifically, the author(s) should elaborate on the U.S. federal and Washington State child welfare policy context, including clearer explanations of key programs and terms such as SIL, BRS, and RLSP. In addition, the author(s) need to provide a clearer definition of “extended foster care,” including whether it primarily refers to family-based placements, residential care, or a combination of both. The relationship and distinctions between extended foster care and independent living programs should also be clarified.

The research questions of this study are not explicitly stated, although this manuscript has broadly identified the aims to examine program effectiveness. The author(s) may need to identify specific research questions or evaluation criteria guiding the study more clearly.

 

Methods

The manuscript reports on the qualitative portion of a larger systems assessment; however, additional detail is needed to contextualize this component. (1) The author(s) should provide a clearer description of the broader assessment in order to situate this study within the larger project. (2) The rationale for including both state child welfare agency staff and community partners as participants should be further elaborated. Clarification of their roles and responsibilities would help readers understand how their perspectives contribute to addressing the research aims. (3) More information is needed regarding sampling strategies and the representativeness of the sample. The manuscript currently lacks sufficient demographic and background information about participants. (4) The use of both focus groups and individual interviews requires clearer justification. The author(s) should explain the rationale for employing these two methods and discuss whether and how differences between them may affect data quality and validity. The choice to conduct data collection via Zoom conferencing versus in-person formats should be addressed, particularly regarding potential implications for data richness and credibility. (5) The study does not appear to follow a formal program evaluation design. Therefore, the author(s) should more explicitly describe the strategies used to ensure methodological rigor, including credibility, trustworthiness, and analytic procedures.

 

Findings

The presentation of findings would benefit from clearer organization and stronger analytical structure. Although three overarching themes are identified, the current narrative feels somewhat diffuse, making it difficult for readers to get the key insights. The author(s) may need to further organize the findings by introducing sub-themes and sharpening the conceptual distinctions within each theme. For example, within the theme of “organizational practice reforms,” much of the content appears to focus on individual-level practices and practitioner–youth relationships, whereas only later paragraphs (e.g., training, evidence-based practices, caseload size) address more directly on organizational-level issues. Greater alignment within themes would improve analytic coherence.

A minor suggestion regards manuscript format. The author(s) may want to be sure about a consistent formatting of participant quotations (e.g., consistent use of italics) to enhance readability.

 

Discussion

The discussion identifies relational permanency as an important finding; however, this concept is not sufficiently developed. The author(s) could elaborate on how and why relational permanency emerges as a key issue within extended foster care, and to engage more deeply with relevant literature. In particular, the manuscript would benefit from a clearer articulation of what makes extended foster care distinct from independent living programs, and how this distinction shapes relational outcomes.

The discussion also notes the importance of incorporating youth and young adults’ participation in services. It might be more important to gather their perspectives in evaluating program effectiveness. However, such perspectives are absent from the current study design. If the author(s) consider this an important dimension, it should be explicitly acknowledged as a limitation.

 

Abstract

The abstract currently lists key themes without clearly presenting the substantive findings of the study. The author(s) are suggested to revise the abstract to include more concrete and descriptive statements of the main findings in complete sentences.

Author Response

Social Sciences Special Issue

Title: Extended Foster Care Practice and Program Reform: Perspectives of Workers and Community Partners

Reviewer 2                                                                 Comments and Suggestions for Authors

This manuscript presents findings from the qualitative component of a state-level systems assessment of an extended foster care program. The topic is important and relevant, particularly given increasing attention to supporting youth transitioning out of care. This study has the potential to contribute valuable insights into service gaps; however, several areas require further clarification to enhance the rigor and clarity. The followings provide comments by the sections.

Introduction/Background

While the manuscript clearly elaborates the overall significance of the topic, the introduction/background section would benefit from providing more foundational information, especially for international readers. Specifically, the author(s) should elaborate on the U.S. federal and Washington State child welfare policy context, including clearer explanations of key programs and terms such as SIL, BRS, and RLSP. In addition, the author(s) need to provide a clearer definition of “extended foster care,” including whether it primarily refers to family-based placements, residential care, or a combination of both. The relationship and distinctions between extended foster care and independent living programs should also be clarified.

Thank you for the suggestion to provide additional foundational information for readers less familiar with the EFC program. We have revised our introduction / background to elaborate on the relevant policy context at the state and federal level. We have also more clearly defined our terms and described the programs to which we refer, including ILPs. 41-46; 58-72; 80-88

The research questions of this study are not explicitly stated, although this manuscript has broadly identified the aims to examine program effectiveness. The author(s) may need to identify specific research questions or evaluation criteria guiding the study more clearly.

We have specified the research questions guiding the present study.  94-105

Methods

The manuscript reports on the qualitative portion of a larger systems assessment; however, additional detail is needed to contextualize this component. (1) The author(s) should provide a clearer description of the broader assessment in order to situate this study within the larger project.

We have revised our manuscript to more clearly describe the larger systems assessment and to better situate the present study within that larger study. 94-114

(2) The rationale for including both state child welfare agency staff and community partners as participants should be further elaborated. Clarification of their roles and responsibilities would help readers understand how their perspectives contribute to addressing the research aims.

We have clarified the rationale for including state CW staff and community partners as participants in the qualitative study, including better explaining how their perspectives helped address the specific research questions guiding the qualitative study. 250-267

(3) More information is needed regarding sampling strategies and the representativeness of the sample. The manuscript currently lacks sufficient demographic and background information about participants.

We have revised our methods section to provide greater detail regarding sampling strategies and information about the sample. We clarified that we did not collect demographic information about participants. We have also clarified that we cannot make claims about the representativeness of the sample however, we note that the focus groups with state agency staff included all regions, so the effort to seek regional representation was achieved. In addition, our focus group with EFC leads included all but one of the regional leads in the state.  250-267

(4) The use of both focus groups and individual interviews requires clearer justification. The author(s) should explain the rationale for employing these two methods and discuss whether and how differences between them may affect data quality and validity.

We appreciate this question and have revised our manuscript to more clearly justify our choice of conducting both focus groups and individual interviews. We provide some discussion about how the differences in these two methods might have affected the data quality and validity. We also note this as a potential limitation of the design and study.   268-281

The choice to conduct data collection via Zoom conferencing versus in-person formats should be addressed, particularly regarding potential implications for data richness and credibility.

We have added discussion in our methods section regarding our choice to conduct online data collection rather than doing so in person. We have also addressed the potential implications for the quality and credibility of the data collected.  283-293

(5) The study does not appear to follow a formal program evaluation design. Therefore, the author(s) should more explicitly describe the strategies used to ensure methodological rigor, including credibility, trustworthiness, and analytic procedures.

We appreciate the opportunity to clarify the methods used in the present study and how we sought to ensure rigor, credibility, trustworthiness and in quality of our analytic procedures. We have augmented our description of our methods, collaborative processes, analytic steps in coding, reconciling differences, member-checking, and stakeholder engagement processes.    322-335

Findings

The presentation of findings would benefit from clearer organization and stronger analytical structure. Although three overarching themes are identified, the current narrative feels somewhat diffuse, making it difficult for readers to get the key insights. The author(s) may need to further organize the findings by introducing sub-themes and sharpening the conceptual distinctions within each theme. For example, within the theme of “organizational practice reforms,” much of the content appears to focus on individual-level practices and practitioner–youth relationships, whereas only later paragraphs (e.g., training, evidence-based practices, caseload size) address more directly on organizational-level issues. Greater alignment within themes would improve analytic coherence.

Thank you for this observation and suggestion. We have re-organized our findings section to help the reader better follow the key points. We have added a summary of the findings for each theme and identified important subthemes. We have also made clearer distinctions between organizational-level and individual-level practice and issues. We have better explained the organizational-level aspects of the suggested reforms.    

A minor suggestion regards manuscript format. The author(s) may want to be sure about a consistent formatting of participant quotations (e.g., consistent use of italics) to enhance readability.

We have addressed the issue that our formatting was inconsistent throughout our manuscript, including with regard to the use of italics for participant quotations. We now use italics and indenting consistently.   

Discussion

The discussion identifies relational permanency as an important finding; however, this concept is not sufficiently developed. The author(s) could elaborate on how and why relational permanency emerges as a key issue within extended foster care, and to engage more deeply with relevant literature. In particular, the manuscript would benefit from a clearer articulation of what makes extended foster care distinct from independent living programs, and how this distinction shapes relational outcomes.

We appreciate this important point. We have elaborated on our discussion of relational permanency in the context of EFC and how EFC represents an opportunity to improve relational outcomes.  668-686

The discussion also notes the importance of incorporating youth and young adults’ participation in services. It might be more important to gather their perspectives in evaluating program effectiveness. However, such perspectives are absent from the current study design. If the author(s) consider this an important dimension, it should be explicitly acknowledged as a limitation.

In our discussion section, our point was to underscore the importance of child welfare organizations “…considering youth and young adults when designing and articulating their practice approach…” – that is, that state agencies should take a developmental perspective and recognize and act on the need for their practice model to be tailored to the transition to adulthood developmental stage. This is, of course, a challenge in a bureaucracy that is historically focused on young children birth to 5. -- We have revised our discussion to make our intended point clearer. At the same time, our manuscript does hope to emphasize the importance of engaging youth and young adults as lived experts in different aspects of child welfare practice, policy, and in evaluation activities. We have acknowledged as a limitation that the current study focused on workforce and community partners and did not involve data collection with participants in EFC. 739-745

Abstract

The abstract currently lists key themes without clearly presenting the substantive findings of the study. The author(s) are suggested to revise the abstract to include more concrete and descriptive statements of the main findings in complete sentences.

We have revised the abstract to add substantive and specific findings in each of the key theme areas in complete sentences. 12-23