Cross-State Validation of a Tool Supporting Implementation of Rural Kinship Navigator Programs

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Summary.  The article reviews the adapted use of a Kinship Navigator Program Needs Assessment Tool in rural settings. It uses qualitative focus group responses to identify phenomenological trends and compares this against a sample baseline for scale reliability.  The methodology is, on the whole, clear and replicable. It validates the need for assessment tools generally in kinship care programs.

General concept comments.  The authors set the context for the importance of their study, including the scale of children in the US in kinship care, the improved outcomes to be had from kinship care versus non-kinship care and the challenges faced by kinship carers.  References to challenges and benefits of kinship care are dated (2002, 2014, etc); more up to date references are offered below.

-          The Child Welfare Information Gateway (2021) Family engagement: Partnering with families to improve child welfare outcomes. USA: Children’s Bureau

-          Wellard, S., Meaking, S., Farmer, E. and Hunt, J. (2018). ‘Growing up in Kinship Care’,

-          Report for the Grandparents’ Association.

-          Family for Every Child (2019). The Paradox of Kinship Care.

Data sources. Diagrams could be usefully deployed to help map data sources and spread across the three states. This should include key descriptors such as kin type, formal or informal care, homeowners or renters, etc. to enable comparison of data.

Body of the article. The authors should reconsider how they’ve organized the content of the piece. For example, organisation of the materials and methods section could be improved, particularly with regard to clearer signposting on data sources being discussed. Section 2.2 Study Design, the discussion starts with the Key Informant focus groups, then moves on to the baseline reliability analysis, and then goes back to the focus group discussion. 2.3 Measures starts with the analysis of the Family Needs Scale, but it is not clear if this is in relation to the baseline or focus groups, and then specifically examines the focus group protocol.  2.4 Data Analysis only discusses only the focus group.  There is missing information as to how many evaluation team members reviewed transcripts.  There is an assumption that missing information on the coding of responses should confirm face validity. Authors should include how many values had missing information and whether this is significant.  There should be clearer analysis on why the assumption made is a reasonable one.

Authors should list the 7 items of the 46 in the tool that were discussed for clarification by multiple focus groups.  Section 3.2.1 Broad Validity should include clearer signposting and organisation. This structure should be reflected in item 3.2.2 Defining Concepts.  Sections 3.2.2 Defining Concepts offers anecdotal quotes give an idea of what required further clarification, however this section’s organisation could be improved to be clear why certain items are being discussed and others not.

Specific comments.

Language. Harmonization of terms and clarification of the population of interest is needed, along with justification for your choice. The paper refers to ‘caregivers’, ‘kinship caregivers’, and ‘key informants’ and ‘fictive kinship carers’ interchangeably or alongside each other. This is confusing and one term should be used to describe participants. If there is a distinction between types of participants, this should be made clearer with definitions offered.

Line 74. Comparison population missing – e.g. one in four kinship caregivers has less than a high school diploma. What is the equivalent in the general US population?

Line 414. This line suggests that participants may always connect these instructions when responding to items. Should this be may NOT always connect these instructions when responding to items.

Comments on the Quality of English Language

No comments on the quality of language

Author Response

Comment 1:  The authors set the context for the importance of their study, including the scale of children in the US in kinship care, the improved outcomes to be had from kinship care versus non-kinship care and the challenges faced by kinship carers.  References to challenges and benefits of kinship care are dated (2002, 2014, etc); more up-to-date references are offered below. The Child Welfare Information Gateway (2021) Family engagement: Partnering with families to improve child welfare outcomes. USA: Children’s Bureau; Wellard, S., Meaking, S., Farmer, E. and Hunt, J. (2018). ‘Growing up in Kinship Care’, Report for the Grandparents’ Association, Family for Every Child (2019). The Paradox of Kinship Care.

Response 1: Thank you for sharing recent references. We have updated our older references in the introduction section with some of the newer references suggested here. We plan to retain some references, even though they are older, given that they are primary sources of empirical knowledge.

Comment 2: Data sources. Diagrams could be usefully deployed to help map data sources and spread across the three states. This should include key descriptors such as kin type, formal or informal care, homeowners or renters, etc. to enable comparison of data.

Response 2: Thank you for this suggestion, to keep the participant burden low, we did not collect extensive demographic data from focus group participants as they were often speaking to both past and present experiences as kinship caregivers potentially reflecting varied demographics.  We added additional demographic data when available for the focus group participants in Table 1 including age of child(ren) at time Kinship care began and gender of child(ren) as well as comparable demographics available for the reliability analysis in Table 2 including, Kinship caregiver and child(ren) age, at the time of completing the Need Assessment, child(ren) gender, relation to kin, and race/ethnicity of the caregiver.

Comment 3: Body of the article. The authors should reconsider how they’ve organized the content of the piece. For example, the organization of the materials and methods section could be improved, particularly with regard to clearer signposting on data sources being discussed. Section 2.2 Study Design, the discussion starts with the Key Informant focus groups, then moves on to the baseline reliability analysis, and then goes back to the focus group discussion. 

Response 3: Thank you for the flow suggestions. The organization of the materials and methods section has been revised by relocating information in Section 2.2 Study Design to have separate descriptions of the focus group, then the reliability survey collection. Additional clarification was provided throughout in the introduction, results, and discussion in addressing multiple reviewer comments as well.

Comment 4: 2.3 Measures starts with the analysis of the Family Needs Scale, but it is not clear if this is in relation to the baseline or focus groups, and then specifically examines the focus group protocol. 

Response 4: We have revised the following text to the methods section starting on line 234. Prior to commencing both the evaluation and focus group studies, a multi-state collaborative of rural kinship stakeholders reviewed and modified Family Needs Scale, building on prior adaptations [33]. The goal for further adapting the Family Needs Scale was to better reflect the experiences of rural kin caregivers. 

Comment 5: 2.4 Data Analysis only discusses the focus group.  

Response 5: We have modified the Data Analysis section of the manuscript to further address quantitative data analysis. The text starting on line 287 reads, “Further, quantitative data were reviewed, cleaned, and subjected to reliability analyses using SPSS version 29. Cronbach's alpha was employed to examine reliability using the scale reliability analysis for 41 kinship caregivers from the KNP Evaluation intake survey across the three participating states.” 

Comment 6: There is missing information as to how many evaluation team members reviewed transcripts.  

Response 6: Thank you for the opportunity to clarify the methodology regarding the focus groups. We have updated the methods to reflect that 5 investigators reviewed transcripts. The text now reads, “Data were explored by five members of study team (i.e., BR, AF, DW, JC, JJ) using content analysis to categorize responses and identify themes explaining item validity [37]. Five evaluation team members familiarized themselves with transcripts and videos. They then rated each participant’s response as support, neutral, or disagreement on each domain or item clarity, understandability, and relevance and discussed to come to consensus.”

Comment 7: There is an assumption that missing information on the coding of responses should confirm face validity. Authors should include how many values had missing information and whether this is significant.  There should be clearer analysis on why the assumption made is a reasonable one.

Response 7: We appreciate the opportunity to clarify the number of items without comment and to provide additional detail regarding that informed the belief that lack of comment reflected face-validity. The following text is presented on lines 280 and lines 283 to address these two reviewer concerns. “While participants were asked about and responded to items in each domain, some individual items had no coded remarks as participants indicated the domain was clear, understandable, or relevant and did not express additional concern about those items. Thus, these items were assumed to confirm face validity without the need for updates.”

Comment 8: Authors should list the 7 items of the 46 in the tool that were discussed for clarification by multiple focus groups.  The text now explicitly identifies related items in each 3.2. And .3 subsection on lines 351-353 and 417-419. First, ”Seven items were discussed related to clarity or shared understanding, with four resources discussed in multiple focus groups including “the ability to complete home repairs or improvements”, “ability to do things myself”, “babysitting for your kin child(ren)”, and “payment for special needs of your kin child(ren)”. Then, “Three resources were discussed in multiple focus groups, with items including “a job that will support my household”, “ability to transport your kin child(ren) to places, including appointments”, and the “ability to plan for a future job for your kin child(ren)”.

Response 8: We agree that it would be helpful to list the 7 items of the 46 in the tool that were discussed for clarification by multiple focus groups. These items are now listed in the results section on line 313. The text now reads, “

Comment 9: Section 3.2.1 Broad Validity should include clearer signposting and organization. This structure should be reflected in item 3.2.2 Defining Concepts.  Sections 3.2.2 Defining Concepts offers anecdotal quotes to give an idea of what required further clarification, however this section’s organization could be improved to be clear why certain items are being discussed and others not.

Response 9: Signposts have more clearly been included or moved into the first paragraph of 3.2 lines 312-321. This now reads “Four themes were identified from the kinship caregivers' discussions and personal considerations on clarity, understandability, and relevance of items. The first theme highlights broad agreement of the tool’s validity to kinship caregivers like themselves. A total of seven items (15% of total) were discussed for minor clarification or relevance by multiple focus groups and 14 additional items were mentioned once across the four focus groups. Themes from these discussions are highlighted in themes 2 through 4  below. These themes included 1. broad validity of the items, 2. defining concepts within some items, 3. considering ages and stages of the kinship family when responding to items, and 4. considering the perspective of the child vs the caregiver when responding to an item.” Further clarifications were added to the beginning of each 3.2 section in addition to previous improvements made in responding to reviewer comments.

Specific comments.

Comment 10: Language. Harmonization of terms and clarification of the population of interest is needed, along with justification for your choice. The paper refers to ‘caregivers’, ‘kinship caregivers’, and ‘key informants’ and ‘fictive kinship carers’ interchangeably or alongside each other. This is confusing and one term should be used to describe participants. If there is a distinction between types of participants, this should be made clearer with definitions offered.

Response 10: We improved consistency to say kinship caregiver throughout. Key informants was kept in the materials and methods section as kinship caregivers were the target audience where as key informant refers to the recruitment strategy (line 157). Fictive was also kept with kinship caregivers as this is a specific descriptor. Additional detail was added to clarify this too though in line 32.

Comment 11: Line 74. Comparison population missing – e.g. one in four kinship caregivers has less than a high school diploma. What is the equivalent in the general US population?

Response 11: Thank you for this request for clarification. We ended up specifying education and included both strengths and disparities related to variables in the study cited to provide a more rounded and specific description of their findings (Line 76-79). The text now reads, “Higher rates of working may be related to unexpected financial needs [18, 20] and difficulties with employment [21] broadly experienced by kinship caregivers across the U.S. compared to their parental caregiver counterparts.”

Comment 12: Line 414. This line suggests that participants may always connect these instructions when responding to items. Should this be may NOT always connect these instructions when responding to items.

Response 12: Line 457 was updated and clarified. It now reads, “findings indicate participants may not always feel confident in their assessment of both themselves and their kinship children at the same time when responding to items.”

Reviewer 2 Report

Comments and Suggestions for Authors

Overall, your article is well-structured, clearly presented and well argued. However, the first part in particular lacks a certain flow which makes reading harder. Introducing more connecting words would help (indeed, therefore, thus, additionally,...). 

In terms of content and ethical intuition, it is very good and transparent.

 

Comments on the Quality of English Language

Generally, the level of English is excellent. There are, though, at some point repetitions. On line 72, 73 and 81,82  "are also" and "also"; on line 93,94 "must be"; on line 101, 102 "increased"; and on line 492-493 "suggestions" are repeated within the space of one or two sentences.  You also use 'Kinship parenting' and "Kin parenting" at the start of two following sentences (line 56).This interrupts the flow of reading. Using an alternative or synonym would make the text more fluent.

A few minor questions that relate to the choice of specific words:

1. On line 81 you use the word "even" ("even their less rural parts").  What do you mean by that? If it is needed, what are you trying to emphasize?

2. On line 107, 108, you write "already meeting these standarts". Which standards you are referring to?

3. On line 116 you write "As such a validated needs assessment tool is critical to the success of KNP's."  Why the use of 'As such'? Is it needed? 

4. On line 119 you write "preschooler families who were of low income."  This seems to lack a qualifying term like 'low income group' or 'low income background'.   

5. On line 138 you speak of a "rich understanding". What do you mean by 'rich'? In a similar vein on line 151 you mention "depth of experience". Do you mean experiential knowledge or something else?

One other remark for consideration: the article is based in, and written from, a US perspective and thus with a cultural and linguistic intuition proper to that. This is right and acceptable. However, words like "experience less education" (74-75) sounds strange and a little pejorative or demeaning to an international public.  It also limits the concept of 'education' to what is formally recognized as such. "Have lower educational levels" might be a better alternative. Writing "did not pursue higher levels of education" is a more positive way of communicating the same.

Author Response

Overall, your article is well-structured, clearly presented and well-argued. However, the first part in particular lacks a certain flow which makes reading harder. 

Response: Thank you! We used your feedback as well as the other reviewer to update and clarify flow throughout.

Comment 1: Introducing more connecting words would help (indeed, therefore, thus, additionally,...). 

Response 1: Additional connector statements have been added throughout the manuscript including “importantly,” added as a connector for for line 37,  “in particular” was added as a connector for line 49 and 137, “in addition” was added to lines 52, 81, “furthermore” was added to line 467 and 503, and “for example” was added to lines 276, 331. 

 

In terms of content and ethical intuition, it is very good and transparent.

Generally, the level of English is excellent. 

Comment 2: There are, though, at some point repetitions. On line 72, 73 and 81,82  "are also" and "also"; on line 93,94 "must be"; on line 101, 102 "increased"; and on line 492-493 "suggestions" are repeated within the space of one or two sentences.  

You also use 'Kinship parenting' and "Kin parenting" at the start of two following sentences (line 56).This interrupts the flow of reading. Using an alternative or synonym would make the text more fluent.

Response 2a: Thanks for identifying these repetitions, the following changes have been made, Line 92 changed to ‘In addition, rural residents‘, Line 102 changed to ‘need to coordinate across state’, line 110 changed to ‘increased access to services by kinship caregivers, greater caregiver well-being, more permanency,’, and line 548-49 changed to ‘any items identified for removal and they felt their examples and counter-examples’

Response 2b: Thanks for identifying this inconsistency, we replaced and highlighted throughout to ensure all are kinship or kinship caregiver for consistency.

A few minor questions that relate to the choice of specific words:

Comment 3: 1. On line 81 you use the word "even" ("even their less rural parts").  What do you mean by that? If it is needed, what are you trying to emphasize?

Response 3: We clarified the wording by specifying FAR categories connection. All FAR categories are rural and have some degree of difficulty accessing higher-order goods, but higher FAR means more remoteness as well as more difficulty in accessing lower-order goods as well.

USDA ERS - Frontier and Remote Area Codes

Comment 4: 2. On line 107, 108, you write "already meeting these standarts". Which standards you are referring to?

Response 4: This has been clarified to use more consistent wording. Standards referred to Clearinghouse criteria for kinship caregiver programs. Language was updated on line 116-118 to “no KNP programs meeting any level of Clearinghouse criteria for evidence-based evaluation and promising dissemination…”

Comment 5: 3. On line 116 you write "As such a validated needs assessment tool is critical to the success of KNP's."  Why the use of 'As such'? Is it needed? 

Response 5: As such has been removed and the language in that state was further clarified to connect to the paragraph introduction sentence. The text now reads “A needs assessment tool that proves accurate for this population”

Comment 6: 4. On line 119 you write "preschooler families who were of low income."  This seems to lack a qualifying term like 'low income group' or 'low income background'.   

Response 6: Thanks for this comment, to keep person-first language, the text has been clarified and revised to “preschooler families with low household incomes.”

Comment 7: 5. On line 138 you speak of a "rich understanding". What do you mean by 'rich'? In a similar vein on line 151 you mention "depth of experience". Do you mean experiential knowledge or something else?

Response 7:  Thanks for seeking this clarification, these words are sometimes used to describe the purpose and process of qualitative research. We have updated to clarify suggested terms in line 149-150, “employed to gain experiential knowledge of kinship caregivers and increase understanding”.

Comment 8: One other remark for consideration: the article is based in, and written from, a US perspective and thus with a cultural and linguistic intuition proper to that. This is right and acceptable. However, words like "experience less education" (74-75) sounds strange and a little pejorative or demeaning to an international public.  It also limits the concept of 'education' to what is formally recognized as such. "Have lower educational levels" might be a better alternative. Writing "did not pursue higher levels of education" is a more positive way of communicating the same.

Response 8: Thank you, we revised this paragraph to be more carefully worded and further clarified the entire paragraph. The text now reads, starting on line 67 “In addition to navigating a new parenting environment, kinship caregivers face unique stressors potentially impacting their own well-being. Research has consistently identified parenting stress as negatively impacting kinship caregivers psychosocial quality of life [16]. While a wide range of kinship types step in to provide care for children, importantly, a substantial portion of these kinship caregivers are grandparents. Together, they raised approximately 1 million children in the U.S. in 2022 [17]. As grandparent kinship caregivers are often older, taking on the role of primary caregiver and associated stressors, they are more likely to have declines to their own physical and mental health care needs [18]. These grandparents were also more likely to report working full-time, being married, and completing at least some college [19]. Higher rates of working may be related to unexpected financial needs [18, 20] and difficulties with employment [21] broadly experienced by kinship caregivers across the U.S. compared to their parental caregiver counterparts. Additionally, grandparents from minority racial and ethnic groups are more likely to be in a kinship caregiver role, potentially compounding intersectionality in accessing resources and disparate health outcomes [22]. Although this body of research will grow as Kinship Navigator Evaluations are published, limited research currently explores kinship caregiver well-being beyond grandparent kinship caregivers.”